Code Watch: What would Map do?

By Larry O Brien

January 16, 2013 — (Page 1 of 2)
In my previous column, I discussed the power of Map/Reduce, both in terms of its common use as an engine for distributed analysis of Big Data and also as a fundamental concept, an insight into the power of the functional programming concepts of Map (which applies a function across a collection of elements) and Reduce (which summarizes or otherwise condenses a collection of elements).

As I've also discussed in my column, for two years Scala was my day-to-day programming language, and for much of that time I waited for the functional programming "light bulb" that would help me understand the zeal of functional programmers, who often seem unrealistically enthusiastic about their approach and offputtingly dismissive of any benefits to be had from object-oriented thinking.

I'm a stupid person. This is not news, but one of the ways my stupidity manifests is that it's hard for me to recognize—until they're no longer easily available—the best elements of a tool. It was only when I started programming in C++ that Smalltalk made sense to me. And it's only been recently, when I returned to the land of more mainstream languages, that I can sense a functional light bulb flickering on.

Many discussions of functional programming argue that the key is the lack of reassignment to variables after they've been initialized (also known as immutability, referential transparency, etc.). And that's certainly beneficial, but as I've argued in the past, the technique of preferring immutability to state is simply good programming and does not, I think, guide technical design or architecture. Admittedly, functional languages’ emphasis on programming without reassignment drives the developer toward separating control flow from data manipulation (and that's a benefit), but it’s not transformative. It also drives the programmer toward recursion over iteration, a choice that is greatly approved of in certain circles but has never struck me as a hugely important design decision.

But there are two aspects of functional programming that I find are dramatically underused in day-to-day object-oriented programming. One is lazy evaluation. This is the ability of a functional language to pause a calculation, return an intermediate value, and then continue the calculation at some time in the future. Most modern languages have a "yield" keyword, and if you combine a yield with a recursive function, you get lazy evaluation-like behavior, including the ability to define very large—even infinite—sequences, and yet work with them efficiently. For instance, with lazy evaluation, you can define an infinite search of a tree-like data structure, but then very easily limit the depth to which you actually search.

Related Search Term(s): Map/Reduce

Pages 1 2

Share this link: http://sdt.bz/37309

Technorati

Digg

Slashdot

Facebook

Friendfeed

Twitter

del.icio.us

Code Watch: Map/Reduce, turn by turn There’s a best way to do Map/Reduce; here’s a quick primer and suggested reading

Diving into the Big Data pool Hadoop is at the center of the big-data universe, but a handful of companies are providing alternatives

Aster Data expands SQL for map/reduce Company releases a thousand new query-building blocks so businesspeople can unlock map/reduce for unstructured data