Can Python take advantage of MapReduce?

Reading Joel’s article yesterday, and then the MapReduce paper I was left wondering if Python could take advantage of this.

Python has a feet in functional programing, and indeed has several functional-style erm… functions like mapreducefilter etc. I’m a big fan of those little misunderstood beasties.

I, too, was lamenting the proposal to remove those function, and was not quite convinced by Guido’s argument.

Now, if one where to implement something similar to MapReduce in Python, it would be pretty trivial I believe with the functions.

Not that doing it with list comprehensions is impossible, would probably be a bit harder yes, but doable I guess.

Even a simpler version, that took advantage of multiple processors on a single machine could in theory speed up things quite a bit. However I wonder how big of a penalty the GIL (Global Interpreter Lock) would take on that.

If someone has ideas/opinions about this, I’m very interested in hearing. Leave your comments!