Map and Filter

Contrarian view on map and filter.


A. Coady


March 31, 2018

Contrarian view on map and filter.

Although PEP 8 is silent on the topic, it’s become recommended in many Python circles to eschew map and filter in favor of generator expressions or list comprehensions. For example, this Stack Overflow question received and accepted a typical response. Ironically, that question misquoted the google style guide, which this author happens to agree with.

Use list comprehensions and for loops instead of filter and map when the function argument would have been an inlined lambda anyway. [emphasis added]

The style guide also shows a non-lambda version as a positive example: > map(math.sqrt, data) # Ok. No inlined lambda expression.

First, a brief history of how the Python community arrived at this state.

Python 1

Prior to version 2.0, Python had neither list comprehensions nor nested scopes. Therefore simple map and filter operations had to use a for... append loop, or lambda. But the lacked of nested scopes was inherently crippling to the latter approach.

x = 2
map(lambda y: y * x, range(5))
<map at 0x108e57250>

A NameError would have been raised on x, because it’s not defined in the inner scope. One clever work-around was to shadow default arguments.

x = 2
list(map(lambda y, x=x: y * x, range(5)))
[0, 2, 4, 6, 8]

Unsurprisingly, that was widely viewed as an ugly hack. Many resigned themselves to for... append loops instead.

Python 2

Then Python added list comprehensions in 2.0, and that became the one obvious way to do it.

x = 2
[y * x for y in range(5)]
[0, 2, 4, 6, 8]

Python acquired nested scopes in the next version, 2.1, but the damage was done. Functional programming in Python in general, and lambda in particular, was widely frowned upon. Even though the lack of nested scopes affected all inner functions used in any context; it was never really about lambda per se.

Python 3

It’s sometimes claimed to this day that map and filter only exist for backwards compatibility. But that belies the history of Python 3. map, filter, and reduce were all considered for removal. But only reduce was banished to the functools module. map and filter were not only retained, but updated to return iterators.

So it’s already dubious to claim that using a built-in is unapproved. But the real point is that map and filter remain a higher level abstraction. Sure, with lambda there are the same number of logical components, and it’s just a matter of syntactic sugar. But there is some abstraction value when the functions already have a name.

It’s also commonly pointed out that generator expressions are superior because they can do a map and filter simultaneously, but crucially only if the filter comes first. Consider this task: normalizing an iterable of strings.

values = 'sample ', ' '
list(filter(None, map(str.strip, values)))

Note list is only being used for printing, and should be ignored for the sake of comparisons.

As for the alternative, surely calling strip twice to use a single expression is just plain cheating. So really the only option is:

[value for value in (value.strip() for value in values) if value]

Some would consider nested comprehensions already enough to separate with a temporary name. But that’s inadvertently acknowledging how much more verbose it is.

Can it really be claimed that the latter is more readable than the former? It’s just boilerplate, which never seems acknowledged in small examples. But if one only has to double the size of the context to show how verbose comprehensions are, doesn’t that demonstrate the value of map and filter.


And now a shameless plug of the author’s placeholder package for readers who appreciate function-style programming. It provides syntactic sugar for lambda.

from placeholder import _

list(map(_ * 2, range(5)))
[0, 2, 4, 6, 8]

But even speaking as the author, map isn’t the best use case. Sort keys are a much better example, since there is no competing syntax.

min(['ab', 'ba'], key=_[-1])

Python 3.8’s new assignment expressions provide yet another alternative.