Blocks (aka lambda expressions or closures) are versatile and powerful. I like them. I have even implemented a programming language in which they play a central role.
However, for a number of common programming situations, I think that some alternative approaches based on higher order messaging provide better solutions than classic block-based approaches. Marcel Weiher provides a good example in Blocked-C. He shows some Objective-C code that uses a block to iterate over a collection of strings and to create a new collection with some suffix added to the original strings:
NSMutableArray *filteredItems = [NSMutableArray array];
[items enumerateObjects WithOptions:0 withBlock:
^(id item, NSUInteger index, BOOL *stop) {
[filteredItems addObject:[item stringByAppendingString:@"suffix"]];
}
];
Then, he goes on to show how the same can be achieved using a higher order messaging (HOM) technique:
[[items collect] stringByAppendingString:@"suffix"];
The conciseness and expressiveness of the HOM version is impressive and should at the very least warrant further examination. Regarding this, we are lucky since this approach, and its application to Objective-C, is described in an OOPSLA paper titled "Higher Order Messaging" (download).
With some support in the programming language, higher order messaging techniques can go even further. F-Script is a Smalltalk dialect for Cocoa which extends Smalltalk with array programming and messaging patterns. Messaging pattern is a generalization of message sending that applies to entire collections of objects. In a number of common situations, this provides an interesting alternative to the classic block-based approach. Indeed, here is our example coded in F-Script:
items stringByAppendingString:'suffix'.
A key aspect of this programming model (and of Weiher’s HOM) is that it liberates us from the classic "one object at a time" style, and lets us manipulate whole collections of objects at once, using higher level constructs. We are freed from the low level details of accessing and dealing with individual elements. In my experience, introducing array programming in object-oriented programming, as F-Script does, has positive effects similar to those of the introduction of vectors in math, which is quite nice… This model is described in another OOPSLA paper titled "OOPAL: Integrating Array Programming in Object-Oriented Programming" (download)
To get a better sense of this programming model, let’s see how it stacks up against classic block based approaches for collections manipulation as seen in Ruby and Smalltalk. For the same price, I also add Python’s list comprehensions and C# Linq to the mix. In some common situations, those are interesting alternatives to blocks that are still representative of the "one object at a time" approach.
In the example below, we have a list of songs (a collection of Song objects) and we want to get their names. Here is how we’d do it using various approaches:
- Smalltalk and Ruby with blocks
- Python with list comprehension
- C# with Linq
- F-Script with array programming
| Smalltalk | songs collect:[:aSong| aSong name] |
| Ruby | songs.collect(&:name) |
| Python | [aSong.name() for aSong in songs] |
| C# | from aSong in songs select aSong.name() |
| F-Script | songs name |
We can note that the array programming version (F-Script) is free from syntactic noise and low-level iteration logic, as is the brain of the programmer. Actually, in that particular situation, it seems hard to do better.
For the ruby version, we use a feature adopted in Ruby 1.9, which allows for automatic conversion of symbols to blocks. This is a useful trick as it provides a way, in some situations, to have a compact notation for some kind of blocks. Note, however, that when we need to provide more that just a method name, we have to use the classic Ruby block notation. This is the case in our original example (i.e., invoking the stringByAppendingString method with a given argument), for which we would have to do: items.collect {|item| item.stringByAppendingString("suffix")}.
We can carry on the comparison further by introducing some need for filtering. In the following example, we want to get the names of songs with a bit rate greater than or equal to 256 kb/s.
| Smalltalk | (songs select:[:aSong| aSong bitRate >= 256]) collect:[:aSong| aSong name] |
| Ruby | songs.select {|aSong| aSong.bitRate >= 256}.collect(&:name) |
| Python | [aSong.name() for aSong in songs if aSong.bitRate() >= 256] |
| C# | from aSong in songs select aSong.name() where aSong.bitRate() >= 256 |
| F-Script | songs name where: songs bitRate >= 256 |
We see that the F-Script programming model allows for conciseness and expressiveness. Note that, in F-Script, where: is not a keyword or a special construct. It is just a regular message selector. The where: method, defined in a category of NSArray, takes an array of booleans as argument and returns the elements of the receiver whose positional matching element in this boolean array is true. Such boolean array is exactly what is produced by the expression songs bitRate >= 256, as messages are automatically broadcasted to each individual elements of the receivers (i.e., the bitRate message is sent to each element of the songs array, producing an array of numbers whose elements are then sent the >= 256 message, producing the array of booleans). This is the powerful compression operation of array programming adapted to object-oriented programming.
Finally, here is an example of sorting a collection of objects. In this example, we want to create a sorted collection of our songs, sorting on the songs’ names. In the case of Python, since its list comprehensions don’t have specific support for sorting, we use a solution based on lambda expressions.
| Smalltalk | songs asSortedCollection:[:song1 :song2| song1 name < song2 name] |
| Ruby | songs.sort_by(&:name) |
| Python | sorted(songs, key=lambda aSong: aSong name) |
| C# | from aSong in songs orderby aSong.name() select aSong |
| F-Script | songs at: songs name sort |
As in the previous examples, the F-Script version shows that we can do sophisticated collection manipulation without relying on blocks but only using message passing. The result of the sort message sent to the array of song names is an array of integers containing the indices that will arrange the receiver of sort in ascending order. We then pass this array of indices to the at: method. The at: method allows for indexing an array by either a single integer or a whole array of indices. This is this latter capability, a fundamental element of array programming, that we use here to produce our sorted list of songs.
Conclusion
Blocks are great, especially in programming languages which don’t have (and don’t want) specific syntax for classic control structures (if/then/else, while, etc.). However, we should take them with a grain of salt and see that, for some common problems, there are alternative approaches that are worth considering.
Further reading
F-Script documentation set
Blocked-C II
Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs
Special bonus
You can easily, and interactively, experiment with the code provided in the examples: launch F-Script.app, connect to iTunes and immediately start querying and manipulating your iTunes song collection from F-Script.
You connect to iTunes using the standard Cocoa Scripting Bridge technology. For example, you can enter the following code in the F-Script console:
iTunes := SBApplication applicationWithBundleIdentifier:'com.apple.iTunes'
The iTunes variable now points to a dynamically generated Cocoa object that stands for the iTunes application.
You can get at your songs with:
songs := ((iTunes sources at:0) playlists at:1) tracks
You can then directly paste, in the console, the code of the examples provided in this article. If your song library is huge you might notice some latency, as Cocoa messages are automatically converted to Apple Events that flow back and forth between F-Script and iTunes.
In Perl, the collect idiom is:
my @names = map { $_->name } @songs;
and the select-then-collect idiom isn’t much harder:
my @names = map { $_->bitrate >= 256 ? $_->name : () } @songs;
Map is pretty handy in Perl.
Not half; array programming + Apple events is the most inefficient combination you could use. It’ll work, but it’ll send at least one event per track.
Array programming, NSPredicates, et-al are great when you’re iterating over local collections, but they’ll cost you inter-procces, particularly when they aren’t even the native idiom of the IPC mechanism being used.
In this case, you’re using Apple event IPC, which is based on RPC + first-class queries, not OOP as Cocoa is (and SB pretends – poorly – to be). The ‘correct’ (i.e. vastly more efficient) way to do it is to build a single query and fire that off for the application to resolve, like this:
Or, if you want it in Smalltalk-y syntax, using objc-appscript:
HTH
Has, you raise an interesting question regarding speed optimization, but your conclusions about array programming are incorrect.
By expressing code at a relatively high level, array programming excels at providing under the hood, transparent speed optimizations. This is a well studied property of array languages. There is nothing in array programming that requires actually sending one event per track. On the contrary, because the intent of the code is expressed in terms of manipulations of whole collections, the array programming system can apply very efficient optimizations.
For example, in this particular situation related to generating Apple Events, F-Script can automatically translate the expression
songs nameinto the following Objective-C code:[songs arrayByApplyingSelector:@selector(name)]. This code will lead to just one round trip between F-Script and iTunes. The same applies for the other messages such assongs bitRate. This is just one possible approach, other optimizations (e.g., generating the equivalent of the AppleScript code you showed) are possible.This optimization is not active by default, but you can activate it by recompiling F-Script after uncommenting the code related to SBElementArray in FSExecEngine.m and see that the queries get executed ultra quickly. The reason this is not on by default is a bug in the Scripting Bridge (it won’t be news for you that some parts of it are half broken!). But regardless, the important point here is that from a programming model perspective, array programming does not leads to inefficient execution but is on the contrary an enabler for optimizations impossible or hard to do in classic “one object at a time” models. You can find more about this in the OOPAL paper referenced in the blog post.
That being said, it is perfectly possible that, given the current state or even the whole approach of the Apple’s Scripting Bridge, appscript is a better Apple Event bridge. I’d love to see it support F-Script.
Well, okay; I’ll give you that. You could in principle abstract over the Apple event mechanism to make it look like array programming. That said, trying to dress up Apple events (RPC+queries) as something they’re not (OOP) is one of the reasons SB doesn’t (and can’t) work quite right. Aside from the inevitable impedance mismatch, there’s the practical problem of dealing with all of the different scriptable applications that are out there and all of the variations in how they do business.
For example, SB likes to dress up the ‘make’ event as Cocoa-style object instantiation – i.e. alloc+init, and add to a collection – but if you try using -[SBElementArray addObject:] with iTunes, it barfs because -addObject: makes certain assumptions about how the ‘make’ event’s parameters should be formed, which are correct for Cocoa Scripting-based apps but often wrong for Carbon ones.
FWIW, early versions of appscript also tried to impose heavier OO-like abstractions on Apple events, so I learned that lesson the hard way. Unlike SB, I had the luxury of being able to rewrite appscript into its current form, which is about as thin an abstraction as it can be without being unpleasant.
If AE behaviours had been rigorously specced and enforced from the get-go, today’s scriptable apps might not be so lamentably inconsistent in how they implement their APIs, and creating robust, reusable high-level abstractions over those APIs might then be relatively straightforward. But I really doubt anyone could abstract over the current mess without the resulting abstractions regularly springing leaks on unsuspecting users.
I love to see that too. FWIW, the Appscript framework’s AEM layer is deliberately language-agnostic, allowing other Cocoa-speaking languages to apply their own choice of high-level syntactic sugar on top – e.g. objc-appscript uses a static glue generator, while macruby-appscript (in svn) uses rb-appscript’s #method_missing approach. Unfortunately, I don’t have the time to do any more appscript ports for now (I can’t even keep up on the existing ones), but if you or anyone else wants to give it a go I’m always happy to provide whatever advice I can.
I’m not sure I understand why the argument for HOM isn’t an argument against closures.
Sequence comprehensions are only a small part of the usefulness of closures, and indeed, there’s no reason that you can’t apply syntactic sugaring to make them as clean as your examples above. Translating the final example to Scala:
for (song = 256) yield song.name
OR
songs filter (_.bitRate >= 256) map (_.name)
Are either worse than F-Script’s syntax?
songs name where: songs bitRate >= 256
I’d argue no, and moreover, the use of closures permits the use of multi-statement expressions.
Sorry, the first sentence of my previous comment should read:
I’m not sure I understand why the argument for HOM is an argument against closures.
I think it is more precisely an argument against closures in sequence comprehensions. There are also arguments for internalizing external iterators (e.g. for the sake of encapsulation), which has you reduce a multi-line closure into another higher-order sequence comprehension. The closure is still being used internally, if applicable, but the external interface uses HOM.
There are also arguments for using the highest-order abstraction that will accomplish what you need; in this case, as Philippe points out, array programming allows for optimizations that would be more explicit otherwise, like building a query to be sent to a database. (This isn’t impossible with closures in the presence of, say, sufficient syntax tree introspection; Ambition is one example of doing this with closures: http://ambition.rubyforge.org/ )
On the whole, I would definitely agree that HOM doesn’t replace closures any more than closures replace blocks. It’s fascinating to see what’s possible these days (:
Craig, Scala does indeed introduce sophisticated syntactic sugar. The “_ ” placeholder trick for anonymous functions is quite nice (btw, your code examples appear to have been somewhat mangled). The point of the article is not to provide an argument against blocks in general, but to show that for some common problems there are alternative approaches that provide solutions that can be as nice or even better than the classic block-based approach (of course, this is open for debate). The idea is also to introduce a more general message sending mechanism (i.e., messaging patterns) and then to look at what kind of effect this has on the code we write and on the way we think when programming. One of the effect, in F-Script, is that people tend to use blocks less than in classic Smalltalk, and, for some parts of their code, can more easily think in terms of high level manipulation of entire sets of objects (vs. “one object at a time”). But block closures are still useful for a number of other things in F-Script.
As of Ruby 1.9, all of those samples along the lines of songs.collect {|aSong| aSong.name} and so on can be more concisely written as songs.map(&:name). (The &: is sugar for a block that just sends a single message to its argument, so it’s somewhat similar to the simple syntax of higher-order messaging.)
Chuck, thanks for the comment. It is nice to see Ruby 1.9 standardize this automatic conversion of symbols to blocks. I will update the article to include it.
Actually, for new languages I think it is worth going one step further and completely unify things like symbols and blocks (what is now a symbol becomes a compact notation for a block). That’s how it works in F-Script and this is very useful.