Michael Sperber - Concurrent ML - The One That Got Away - Code Mesh 2017

Summary

Michael Sperber argues that Concurrent ML (CML) is a powerful, underappreciated approach to concurrent programming whose central ideas deserve renewed attention. His broader point is that programming has sometimes forgotten useful ideas from earlier decades—and that the abstractions we choose strongly affect whether concurrent programs can be composed and reasoned about.

Why composition matters

Sperber recounts trying to translate a CML example—a network that generates Fibonacci numbers—into F# using mailbox processors. The attempt proved awkward because mailbox processors follow an actor-style model: processes receive messages through their own mailboxes. That model works well for many systems, but it does not naturally support the kind of direct composition the example requires.

The experience illustrates his main design principle: good concurrency abstractions should compose. A program built from independent concurrent components should be easy to assemble into larger systems without awkward workarounds.

CML channels and rendezvous

CML uses channels to connect concurrent components. Its communication is synchronous: a send and a receive must meet for the communication to complete.

This supports network-like programs in which components—such as delay, copy, and add operations—are connected through channels. In contrast with an actor mailbox, a CML channel can be shared among multiple processes, allowing a more flexible arrangement of readers and writers.

Sperber notes a problem with naïvely writing these components: ordinary sequential code can impose an arbitrary order on operations. For example, a copy component might send to output one and then output two. In concurrent programs, that ordering can create deadlocks or other problems if the other side is ready for the second operation first.

Selective communication and CML’s composable events

CML addresses this with selective communication: a program can describe several possible communications and proceed with one that is ready, rather than committing to a fixed order.

CML represents such potential communications as rendezvous (also called events in CML terminology). Its key operations and ideas include:

• Send and receive event constructors: Describe a possible send or receive without performing it immediately.
• select / synchronization: Attempt one of several possible communications and carry out a successful one.
• wrap: Transform the value produced by an event—for example, to perform a follow-up action after one communication succeeds.
• choose: Combine several events into a larger event. This lets programmers build up a communication description compositionally before synchronizing on it.
• guard: Create an event from a function that is evaluated when the event is attempted. This is useful when each attempt needs fresh resources, such as a newly allocated channel.

Sperber demonstrates how these mechanisms let components such as copy and add be expressed without hard-coding which communication must happen first. More importantly, events form an algebra that programmers can combine, rather than a limited set of special cases.

An example of a custom protocol: swapping values

CML’s combinators can also support communication patterns beyond simple sends and receives. Sperber presents a swap operation, in which two parties exchange values. The protocol uses selective communication so either side can initiate the exchange, with a fresh channel used to return the other value.

This example illustrates that CML’s composability can be used to build higher-level communication abstractions.

Negative acknowledgements and cancellation

Sperber also describes negative acknowledgements (NACKs), which can help coordinate protocols involving a server. In a lock-server example, a client requests a lock as part of a selective communication. If the client’s request is no longer relevant because another alternative succeeds, the server needs a way to learn that the client has lost interest.

The general pattern is:

1. The client creates a NACK event and includes it with its request.
2. The client waits for a reply from the server.
3. The server selects between acknowledging the request and the client’s NACK.
4. If the NACK wins, the server can abandon the request and make the lock available to someone else; if acknowledgement wins, the client can proceed.

This demonstrates how CML can express coordination and cancellation patterns that can be difficult to implement cleanly in other frameworks.

Comparisons with other concurrency systems

Sperber compares CML with several approaches:

• Erlang and actor systems: Useful for building systems, but their process-and-mailbox model combines multiple concerns and is less naturally composable in the ways CML enables.
• Go: Its select construct supports choosing among communication operations, though the examples discussed are less general than CML’s composable event model.
• F# mailbox processors: Communication is typically asynchronous and centered on agents, making the CML-style channel network difficult to express directly.
• Clojure core.async: Offers selection among communication operations and comes closer to CML, but Sperber argues that composition is constrained to particular kinds of actions.
• Akka Streams and related systems: Provide useful stream abstractions, but the talk questions whether their underlying concurrency abstractions offer the same general composability.
• Reagents: A related approach that Sperber places near CML in the design space, with potential relevance to both concurrent systems and parallel algorithms.
• CSP: A useful model that Sperber regards as less expressive than CML for the compositional techniques he demonstrates.

Threads, buffering, and open problems

Sperber argues that modern frameworks sometimes conflate lightweight concurrency with message-based communication. He says that efficient threads and runtime techniques were better understood in the 1980s than many programmers assume, and that this history is worth revisiting.

He also discusses buffering and back pressure. Asynchronous communication can create queues for messages and for threads waiting to send or receive. In a large system, unbounded queues can grow faster than they are drained and make the system unstable. Sperber’s concern is that APIs may not provide sufficiently composable ways to manage those queues and apply back pressure.

He does not claim that CML resolves every such issue. Instead, he identifies combining composability with back pressure and distribution as an open challenge.

Takeaway

Sperber’s central lesson is that concurrency is not only about choosing between locks, actors, or channels; it is also about choosing abstractions that make concurrent components easy to combine. CML’s synchronous channels and composable events offer an elegant model, and he encourages programmers to try CML or its descendants—even if only to experience how enjoyable well-composed concurrent programming can be.

Speakers and sources featured

• Michael Sperber — presenter.
• An unidentified audience member — asks whether channels could be modeled using processes in Erlang; Sperber answers that this can be made to work, but is not pleasant and remains constrained by the mailbox model.
• Concurrent ML and its associated book — discussed as the talk’s principal conceptual and instructional source; the book’s title is not clearly captured in the subtitles.
• Referenced technologies and approaches: Scheme, Standard ML, Racket, Guile, Haskell, F#, Erlang, Go, Clojure core.async, Akka Streams, CSP, and Reagents.