Concurrency in Go, Pony, Erlang/Elixir, and Rust

And what we can learn from it

I’ll spare the usual “Moore’s Law, why concurrency is important” yada-yada… It’s just the fact of programmers’ life now that it is.

The Concurrency Model is very hard to get right. In the early days, we’ve just programmed with threads[1] & locks, and that was it. Until we’ve realized that locks are not composable and a terrible concurrency primitive (except for the lowest levels of the system). It is insanely hard to write correct multithread-safe code, especially by a team of people, and it’s very easy to break it. And the more experienced you become, the more you realize how hard it is.

Nowadays developers expect their programming language to help them create concurrent programs that are thread-safe (as well as memory-safe). And the best solution we came up with (surprisingly in 1978) is message passing [2]. Instead of sharing the same memory region, and collaborating by making carefully-orchestrated changes to it, threads send each other messages to collaborate [3].

Now let’s look at how the languages mentioned above have implemented this idea and what can be improved.

Go

We’ll start with Go, because it’s the easiest to analyze. Go has the most faithful implementation of the CSP paper ideas, with Channels as a first-class concept in the language, including the ability to receive from multiple channels, type system support for send-only and receive-only channels, and the ability to send channels in channels. Paired with lightweight Go-routines, Go provides excellent primitives.

Unfortunately, one missing key ingredient prevents Go programs from being provably thread-safe, even if they only rely on channels for concurrency. And that missing ingredient is enforcable immutable data. The ability to send mutable objects thru channels creates a glaring hole in the overall Concurrency Model. We are back to relying on developer’s discipline (as with locks) to not change the data after it is sent thru a channel. I.e. humans are ensuring thread-safety and program correctness.

Granted, this was a non-goal for Go to provide provable thread-safety. And by giving CSP primitives to developers, Go ups the game significantly! Yet, all the same, when analyzing a misbehaving/crashed program, a developer must consider the possibility of data races as one of the reasons for the bug. This dramatically expands the surface area that must be analyzed (essentially to the entire program).

Pony

Pony programs are provably thread-safe[4]. The Concurrency Model is based on the observation that data is safe to share across threads (without a copy) if it’s either immutable or moved. “Moved” here means, that the sending thread provably loses all references to the data.

The concurrency primitive in Pony is an Actor. Actor is a lightweight process combined with a mailbox for receiving messages. There are no channels. A method call on an Actor gets converted into a message that is queued into Actor’s mailbox. The Actor itself is single-threaded, processes messages sequentially in the order they are received, and never processes more than one message at the same time.

So what’s wrong with this model? Actually nothing, it’s completely sound. But one consequence of this model is bothersome: the clients (who send messages) dictate the order in which these messages must be processed by the Actor. This forces every non-trivial Actor implementation into a State Machine.

And we (as an industry) have learned in the early days of async programming with callbacks, that developers don’t find this model intuitive or desirable. That’s why all modern languages offer either async/await constructs or true green threads. The goal is the same: let developers write sequentially-looking code. Which is not the case with Pony Actors.

The State Machine problem could be aliviated with libraries, but that would not be an idiomatic Pony anymore.

Pony remains an incredibly innovative language, with the most sophisticated reference capabilities model. It’s a young language, and nothing prevents it from improving in this area.

Erlang (and Elixir)

Erlang is the hardest to criticize, and for a while, it looks like it offers the perfect model. It is thread-safe and sound. And even though it has no Channels, only lightweight processes with mailboxes, Erlang allows for message processing order to be decided by the receiving process (the server). So it almost looks like it solves the Pony’s problem of making each process into a State Machine. Well, almost.

Erlang allows a process to receive the very next message in the mailbox, or the next message satisfying a pattern (or a list of patterns). And by simply embedding unique IDs into the messages it allows to emulate receiving from multiple channels. I.e. a server may receive a message, call another process and wait for a response from it before resuming general mailbox processing.

The problem lies in the implementation of pattern-based message receive. Namely in its performance: it’s a linear scan of the mailbox. It works well in most cases, but when your server gets under heavy load (especially when it’s a sudden spike), or when the process blocks on some external call longer than usual (while clients keep sending messages into its mailbox), the linear scan becomes prohibitively slow. The server slows down even more, leading to a vicious cycle of even slower mailbox scans. And eventually, the process becomes unresponsive, while mailbox (and memory consumption) grows.

Arguably it’s nit-picking. And it is. But only until your production system exhibits this behavior. You look at your code, and it’s correct, there is no bug, and it should work. And only some hours/days later you finally understand what’s going on, and the fixes are not that straight-forward.

So, Erlang has made a design mistake by coupling Processes with Mailboxes. If it had Processes + Channels and a way to receive from multiple channels, it would avoid the perf problem above. This would also allow a Channel to outlive a Process, possibly allowing Supervisor to create a new one that would resume Channel processing, making clients unaware that the crash has ever happened.

Despite this critique, Erlang creators deserve an immense amount of respect, especially given it was created in 1986. Erlang is still one of the best and most coherent systems ever created for parallel processing. And we should learn from it, both good and bad.

Rust

Simply put, Rust checks all the boxes: it has lightweight async activities, different kinds of channels (many to one, broadcast, etc), multi-channel receive, and guaranteed thread safety — no way to share mutable data across threads. And community is actively building Erlang/OTP-like libraries based on “let it crash” error handling philosophy and Supervision trees.

Rust is not as approachable as other languages above, with very explicit memory management, which requires especially careful planning when multithreaded communications are involved. However, as the Rust community would want you to believe, “if it compiles, it works” [4].

Regardless, having a compiler that guarantees thread-safety is a huge deal when troubleshooting a misbehaving system. Especially for a system language competing with C++ on performance.

[1] For the purposes of this article, thread means a thread of execution, regardless of how it’s implemented (e.g. using OS threads, green threads, lightweight processes, async activities, actors, etc.).

[2] “Communicating Sequential Processes (CSP)”, Hoare

[3] Go community summarizes it as “Do not communicate by sharing memory; instead, share memory by communicating”.

[4] In the absence of unsafe code and bugs in the compiler/runtime.

[5] The picture is © Amazon