Why events are a bad idea (for high concurrency servers) – von Behren et al. 2003

Amongst the authors of this paper you’ll also find Eric Brewer, of CAP-theorem fame.

This is part 3 of a mini-series on thread-based vs event-based programming models. In part 1 we saw the argument that they are equivalent. Despite this, in part 2 we saw the argument that threads are just too hard for mere mortals and we’d be better off using events. Today’s paper argues that events are too hard as well, and we’d be better off with threads in most cases. I guess that does make some kind of a dual – they’re both too hard!

Event-based programming has been highly touted in recent years as the best way to write highly concurrent applications. Having worked on several of these systems, we now believe this approach to be a mistake. Specifically, we believe that threads can achieve all of the strengths of events, including support for high concurrency, low overhead, and a simple concurrency model. Moreover, we argue that threads allow a simpler and more natural programming style.

The authors cite a number of claimed problems with threads, and proceed to address each one.

1. Many attempts to use threads for high concurrency have not performed well: “We believe this is an artefact of poor thread implementations.” The authors created a threading library easily able to scale to 100,000 threads, matching the performing of their event-based implementation of SEDA.
2. Threads restrict control flow: not so; the authors analysed several event-based systems and concluded “In all cases the control flow patterns used by these applications fell into three simple categories: call-return, parallel calls, and pipeline. All of which are more easily expressed with threads.” It is conceded that dynamic fan-in and fan-out patterns (e.g. pub-sub) seem to fit more naturally with events.
3. Thread synchronization is too heavyweight: not so; cooperative multi-tasking support makes the two models equivalent.
4. Thread stacks are an ineffective way to manage live state: not necessarily so; the authors introduce a new dynamic stack growth model later in the paper to address this.
5. Threads prevent the runtime from making optimal scheduling decisions: not so; the Lauer and Needham dual shows that this need not be the case.

Two observations are made that suggest that not only are threads not inherently bad, but they might also be a better model for building servers handling large numbers of concurrent requests:

First, the concurrency in modern servers results from concurrent requests that are largely independent. Second, the code that handles each request is usually sequential. We believe that threads provide a better programming abstraction for servers with these two properties.

• Thread-based systems have much clearer control flow, “event-based programming tends to obfuscate the control flow of the application.”
• Threads provide much simpler exception management and state management, “in event-based systems, task state is typically heap allocated – freeing it at the correct time can be extremely difficult.”
• Even in event-based systems, some things are just too hard to program in an event-based manner:

For example, our own Ninja system [16] ended up using threads for the most complex parts, such as recovery, simply because it was nearly impossible to get correct behavior using events (which we tried first). In addition, applications that didn’t need high concurrency were always written with threads, just because it was simpler.

(these are authors previously dedicated to building event-based systems pushing the state of the art).

The authors advocate for tighter integration between compilers and runtime systems as a powerful mechanism available to threads but not so easily to events. Such integrations can assist with dynamic stack growth, state management, and some forms of data race detection.

A threaded implementation of the SEDA web server was built and found to perform better than the event-based version under very high concurrency. This was attributed to the overheads of context-switching when events pass between handlers, queueing, and dynamic dispatch in the event model.

The conclusion of this work was that…

…although event systems have been used to obtain good performance in high concurrency systems, we have shown that similar or even higher performance can be achieved with threads. Moreover, the simpler programming model and wealth of compiler analyses that threaded systems afford gives threads an important advantage over events when writing highly concurrent servers.