Category: Distributed Systems

Core distributed systems topics, for example consistency, availability and so on.

Wormhole: Reliable pub-sub to support Geo-Replicated Internet Services

~ adriancolyer

Wormhole: Reliable pub-sub to support Geo-Replicated Internet Services - Sharma et al. 2015 At Facebook, lots of applications are interested in data being written to Facebook's data stores. Having each of these applications poll the data stores of interest would be untenable, so Facebook built a pub-sub system to identify updates and transmit notifications to … Continue reading Wormhole: Reliable pub-sub to support Geo-Replicated Internet Services

Extensible Distributed Coordination

~ adriancolyer

Extensible Distributed Coordination - Distler et al. 2015 Coordination services such as ZooKeeper offer a deliberately limited API. As a consequence, more complex coordination tasks have to be implemented as multiple RPCs. In Extensible Distributed Coordination, Distler et al. describe a sandboxed extension mechanism for coordination services that allows execution of client logic in the … Continue reading Extensible Distributed Coordination

Taming uncertainty in distributed systems with help from the network

~ adriancolyer ~ 2 Comments

Taming uncertainty in distributed systems with help from the network - Leners et al. 2015 Albatross is a membership service with a very interesting new twist: it exploits SDN functionality to actively enforce partitions! Perhaps it is not immediately obvious why that might be a good thing :). It turns out there are several benefits: … Continue reading Taming uncertainty in distributed systems with help from the network

Putting Consistency Back into Eventual Consistency

~ adriancolyer ~ 3 Comments

Putting Consistency Back into Eventual Consistency - Balegas et al. 2015 Today's choice is another pick from the recent crop of Eurosys 2015 papers. Balegas et al. show us that we don't have to put up with weak forms of eventual consistency, even in geo-replicated settings. In Building on Quicksand Helland argued that we need … Continue reading Putting Consistency Back into Eventual Consistency

Musketeer – Part II: all for one, and one for all in data processing systems

~ adriancolyer ~ 3 Comments

Musketeer: all for one, one for all in data processing systems - Gog et al. 2015 Musketeer gives you portability of data processing workflows across across data processing systems. It can even analyse your workflow and recommend the best system to run it on, as well as combining systems for different parts of the workflow. … Continue reading Musketeer – Part II: all for one, and one for all in data processing systems

Musketeer – Part I : What’s the best data processing system?

~ adriancolyer ~ 18 Comments

Musketeer: all for one, one for all in data processing systems - Gog et al. 2015 For between 40-80% of the jobs submitted to MapReduce systems, you'd be better off just running them on a single machine... It was Eurosys 2015 last week, and a great new crop of papers were presented. Gog et al. … Continue reading Musketeer – Part I : What’s the best data processing system?

Distributed Snapshots: Determining Global States of Distributed Systems

~ adriancolyer ~ 14 Comments

Distributed Snapshots: Determining Global States of Distributed Systems - Chandy & Lamport 1985. What state is your distributed system in? In the absence of a universal clock, is that even a well-formed question? And if you could take a distributed snapshot of system state, would that be useful? Through an algorithm that has simply become … Continue reading Distributed Snapshots: Determining Global States of Distributed Systems