Scaling Concurrent Log-Structured Data Stores - Golan-Gueta et al. 2015 Key-value stores based on log-structured merge trees are everywhere. The original design was intended to mitigate slow disk I/O. Once this is achieved, as we scale to more and more cores the authors find that in-memory contention now becomes the bottleneck (see yesterday's piece on … Continue reading Scaling Concurrent Log-Structured Data Stores
How to Quantify Scalability: The Universal Scalability Law (USL) - Gunther Update: corrected sign in USL equation - many thanks to Rob Fielding for pointing out the error. TL;DR: The Universal Scalability Law, Little's Law, and Kingman's Formula can tell you a lot about the behaviour of your systems, and also your organisation. From these … Continue reading Applying the Universal Scalability Law to organisations
Musketeer: all for one, one for all in data processing systems - Gog et al. 2015 Musketeer gives you portability of data processing workflows across across data processing systems. It can even analyse your workflow and recommend the best system to run it on, as well as combining systems for different parts of the workflow. … Continue reading Musketeer – Part II: all for one, and one for all in data processing systems
Musketeer: all for one, one for all in data processing systems - Gog et al. 2015 For between 40-80% of the jobs submitted to MapReduce systems, you'd be better off just running them on a single machine... It was Eurosys 2015 last week, and a great new crop of papers were presented. Gog et al. … Continue reading Musketeer – Part I : What’s the best data processing system?
The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors - Clements et al. 2013 The way you design your interface (API) has a significant impact on the scalability you can achieve with any implementation. Clements et al. define the Scalable Commutativity Rule - which will look familiar to those who study distributed systems - … Continue reading The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors
From the Aether to the Ethernet - Attacking the Internet using Broadcast Digital Television - Oren & Koremytis 2014 Before reading any further, please ensure you are in a carpeted area or other soft ground. Your jaw may hit the floor a few times when you hear what Oren & Koremytis have to tell us, … Continue reading From the Aether to the Ethernet – Attacking the Internet using Broadcast Digital Television
Distributed Snapshots: Determining Global States of Distributed Systems - Chandy & Lamport 1985. What state is your distributed system in? In the absence of a universal clock, is that even a well-formed question? And if you could take a distributed snapshot of system state, would that be useful? Through an algorithm that has simply become … Continue reading Distributed Snapshots: Determining Global States of Distributed Systems