Programming and proving with distributed protocols

January 22, 2018 ~ Adrian Colyer ~ Leave a comment

Programming and proving with distributed protocols Sergey et al., POPL 18 Last week we looked at the verification of Rust’s type-based safety guarantees, and a verification toolchain for proving properties of JavaScript programs. Today it’s the turn of Disel, a framework for developing and verifying message-passing based distributed systems (it would seem to fit beautifully ... Continue Reading

SVE: Distributed video processing at Facebook scale

November 27, 2017 ~ Adrian Colyer ~ Leave a comment

SVE: Distributed video processing at Facebook scale Huang et al., SOSP’17 SVE (Streaming Video Engine) is the video processing pipeline that has been in production at Facebook for the past two years. This paper gives an overview of its design and rationale. And it certainly got me thinking: suppose I needed to build a video ... Continue Reading

KV-Direct: High-performance in-memory key-value store with programmable NIC

November 23, 2017 ~ Adrian Colyer ~ 14 Comments

KV-Direct: High-performance in-memory key-value store with programmable NIC Li et al., SOSP’17 We’ve seen some pretty impressive in-memory datastores in past editions of The Morning Paper, including FaRM, RAMCloud, and DrTM. But nothing that compares with KV-Direct: With 10 programmable NIC cards in a commodity server, we achieve 1.22 billion KV operations per second, which ... Continue Reading

Canopy: an end-to-end performance tracing and analysis system

November 22, 2017 ~ Adrian Colyer ~ 1 Comment

Canopy: an end-to-end performance tracing and analysis system Kaldor et al., SOSP’17 In 2014, Facebook published their work on ‘The Mystery Machine,’ describing an approach to end-to-end performance tracing and analysis when you can’t assume a perfectly instrumented homogeneous environment. Three years on, and a new system, Canopy, has risen to take its place. Whereas ... Continue Reading

Algorand: scaling Byzantine agreements for cryptocurrencies

November 21, 2017 ~ Adrian Colyer ~ 10 Comments

Algorand: scaling Byzantine agreements for cryptocurrencies Gilad et al., SOSP 17 The figurehead for Algorand is Silvio Micali, winner of the 2012 ACM Turing Award. Micali has the perfect background for cryptocurrency and blockchain advances: he was instrumental in the development of many of the cryptography building blocks, has published works on game theory and ... Continue Reading

Occupy the cloud: distributed computing for the 99%

October 30, 2017 ~ Adrian Colyer ~ 2 Comments

Occupy the cloud: distributed computing for the 99% Jonas et al., SoCC’17 ‘Occupy the cloud’ won the best vision paper award at the recent ACM Symposium on Cloud Computing event. In the spirit of a vision paper, you won’t find detailed implementation and evaluation information here, but hopefully you’ll find something to make you think. ... Continue Reading

Distributed deep neural networks over the cloud, the edge, and end devices

September 22, 2017 ~ Adrian Colyer ~ 6 Comments

Distributed deep neural networks over the cloud, the edge, and end devices Teerapittayanon et al., ICDCS 17 Earlier this year we looked at Neurosurgeon, in which the authors do a brilliant job of exploring the trade-offs when splitting a DNN such that some layers are processed on an edge device (e.g., mobile phone), and some ... Continue Reading

A cloud-based content gathering network

August 24, 2017 ~ Adrian Colyer ~ 2 Comments

A cloud-based content gathering network Bhattacherjee et al., HotCloud'17 We all know what a content distribution network is, but what's a content gathering network?! CDNs are great, but their benefits are reduced for clients that have a relatively slow last mile connection - especially given that a typical web page request will involve many round ... Continue Reading

Growing a protocol

August 23, 2017 ~ Adrian Colyer ~ Leave a comment

Growing a protocol Ramasubramanian et al., HotCloud'17 I've been really enjoying working my way through a selection of the HotCloud papers - they're relatively short, thought-provoking, and designed to promote discussion (each paper has a set of discussion questions at the very end - great if you're looking at them in a group of some ... Continue Reading

On the design of distributed programming models

August 17, 2017 ~ Adrian Colyer ~ 2 Comments

On the design of distributed programming models Meiklejohn, arXiv 2017. Today's choice is a lovely thought piece by Christopher Meiklejohn, making the case for distributed programming models. We've witnessed a progression in data structures from sequential (non-thread safe) to concurrent, to distributed (think CRDTs). Will the same thing happen with our programming models? And if ... Continue Reading

the morning paper

a random walk through Computer Science research, by Adrian Colyer
Made delightfully fast by strattic

Distributed Systems