Omid reloaded: scalable and highly-available transaction processing

March 17, 2017November 11, 2019 ~ Adrian Colyer ~ Leave a comment

Omid, reloaded: scalable and highly-available transaction processing Shacham et al., FAST '17 Omid is a transaction processing service powering web-scale production systems at Yahoo that digest billions of events per day and push them into a real-time index. It's also been open-sourced and is currently incubating at Apache as the Apache Omid project. What's interesting ... Continue Reading

Dependency-driven analytics: a compass for uncharted data oceans

January 20, 2017November 11, 2019 ~ Adrian Colyer ~ 4 Comments

Dependency-driven analytics: a compass for uncharted data oceans Mavlyutov et al. CIDR 2017 Like yesterday's paper, today's paper considers what to do when you simply have too much data to be able to process it all. Forget data lakes, we're in data ocean territory now. This is a problem Microsoft faced with their large clusters ... Continue Reading

SnappyData: A unified cluster for streaming, transactions, and interactive analytics

January 18, 2017November 11, 2019 ~ Adrian Colyer ~ 4 Comments

SnappyData: A unified cluster for streaming, transactions, and interactive analytics Mozafari et al., CIDR 2017 Update: fixed broken paper link, thanks Zteve. On Monday we looked at Weld which showed how to combine disparate data processing and analytic frameworks using a common underlying IR. Yesterday we looked at Peloton that adapts to mixed OLTP and ... Continue Reading

Weld: A common runtime for high performance data analytics

January 16, 2017November 11, 2019 ~ Adrian Colyer ~ 8 Comments

Weld: A common runtime for high performance data analytics Palkar et al. CIDR 2017 This is the first in a series of posts looking at papers from CIDR 2017. See yesterday's post for my conference overview. We have a proliferation of data and analytics libraries and frameworks - for example, Spark, TensorFlow, MxNet, Numpy, Pandas, ... Continue Reading

Innovation, experience-based insight and vision at CIDR ’17

January 15, 2017November 11, 2019 ~ Adrian Colyer ~ 2 Comments

Last week was CIDR 2017, the biennial Conference on Innovative Data Systems Research. CIDR encourages authors to take a whole system perspective and especially values "innovation, experience-based insight, and vision." That's a very good match with the attributes of papers I like to cover on The Morning Paper. So what innovation, insight, and vision does ... Continue Reading

Optimizing Distributed Actor Systems for Dynamic Interactive Services

April 29, 2016 ~ Adrian Colyer ~ 1 Comment

Optimizing Distributed Actor Systems for Dynamic Interactive Services - Newell et al. 2016 I'm sure many of you have heard of the Orleans distributed actor system, that was used to build some of the systems supporting Microsoft's online Halo game. Halo Presence is an interactive application which implements presence services for a multi-player game running ... Continue Reading

GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server

April 27, 2016 ~ Adrian Colyer ~ 2 Comments

GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server - Cui et al. 2016 (EuroSys 2016) We know that deep learning is well suited to GPUs since it has inherent parallelism. But so far this has mostly been limited to either a single GPU (e.g. using Caffe) or to specially built distributed ... Continue Reading

Delta State Replicated Data Types

April 25, 2016 ~ Adrian Colyer ~ Leave a comment

Delta State Replicated Data Types - Almeida et al. 2016 You know when you want to use CRDTs for their convergence properties, but the amount of state you're required to pass around gets out of hand? In this paper, Almeida et al. show how to retain the advantages of state-based CRDTs, but with much smaller ... Continue Reading

Fast In-memory Transaction Processing using RDMA and HTM

October 26, 2015 ~ Adrian Colyer ~ 4 Comments

Fast In-memory Transaction Processing using RDMA and HTM - Wei et al. 2015 This paper tries to answer a natural question: with advanced processor features and fast interconnects, can we build a transaction processing system that is at least one order of magnitude faster than the state-of-the-art systems without using such features? The authors build ... Continue Reading

Implementing Linearizability at Large Scale and Low Latency

October 22, 2015 ~ Adrian Colyer ~ 8 Comments

Implementing Linearizability at Large Scale and Low Latency - Lee at al. 2015 Yesterday we saw how to layer a strictly serializable transaction model on top of an inconsistent replication protocol. Previously we've also looked at how to bolt-on a causal consistency model on top of eventual consistency. Today's paper demonstrates how to bolt-on (layer) ... Continue Reading

the morning paper

a random walk through Computer Science research, by Adrian Colyer
Made delightfully fast by strattic

Distributed Systems