Datastores

Databases of all shapes and sizes.

Amazon Aurora: design considerations for high throughput cloud-native relational databases

~ Adrian Colyer ~ 5 Comments
Amazon Aurora: design considerations for high throughput cloud-native relational databases Verbitski et al., SIGMOD'17 Werner Vogels recently published a blog post describing Amazon Aurora as their fastest growing service ever. That post provides a high level overview of Aurora and then links to two SIGMOD papers for further details. Also of note is the recent ... Continue Reading

The case for network-accelerated query processing

~ Adrian Colyer ~ 2 Comments
The case for network-accelerated query processing Lerner et al., CIDR'19 Datastores continue to advance on a number of fronts. Some of those that come to mind are adapting to faster networks (e.g. ‘FARM: Fast Remote Memory’) and persistent memory (see e.g. ‘Let’s talk about storage and recovery methods for non-volatile memory database systems’), deeply integrating ... Continue Reading

Design continuums and the path toward self-designing key-value stores that know and learn

~ Adrian Colyer ~ 3 Comments
Design continuums and the path toward self-designing key-value stores that know and learn Idreos et al., CIDR'19 We’ve seen systems that help to select the best data structure from a pre-defined set of choices (e.g. ‘Darwinian data structure selection’), systems that synthesise data structure implementations given an abstract specification (‘Generalized data structure synthesis’), systems that ... Continue Reading

Towards a hands-free query optimizer through deep learning

~ Adrian Colyer ~ 3 Comments
Towards a hands-free query optimizer through deep learning Marcus & Papaemmanouil, CIDR'19 Where the SageDB paper stopped— at the exploration of learned models to assist in query optimisation— today’s paper choice picks up, looking exclusively at the potential to apply learning (in this case deep reinforcement learning) to build a better optimiser. Why reinforcement learning? ... Continue Reading

ApproxJoin: approximate distributed joins

~ Adrian Colyer ~ 1 Comment
ApproxJoin: approximate distributed joins Le Quoc et al., SoCC'18 GitHub: https://ApproxJoin.github.io The join is a fundamental data processing operation and has been heavily optimised in relational databases. When you’re working with large volumes of unstructured data though, say with a data processing framework such as Flink or Spark, joins become distributed and much more expensive. ... Continue Reading

Noria: dynamic, partially-stateful data-flow for high-performance web applications

~ Adrian Colyer ~ 3 Comments
Noria: dynamic, partially-stateful data-flow for high-performance web applications Gjengset, Schwarzkopf et al., OSDI'18 I have way more margin notes for this paper than I typically do, and that’s a reflection of my struggle to figure out what kind of thing we’re dealing with here. Noria doesn’t want to fit neatly into any existing box! We’ve ... Continue Reading