Write-limited sorts and joins for persistent memory

September 30, 2016November 11, 2019 ~ Adrian Colyer ~ 1 Comment

Write-limited sorts and joins for persistent memory Viglas, VLDB 2014 This is the second of the two research-for-practice papers for this week. Once more the topic is how database storage algorithms can be optimised for NVM, this time examining the asymmetry between reads and writes on NVM. This is premised on Viglas’ assertion that: Writes ... Continue Reading

Let’s talk about storage and recovery methods for non-volatile memory database systems

September 29, 2016November 11, 2019 ~ Adrian Colyer ~ 6 Comments

Let's talk about storage and recovery methods for non-volatile memory database systems Arulraj et al., SIGMOD 2015 Update: fixed a bunch of broken links. I can't believe I only just found out about this paper! It's exactly what I've been looking for in terms of an analysis of the impacts of NVM on data storage ... Continue Reading

Ambry: LinkedIn’s scalable geo-distributed object store

July 13, 2016 ~ Adrian Colyer ~ 2 Comments

Ambry: LinkedIn’s scalable geo-distributed object store Noghabi et al. SIGMOD ’16 Ambry is LinkedIn’s blob store, designed to handle the demands of a modern social network: Hundreds of millions of users continually upload and view billions of diverse massive media objects, from photos and videos to documents. These large media objects, called blobs, are uploaded ... Continue Reading

Goods: organizing Google’s datasets

July 12, 2016 ~ Adrian Colyer ~ 9 Comments

Goods: organizing Google’s datasets Havely et al. SIGMOD 2016 You can (try and) build a data cathedral. Or you can build a data bazaar. By data cathedral I’m referring to a centralised Enterprise Data Management solution that everyone in the company buys into and pays homage to, making a pilgrimage to the EDM every time ... Continue Reading

A DNA-based archival storage system

May 27, 2016 ~ Adrian Colyer ~ 2 Comments

A DNA-Based Archival Storage System - Bornholt et al. ASPLOS '16 It's pretty cool that a paper on DNA-based storage makes a conference such as ASPLOS. And as you'll see, there are good reasons we should be taking it very seriously indeed. DNA has some very interesting properties - it's extremely dense (1 exabyte (109 ... Continue Reading

NOVA: A Log-Structured File System for Hybrid Volatile/Non-Volatile Main Memories

May 6, 2016 ~ Adrian Colyer ~ Leave a comment

NOVA: A Log-structured file system for hybrid volatile/non-volatile main memories - Xu & Swanson 2016 Another paper looking at the design implications of mixed DRAM and NVMM systems (it's the future!), this time in the context of file systems. (NVMM = Non-volatile Main Memory). Hybrid DRAM/NVMM storage systems present a host of opportunities and challenges ... Continue Reading

Uncovering bugs in Distributed Storage Systems during Testing (not in production!)

May 5, 2016 ~ Adrian Colyer ~ 4 Comments

Uncovering bugs in Distributed Storage Systems during Testing (not in production!) - Deligiannis et al. 2016 We interviewed technical leaders and senior managers in Microsoft Azure regarding the top problems in distributed system development. The consensus was that one of the most critical problems today is how to improve testing coverage so that bugs can ... Continue Reading

Data Tiering in Heterogeneous Memory Systems

April 28, 2016 ~ Adrian Colyer ~ 2 Comments

Data Tiering in Heterogeneous Memory Systems - Dulloor et al. 2016 Another fantastic EuroSys 2016 paper for today, and one with results that are of great importance in understanding the cost and performance implications of the new generation of non-volatile memory (NVM) heading to our data centers soon. Furthermore, we also get some great insight ... Continue Reading

Using Crash Hoare Logic for Certifying the FSCQ File System

February 12, 2016 ~ Adrian Colyer ~ 2 Comments

Using Crash Hoare Logic for Certifying the FSCQ File System - Chen et. al 2015 If you found yesterday's look at POSIX file system semantics and the crash recovery of applications built on top a little sobering, then today's paper offers a glimpse of a light at the end of the tunnel. Chen et al. ... Continue Reading

All File Systems are Not Created Equal: On the Complexity of Crafting Crash Consistent Applications

February 11, 2016 ~ Adrian Colyer ~ 10 Comments

All File Systems are Not Created Equal: On the Complexity of Crafting Crash Consistent Applications - Pillai et al. 2014 Last week we looked at Reducing Crash Recoverability to Reachability for file system-based applications. Today's choice predates that work and investigates the semantics of real world file systems and how this affects applications that run ... Continue Reading

the morning paper

a random walk through Computer Science research, by Adrian Colyer
Made delightfully fast by strattic

Storage