Scaling Spark in the real world: performance and usability

November 4, 2016November 11, 2019 ~ Adrian Colyer ~ 5 Comments

Scaling Spark in the real world: performance and usability Armbrust et al. VLBD 2015 A short and easy paper from the Databricks team to end the week. Given the pace of development in the Apache Spark world, a paper published in 2015 about enhancements to Spark will of course be a little dated. But this ... Continue Reading

Algorithmic improvements for fast concurrent cuckoo hashing

November 3, 2016November 11, 2019 ~ Adrian Colyer ~ Leave a comment

Algorithmic improvements for fast concurrent cuckoo hashing Li et al, EuroSys 2014 Today’s paper continues the work on optimistic cuckoo hashing that we looked at yesterday, extending it to support multiple writers and even higher throughput. One of the original goals for the research was to take advantage of the hardware transactional memory support in ... Continue Reading

MemC3: Compact and concurrent Memcache with dumber caching and smarter hashing

November 2, 2016November 11, 2019 ~ Adrian Colyer ~ 5 Comments

MemC3: Compact and Concurrent MemCache with Dumber Caching and Smarter Hashing Fan et al. NSDI 2013 At the core of this paper is an improved hashing algorithm called optimistic cuckoo hashing, and a CLOCK-based eviction algorithm that works in tandem with it. They are evaluated in the context of Memcached, where combined they give up ... Continue Reading

Getting beyond MVP

November 1, 2016November 11, 2019 ~ Adrian Colyer ~ 4 Comments

Something a little different for today, instead of a new paper I wanted to pause and reflect on the design advice we've been reading from Tony Hoare, Barbara Liskov, and David Parnas among others. In particular, how it can be applied in a situation I've recently been through with a codebase of my own, and ... Continue Reading

Designing software for ease of extension and contraction

October 31, 2016November 11, 2019 ~ Adrian Colyer ~ 3 Comments

Designing software for ease of extension and contraction Parnas, IEEE Transactions on Software Engineering, 1979 A couple of weeks ago we looked at ‘A design methodology for reliable software systems’. David Parnas posted a comment (thank you!) giving some more insight into the history of Dijkstra’s “levels of abstraction” concept: This paper followed Dijkstra in ... Continue Reading

A Bayesian approach to graphical record linkage and de-duplication

October 28, 2016November 11, 2019 ~ Adrian Colyer ~ 3 Comments

A Bayesian approach to graphical record linkage and de-duplication Steorts et al. AISTATS, 2014 I don't normally cover papers from statistics conferences and journals, but this one caught my eye as addressing a high-value problem. Through a different lens, it also shows some of the challenges in maintaining privacy when it is possible to join ... Continue Reading

Replex: A scalable, highly available multi-index data store

October 27, 2016November 11, 2019 ~ Adrian Colyer ~ 2 Comments

Replex: A scalable, highly available multi-index data store Tai et al. USENIX 2016 Today’s choice won a best paper award at USENIX this year. Replex addresses the problem of key-value stores in which you also want to have an efficient query capability by values other than the primary key. … NoSQL databases achieve scalability by ... Continue Reading

Preemptive intrusion detection: theoretical framework and real-world measurements

October 26, 2016November 11, 2019 ~ Adrian Colyer ~ 1 Comment

Preemptive intrusion detection: theoretical framework and real-world measurements Cao et al, HotSoS 2015 Phuong Cao (the first author of this paper) got in touch following my review of DeepDive to say "Thanks for the review on DeepDive. I was inspired by that paper to apply factor graph on detecting intrusions at an early stage..." Preemptive ... Continue Reading

Purposes, concepts, misfits, and a redesign of Git

October 25, 2016November 11, 2019 ~ Adrian Colyer ~ 1 Comment

Purposes, concepts, misfits, and a redesign of GitDe Rosso & Jackson OOPSLA '16 It's OOPSLA in just a few weeks time, and 'Purposes, concepts, misfits, and a redesign of Git' will be presented there, bringing the story we looked at yesterday up to date. The subject matter is the same of course - looking at ... Continue Reading

What’s wrong with Git? A conceptual design analysis

October 24, 2016November 11, 2019 ~ Adrian Colyer ~ 16 Comments

What's wrong with Git? A conceptual design analysis De Rossi & Jackson Onward! 2013 We finished up last week talking about the how to find good concepts / abstractions in a software design and what good modularization looks like. Today's paper jumps 40+ years to look at some of those issues in a modern context ... Continue Reading

the morning paper

a random walk through Computer Science research, by Adrian Colyer
Made delightfully fast by strattic

Uncategorized