On the duality of operating system structures

On the Duality of Operating System Structures - Lauer and Needham, 1978 The pendulum currently says "threads and locks are bad, events are good." Vigourous defences are mounted in favour of one system over the other, and manifestos are written. Nowadays this debate rages over the best way to build applications and frameworks, but it … Continue reading On the duality of operating system structures

Tachyon: Reliable, Memory Speed Storage for Cluster Computing

Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks - Li et al. 2014 Data processing can often be naturally expressed as a sequence of steps in a pipeline. For example, the unix command line below that pipes a file through a series of transforms to ultimately generate some output. cat Fin.csv | a | … Continue reading Tachyon: Reliable, Memory Speed Storage for Cluster Computing

Photon: Fault-tolerant and scalable joining of continuous data streams

Photon: Fault-tolerant and scalable joining of continuous data streams - Google 2013 To the best of our knowledge, this is the first paper to formulate and solve the problem of joining multiple streams continuously under these system constraints: exactly-once semantics, fault-tolerance at datacenter-level, high scalability, low latency, unordered streams, and delayed primary stream. It's interesting … Continue reading Photon: Fault-tolerant and scalable joining of continuous data streams

The 8 Requirements of Real-Time Stream Processing

The 8 Requirements of Real-Time Stream Processing - Stonebraker et al. 2005 Applications that require real-time processing of high-volume data streams are pushing the limits of data processing infrastructures. Stonebraker et al. make the case in 2005 that stream processing is going to become increasingly important. Not just for the usual finance, fraud, and command-and-control … Continue reading The 8 Requirements of Real-Time Stream Processing

The case for distributed operating systems in the data center

New wine in old skins: the case for distributed operating systems in the data center - Schwarzkopf et al. 2013. I attended the New Directions in Operating Systems one-day event in London last week, and came away with the impression that the beginning of the end of the traditional operating system is in sight. Today's … Continue reading The case for distributed operating systems in the data center