Towards web-based delta synchronization for cloud storage systems Xiao et al., FAST’18 If you use Dropbox (or an equivalent service) to synchronise file between your Mac or PC and the cloud, then it uses an efficient delta-sync (rsync) protocol to only upload the parts of a file that have changed. If you use a web … Continue reading Towards web-based delta synchronization for cloud storage systems
Tag: Storage
File systems, object stores, and other low-level storage concerns.
Clay codes: moulding MDS codes to yield an MSR code
Clay codes: moulding MDS codes to yield an MSR code Vajha et al., FAST’18 As we know, storage fails (or the nodes to which it is directly attached, which amounts to pretty much the same thing). Assuming we can detect the failure, we need to recover from it. In order to be able to recover, … Continue reading Clay codes: moulding MDS codes to yield an MSR code
Barrier-enabled IO stack for Flash storage
Barrier-enabled IO stack for flash storage Won et al., FAST’18 The performance of Flash storage has benefited greatly from concurrency and parallelism - for example, multi-channel controllers, large caches, and deep command queues. At the same time, the time to program an individual Flash cell has stayed fairly static (and even become slightly worse in … Continue reading Barrier-enabled IO stack for Flash storage
Protocol aware recovery for consensus-based storage
Protocol aware recovery for consensus based storage Alagappan et al., FAST’18 Following on from their excellent previous work on ‘All file systems are not created equal’ (well worth a read if you haven’t encountered it yet), in this paper the authors look at how well some of our most reliable protocols — those used in … Continue reading Protocol aware recovery for consensus-based storage
Azure Data Lake Store: a hyperscale distributed file service for big data analytics
Azure data lake store: a hyperscale distributed file service for big data analytics Douceur et al., SIGMOD'17 Today's paper takes us inside Microsoft Azure's distributed file service called the Azure Data Lake Store (ADLS). ADLS is the successor to an internal file system called Cosmos, and marries Cosmos semantics with HDFS, supporting both Cosmos and … Continue reading Azure Data Lake Store: a hyperscale distributed file service for big data analytics
The design, implementation and deployment of a system to transparently compress hundreds of petabytes of image files for a file storage service
The design, implementation, and deployment of a system to transparently compress hundreds of petabytes of image files for a file storage service Horn et al., NSDI'17 When I first started reading, I thought this paper was going to be about a new compression format Dropbox had introduced for JPEG images. And it is about that, … Continue reading The design, implementation and deployment of a system to transparently compress hundreds of petabytes of image files for a file storage service
Application crash consistency and performance with CCFS
Application crash consistency and performance with CCFS Pillai et al., FAST 2017 I know I tend to get over-excited about some of the research I cover, but this is truly a fabulous piece of work. We looked "All file systems are not created equal" in a previous edition of The Morning Paper, which showed that … Continue reading Application crash consistency and performance with CCFS
