Beyond data and model parallelism for deep neural networks

June 12, 2019May 25, 2020 ~ Adrian Colyer ~ 12 Comments

Beyond data and model parallelism for deep neural networks Jia et al., SysML'2019 I’m guessing the authors of this paper were spared some of the XML excesses of the late nineties and early noughties, since they have no qualms putting SOAP at the core of their work! To me that means the "simple" object access ... Continue Reading

PyTorch-BigGraph: a large-scale graph embedding system

June 10, 2019May 25, 2020 ~ Adrian Colyer ~ Leave a comment

PyTorch-BigGraph: a large-scale graph embedding system Lerer et al., SysML'19 We looked at graph neural networks earlier this year, which operate directly over a graph structure. Via graph autoencoders or other means, another approach is to learn embeddings for the nodes in the graph, and then use these embeddings as inputs into a (regular) neural ... Continue Reading

Towards federated learning at scale: system design

June 7, 2019May 25, 2020 ~ Adrian Colyer ~ 1 Comment

Towards federated learning at scale: system design Bonawitz et al., SysML 2019 This is a high level paper describing Google’s production system for federated learning. One of the most interesting things to me here is simply to know that Google are working on this, have a first version in production working with tens of millions ... Continue Reading

Data validation for machine learning

June 5, 2019May 25, 2020 ~ Adrian Colyer ~ 15 Comments

Data validation for machine learning Breck et al., SysML'19 Last time out we looked at continuous integration testing of machine learning models, but arguably even more important than the model is the data. Garbage in, garbage out. In this paper we focus on the problem of validation the input data fed to ML pipelines. The ... Continue Reading

Continuous integration of machine learning models with ease.ml/ci

June 3, 2019May 25, 2020 ~ Adrian Colyer ~ 2 Comments

Continuous integration of machine learning models with ease.ml/ci: towards a rigorous yet practical treatment Renggli et al., SysML'19 Developing machine learning models is no different from developing traditional software, in the sense that it is also a full life cycle involving design, implementation, tuning, testing, and deployment. As machine learning models are used in more ... Continue Reading

A case for lease-based, utilitarian resource management on mobile devices

May 31, 2019May 25, 2020 ~ Adrian Colyer ~ 4 Comments

A case for lease-based, utilitarian resource management on mobile devices Hu et al., ASPLOS'19 I’ve chosen another energy-related paper to end the week, addressing a problem many people can relate to: apps that drain your battery. LeaseOS borrows the concept of a lease from distributed systems, but with a rather nice twist, and is able ... Continue Reading

Boosted race trees for low energy classification

May 29, 2019May 25, 2020 ~ Adrian Colyer ~ Leave a comment

Boosted race trees for low energy classification Tzimpragos et al., ASPLOS'19 We don’t talk about energy as often as we probably should on this blog, but it’s certainly true that our data centres and various IT systems consume an awful lot of it. So it’s interesting to see a paper using nano-Joules per prediction as ... Continue Reading

CheriABI: enforcing valid pointer provenance and minimizing pointer privilege in the POSIX C run-time environment

May 28, 2019May 25, 2020 ~ Adrian Colyer ~ 1 Comment

CheriABI: enforcing valid pointer provenance and minimizing pointer privilege in the POSIX C run-time environment Davis et al., ASPLOS'19 Last week we saw the benefits of rethinking memory and pointer models at the hardware level when it came to object storage and compression (Zippads). CHERI also rethinks the way that pointers and memory work, but ... Continue Reading

Compress objects, not cache lines: an object-based compressed memory hierarchy

May 24, 2019May 25, 2020 ~ Adrian Colyer ~ 14 Comments

Compress objects, not cache lines: an object-based compressed memory hierarchy Tsai & Sanchez, ASPLOS'19 Last time out we saw how Google have been able to save millions of dollars though memory compression enabled via zswap. One of the important attributes of their design was easy and rapid deployment across an existing fleet. Today’s paper introduces ... Continue Reading

Software-defined far memory in warehouse scale computers

May 22, 2019May 25, 2020 ~ Adrian Colyer ~ 13 Comments

Software-defined far memory in warehouse-scale computers Lagar-Cavilla et al., ASPLOS'19 Memory (DRAM) remains comparatively expensive, while in-memory computing demands are growing rapidly. This makes memory a critical factor in the total cost of ownership (TCO) of large compute clusters, or as Google like to call them "Warehouse-scale computers (WSCs)." This paper describes a "far memory" ... Continue Reading

the morning paper

a random walk through Computer Science research, by Adrian Colyer
Made delightfully fast by strattic