Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics - Venkataraman et al. 2016 With cloud computing environments such as Amazon EC2, users typically have a large number of choices in terms of the instance types and number of instances they can run their jobs on. Not surprisingly, the amount of memory per core, storage media, … Continue reading Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics
Tag: Analytics
Data analytics
MacroBase: Analytic Monitoring for the Internet of Things
MacroBase: Analytic Monitoring for the Internet of Things - Bailis et al. 2016 It looks like Peter Alvaro is not the only one to be doing some industrial collaboration recently! MacroBase is the result of Peter Bailis' collaboration with Cambridge Mobile Telematics (CMT), an IoT company. The topic at hand is analytic monitoring - detecting … Continue reading MacroBase: Analytic Monitoring for the Internet of Things
Trajectory Data Mining: An Overview
Trajectory Data Mining: An Overview - Zheng 2015 In 'Trajectory Data Mining,' Zheng conducts a high-level tour of the techniques involved in working with trajectory data. This is the data created by a moving object, as a sequence of locations, often with uncertainty around the exact location at each point. This could be GPS trajectories … Continue reading Trajectory Data Mining: An Overview
Arabesque: A System for Distributed Graph Mining
Arabesque: A System For Distributed Graph Mining - Teixeira et al. 2015 We've studied graph computation systems before in The Morning Paper: systems such as Pregel, Giraph and GraphLab that provide vertex-centric programming models ('think like a vertex') on top of a Bulk Synchronous Parallel compute model. We've also seen some of the limitations of … Continue reading Arabesque: A System for Distributed Graph Mining
Pregelix: Big(ger) Graph Analytics on a Dataflow Engine
Pregelix: Big(ger) Graph Anayltics on a Dataflow Engine - Bu et al. 2015 FlashGraph shows us that it's possible to efficiently process graphs that aren't solely in-memory, and GraphX showed us that we can map graph abstractions on top of a dataflow engine. Put the two ideas together, and you get something that looks like … Continue reading Pregelix: Big(ger) Graph Analytics on a Dataflow Engine
Making Sense of Performance in Data Analytics Frameworks
Making Sense of Performance in Data Analytics Frameworks - Ousterhout et al. 2015 We all know the causes of poor performance in big data analytics workloads: network I/O, disk I/O, and straggler tasks. Ousterhout et al. set out to try and quantify this, and found that what we think we know isn't necessarily so. Yet … Continue reading Making Sense of Performance in Data Analytics Frameworks
ApproxHadoop: Bringing Approximations to MapReduce Frameworks
ApproxHadoop: Bringing Approximations to MapReduce Frameworks - Goiri et al. 2015 Yesterday we saw how including networking concerns in scheduling decisions can increase throughput for MapReduce jobs (and Storm topologies) by ~30%. Today we look at an even more effective strategy for getting the most out of your Hadoop cluster: doing less work! On one … Continue reading ApproxHadoop: Bringing Approximations to MapReduce Frameworks
Impala: a modern, open-source SQL engine for Hadoop
Impala: A modern, open-source SQL engine for Hadoop - Kornacker et al . 2015 (Cloudera*) This is post 4 of 5 in a series looking at the latest research from CIDR'15. Also in the series so far this week: 'The missing piece in complex analytics', 'WANalytics, analytics for a geo-distributed, data intensive world', and 'Liquid: … Continue reading Impala: a modern, open-source SQL engine for Hadoop
WANalytics: Analytics for a geo-distributed, data intensive world
WANalytics: analytics for a geo-distributed data intensive world - Vulimiri et al. 2015 ...data is born distributed; we only control data replication and distributed execution strategies. This is true for so many sources of data. Combine this with Dave McCrory's observation that 'Data has Gravity' (i.e. it attracts applications and other data processing workloads to … Continue reading WANalytics: Analytics for a geo-distributed, data intensive world
The Missing Piece in Complex Analytics
The Missing Piece in Complex Analytics: Low latency scalable model management and serving with Velox - Crankshaw et al. 2015. Analytics at scale can be used to create statistical models for making predictions about the world, but once the data scientists and analysts have done their initial work and a model has been built and … Continue reading The Missing Piece in Complex Analytics
the morning paper 