ebooks and download videos Search All  Title  Author 
Home / Nonfiction / Computers / Computer Engineering

An Architecture for Fast and General Data Processing on Large Clusters

| £36.63 | €41.19 | Ca$59.43 | Au$58.65
by Matei Zaharia
What is this?DRM-EPUB by download  |  $43.96
What is this?DRM-PDF by download  |  $43.96
add to wish list
An Architecture for Fast and General Data Processing on Large Clusters by Matei Zaharia

The past few years have seen a major change in computing systems, as growing data volumes and stalling processor speeds require more and more applications to scale out to clusters. Today, a myriad data sources, from the Internet to business operations to scientific instruments, produce large and valuable data streams. However, the processing capabilities of single machines have not kept up with the size of data. As a result, organizations increasingly need to scale out their computations over clusters.

At the same time, the speed and sophistication required of data processing have grown. In addition to simple queries, complex algorithms like machine learning and graph analysis are becoming common. And in addition to batch processing, streaming analysis of real-time data is required to let organizations take timely action. Future computing platforms will need to not only scale out traditional workloads, but support these new applications too.

This book, a revised version of the 2014 ACM Dissertation Award winning dissertation, proposes an architecture for cluster computing systems that can tackle emerging data processing workloads at scale. Whereas early cluster computing systems, like MapReduce, handled batch processing, our architecture also enables streaming and interactive queries, while keeping MapReduce's scalability and fault tolerance. And whereas most deployed systems only support simple one-pass computations (e.g., SQL queries), ours also extends to the multi-pass algorithms required for complex analytics like machine learning. Finally, unlike the specialized systems proposed for some of these workloads, our architecture allows these computations to be combined, enabling rich new applications that intermix, for example, streaming and batch processing.

We achieve these results through a simple extension to MapReduce that adds primitives for data sharing, called Resilient Distributed Datasets (RDDs). We show that this is enough to capture a wide range of workloads. We implement RDDs in the open source Spark system, which we evaluate using synthetic and real workloads. Spark matches or exceeds the performance of specialized systems in many domains, while offering stronger fault tolerance properties and allowing these workloads to be combined. Finally, we examine the generality of RDDs from both a theoretical modeling perspective and a systems perspective.

This version of the dissertation makes corrections throughout the text and adds a new section on the evolution of Apache Spark in industry since 2014. In addition, editing, formatting, and links for the references have been added.

To view this DRM protected ebook on your desktop or laptop you will need to have Adobe Digital Editions installed. It is a free software. We also strongly recommend that you sign up for an AdobeID at the Adobe website. For more details please see FAQ 1&2. To view this ebook on an iPhone, iPad or Android mobile device you will need the Adobe Digital Editions app, or BlueFire Reader or Txtr app. These are free, too. For more details see this article.

SHARE  Share by Email  Share on Facebook  Share on Twitter  Share on Linked In  Share on Delicious
or call in the US toll free 1-888-866-9150 product ID: 840262

Ebook Details
Pages: 141
Size: 5.3 MB
Publisher: ACM Books
Date published:   2016
ISBN: 9781970001587 (DRM-EPUB)
9781970001570 (DRM-PDF)

DRM Settings
Copying:not allowed
Printing:not allowed
Read Aloud:  not allowed

This product is listed in the following category:

Nonfiction > Computers > Computer Engineering

If you find anything wrong with this product listing, perhaps the description is wrong, the author is incorrect, or it is listed in the wrong category, then please contact us. We will promptly address your feedback.

Submit 5 page SummaryWhat is this?

© 2016