東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Resource Management for Advanced Dat...

Zhang, Haoyu.

Linked to FindBook

Google Book

Amazon

博客來

Resource Management for Advanced Data Analytics at Large Scale.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Resource Management for Advanced Data Analytics at Large Scale./
Author:	Zhang, Haoyu.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2018,
Description:	149 p.
Notes:	Source: Dissertation Abstracts International, Volume: 79-10(E), Section: B.
Contained By:	Dissertation Abstracts International79-10B(E).
Subject:	Computer science. -
Online resource:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=10823767
ISBN:	9780438049635

Resource Management for Advanced Data Analytics at Large Scale.
Zhang, Haoyu.

Resource Management for Advanced Data Analytics at Large Scale. - Ann Arbor : ProQuest Dissertations & Theses, 2018 - 149 p.

Source: Dissertation Abstracts International, Volume: 79-10(E), Section: B.

Thesis (Ph.D.)--Princeton University, 2018.

The rapidly growing size of data and the complexity of analytics present new challenges for large-scale data analytics systems. Modern distributed computing frameworks need to support not only embarrassingly parallelizable batch jobs, but also advanced applications analyzing text and multimedia data using complex queries and machine learning (ML) models. Given the computation and storage costs of advanced data analytics, resource management is crucial. New applications and workloads expose vastly different characteristics which make traditional scheduling systems inadequate, and at the same time offer great opportunities that lead to new system designs for better performance.

ISBN: 9780438049635Subjects--Topical Terms:

523869
Computer science.

Resource Management for Advanced Data Analytics at Large Scale.
LDR:03574nmm a2200349 4500 001 2164694
005 20181116131021.5
008 190424s2018 ||||||||||||||||| ||eng d
020 $a 9780438049635
035 $a (MiAaPQ)AAI10823767
035 $a (MiAaPQ)princeton:12602
035 $a AAI10823767
040 $a MiAaPQ $c MiAaPQ
100 1 $a Zhang, Haoyu. $3 1282289
245 1 0 $a Resource Management for Advanced Data Analytics at Large Scale.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2018
300 $a 149 p.
500 $a Source: Dissertation Abstracts International, Volume: 79-10(E), Section: B.
500 $a Adviser: Michael J. Freedman.
502 $a Thesis (Ph.D.)--Princeton University, 2018.
520 $a The rapidly growing size of data and the complexity of analytics present new challenges for large-scale data analytics systems. Modern distributed computing frameworks need to support not only embarrassingly parallelizable batch jobs, but also advanced applications analyzing text and multimedia data using complex queries and machine learning (ML) models. Given the computation and storage costs of advanced data analytics, resource management is crucial. New applications and workloads expose vastly different characteristics which make traditional scheduling systems inadequate, and at the same time offer great opportunities that lead to new system designs for better performance.
520 $a In this thesis, we present resource management systems that significantly improve cloud resource efficiency by leveraging the specific characteristics of advanced data analytics applications. We present the design and implementation of the following systems:
520 $a (i) VideoStorm: a video analytics system that scales to process thousands of vision queries on live video streams over large clusters. VideoStorm's offline profiler generates resource-quality profiles for vision queries, and its online scheduler allocates resources to maximize performance in terms of vision processing quality and lag.
520 $a (ii) SLAQ: a cluster scheduling system for approximate ML training jobs that aims to maximize the overall model quality. In iterative and exploratory training settings, better models can be obtained faster by directing resources to jobs with the most potential for improvement. SLAQ allocates resources to maximize the cluster-wide quality improvement based on highly-tailored model quality predictions.
520 $a (iii) Riffle: an optimized shuffle service for big-data analytics frameworks that significantly improves I/O efficiency. The all-to-all data transfer (i.e., shuffle) in modern big-data systems (such as Spark and Hadoop) becomes the scaling bottleneck for multi-stage analytics jobs, due to the superlinear increase in disk I/O operations as data volume grows. Riffle boosts system performance by merging fragmented intermediate files and efficiently scheduling the merge operations.
520 $a Taken together, this thesis demonstrates a novel set of methods in both job-level and task-level scheduling for building scalable, highly-efficient, and cost-effective resource management systems. We have performed extensive evaluation with real production workloads, and our results show significant improvement in resource efficiency, job completion time, and system throughput for advanced data analytics.
590 $a School code: 0181.
650 4 $a Computer science. $3 523869
690 $a 0984
710 2 $a Princeton University. $b Computer Science. $3 2099280
773 0 $t Dissertation Abstracts International $g 79-10B(E).
790 $a 0181
791 $a Ph.D.
792 $a 2018
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=10823767