東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Efficient Sparsity for Deep Learning /

Gale, Trevor,

Linked to FindBook

Google Book

Amazon

博客來

Efficient Sparsity for Deep Learning /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Efficient Sparsity for Deep Learning // Trevor Gale.
Author:	Gale, Trevor,
Description:	1 electronic resource (93 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-07, Section: B.
Contained By:	Dissertations Abstracts International86-07B.
Subject:	Neural networks. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31821854
ISBN:	9798302378286

Efficient Sparsity for Deep Learning /
Gale, Trevor,

Efficient Sparsity for Deep Learning /Trevor Gale. - 1 electronic resource (93 pages)

Source: Dissertations Abstracts International, Volume: 86-07, Section: B.

The salient property of deep learning is that model quality increases with scale - namely, model size and the amount of data the model is trained on. This simple fact is the primary driver of the numerous successes of deep neural networks over the last decade. A consequence of this is that our ability to push the state of the art in model capabilities is directly tied to our ability to improve the efficiency with which we're able to train and serve models.This dissertation shows how sparsity can be exploited to increase the efficiency of deep neural networks, an often sought after goal that had previously proven difficult for the field to realize. In particular, we show how careful co-design across algorithms, software and hardware can lead to large reductions in the cost to train and serve a model of a given quality.In the first part of this dissertation, we study the use of unstructured and lightly-structured sparsity to accelerate convolutional neural network inference on mobile and edge devices. To enable efficient sparse computation, we introduce a family of sparse kernels for several hardware platforms. Our kernels are designed for the sparse matrix multiplication operations found in sparse mobile convolutional neural network architectures and exploit careful data layouts, latency hiding and cache-aware loop ordering to maximize throughput on mobile and edge CPUs. Equipped with these kernels, we design sparse versions of MobileNet v1, MobileNet v2 and EfficientNet architectures substantially outperform strong dense baselines on the efficiency-accuracy curve. On Snapdragon 835 our sparse networks outperform their dense equivalents by 1.3 - 2.4 X.In the second part, we study the use of unstructured sparsity to accelerate neural network training and inference on GPUs. To inform our design, we study sparse matrices from deep neural networks and identify favorable properties that can be exploited to accelerate computation. Based on these insights, we develop high-performance GPU kernels for two sparse matrix operations widely applicable in neural networks: sparse matrix-dense matrix multiplication and sampled dense-dense matrix multiplication. Our kernels reach 27% of single precision peak on Nvidia V100 GPUs. Using our kernels, we demonstrate sparse Transformer and MobileNet models that achieve 1.2 - 2.1X speedups and up to 12.8X memory savings without sacrificing accuracy.In the third part, we study the use of coarse-grained structured sparsity to accelerate training mixtureof-experts (MoE) models on GPUs, resulting in a new dropless-MoE (dMoE) architecture. Our system is motivated by the limitations of current frameworks, which restrict the dynamic routing in MoE layers to satisfy the constraints of existing software and hardware. These formulations force a tradeoff between model quality and hardware efficiency, as users must choose between dropping tokens from the computation or wasting computation and memory on padding. To address these limitations, we reformulate MoE computation in terms of block-sparse operations and develop new block-sparse GPU kernels that efficiently handle the dynamism present in MoEs. Our approach never drops tokens and maps efficiently to modern hardware, enabling end-to-end training speedups of up to 40% over MoEs trained with the state-of-the-art Tutel library and 2.4X over DNNs trained with the highly-optimized Megatron-LM framework.Across these efforts, we deliver over 2X reductions in inference latency and training time for a given model quality compared to state-of-the-art dense neural network architectures and implementations. The systems we developed are open-source and deployed at companies including Google and Databricks.

English

ISBN: 9798302378286Subjects--Topical Terms:

677449
Neural networks.

Efficient Sparsity for Deep Learning /
LDR:04979nmm a22003733i 4500 001 2435943
005 20251014050546.5
006 m o d
007 cr|nu||||||||
008 261011s2024 miu||||||m |||||||eng d
020 $a 9798302378286
035 $a (MiAaPQD)AAI31821854
035 $a (MiAaPQD)Stanfordwt784hn3379
035 $a AAI31821854
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Gale, Trevor, $e author. $3 3826432
245 1 0 $a Efficient Sparsity for Deep Learning / $c Trevor Gale.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 1 electronic resource (93 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-07, Section: B.
500 $a Advisors: Kjoelstad, Fredrik Committee members: Olukotun, Oyekunle; Zaharia, Matei.
502 $b Ph.D. $c Stanford University $d 2024.
520 $a The salient property of deep learning is that model quality increases with scale - namely, model size and the amount of data the model is trained on. This simple fact is the primary driver of the numerous successes of deep neural networks over the last decade. A consequence of this is that our ability to push the state of the art in model capabilities is directly tied to our ability to improve the efficiency with which we're able to train and serve models.This dissertation shows how sparsity can be exploited to increase the efficiency of deep neural networks, an often sought after goal that had previously proven difficult for the field to realize. In particular, we show how careful co-design across algorithms, software and hardware can lead to large reductions in the cost to train and serve a model of a given quality.In the first part of this dissertation, we study the use of unstructured and lightly-structured sparsity to accelerate convolutional neural network inference on mobile and edge devices. To enable efficient sparse computation, we introduce a family of sparse kernels for several hardware platforms. Our kernels are designed for the sparse matrix multiplication operations found in sparse mobile convolutional neural network architectures and exploit careful data layouts, latency hiding and cache-aware loop ordering to maximize throughput on mobile and edge CPUs. Equipped with these kernels, we design sparse versions of MobileNet v1, MobileNet v2 and EfficientNet architectures substantially outperform strong dense baselines on the efficiency-accuracy curve. On Snapdragon 835 our sparse networks outperform their dense equivalents by 1.3 - 2.4 X.In the second part, we study the use of unstructured sparsity to accelerate neural network training and inference on GPUs. To inform our design, we study sparse matrices from deep neural networks and identify favorable properties that can be exploited to accelerate computation. Based on these insights, we develop high-performance GPU kernels for two sparse matrix operations widely applicable in neural networks: sparse matrix-dense matrix multiplication and sampled dense-dense matrix multiplication. Our kernels reach 27% of single precision peak on Nvidia V100 GPUs. Using our kernels, we demonstrate sparse Transformer and MobileNet models that achieve 1.2 - 2.1X speedups and up to 12.8X memory savings without sacrificing accuracy.In the third part, we study the use of coarse-grained structured sparsity to accelerate training mixtureof-experts (MoE) models on GPUs, resulting in a new dropless-MoE (dMoE) architecture. Our system is motivated by the limitations of current frameworks, which restrict the dynamic routing in MoE layers to satisfy the constraints of existing software and hardware. These formulations force a tradeoff between model quality and hardware efficiency, as users must choose between dropping tokens from the computation or wasting computation and memory on padding. To address these limitations, we reformulate MoE computation in terms of block-sparse operations and develop new block-sparse GPU kernels that efficiently handle the dynamism present in MoEs. Our approach never drops tokens and maps efficiently to modern hardware, enabling end-to-end training speedups of up to 40% over MoEs trained with the state-of-the-art Tutel library and 2.4X over DNNs trained with the highly-optimized Megatron-LM framework.Across these efforts, we deliver over 2X reductions in inference latency and training time for a given model quality compared to state-of-the-art dense neural network architectures and implementations. The systems we developed are open-source and deployed at companies including Google and Databricks.
546 $a English
590 $a School code: 0212
650 4 $a Neural networks. $3 677449
650 4 $a Linear algebra. $3 2923381
650 4 $a Benchmarks. $3 3548256
650 4 $a Computer science. $3 523869
650 4 $a Sparsity. $3 3680690
650 4 $a Software. $2 gtt. $3 619355
650 4 $a Deep learning. $3 3554982
650 4 $a Computer vision. $3 540671
690 $a 0800
690 $a 0984
710 2 $a Stanford University. $e degree granting institution. $3 3765820
720 1 $a Kjoelstad, Fredrik $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-07B.
790 $a 0212
791 $a Ph.D.
792 $a 2024
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31821854