東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Statistical and High-Dimensional Per...

Lee, Donghwan.

Linked to FindBook

Google Book

Amazon

博客來

Statistical and High-Dimensional Perspectives on Machine Learning.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Statistical and High-Dimensional Perspectives on Machine Learning./
Author:	Lee, Donghwan.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2024,
Description:	270 p.
Notes:	Source: Dissertations Abstracts International, Volume: 85-12, Section: A.
Contained By:	Dissertations Abstracts International85-12A.
Subject:	Applied mathematics. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31237254
ISBN:	9798382830889

Statistical and High-Dimensional Perspectives on Machine Learning.
Lee, Donghwan.

Statistical and High-Dimensional Perspectives on Machine Learning. - Ann Arbor : ProQuest Dissertations & Theses, 2024 - 270 p.

Source: Dissertations Abstracts International, Volume: 85-12, Section: A.

Thesis (Ph.D.)--University of Pennsylvania, 2024.

In the first chapter, we consider the problem of calibration. While the accuracy of modern machine learning techniques continues to improve, many models exhibit mis-calibration, wherein the probability scores produced by the models fail to align with the actual frequencies of the labels. This discrepancy can lead to unreliable predictions and hinder the practical application of these models. To address this issue, we frame the task of detecting mis-calibration as a hypothesis testing problem. Drawing inspiration from nonparametric hypothesis testing, we propose T-Cal, a minimax optimal test for calibration based on a debiased plug-in estimator of the ℓ2-Expected Calibration Error (ECE). T-Cal offers a principled and statistically sound approach to assess the calibration of machine learning models.The second chapter focuses on out-of-distribution performance estimation. Evaluating model performance under distribution shift is particularly challenging when we are only given unlabeled data from the target domain. Recent work suggests the notion of disagreement, the degree to which two models trained with different randomness differ on the same input, can be used as a proxy for the accuracy. We establish a theoretical foundation for analyzing disagreement in high-dimensional random features regression. Our analysis shows that there is a linear relationship between source and target disagreement, which we can leverage to estimate the out-of-distribution performance.The third chapter studies feature learning in two-layer neural networks, which is considered one of the fundamental reasons behind the success of deep neural networks. Despite its significance, existing theoretical frameworks do not fully explain the mechanism of feature learning, even in the simplest case of two-layer neural networks. In this work, we enrich our understanding of feature learning by considering a general setting where the learning rate grows with the sample size. Under this setting, we demonstrate that a single step of gradient descent introduces multiple rank-one components to the feature matrix, each corresponding to a specific polynomial feature. Furthermore, we prove that the limiting training and test errors of the updated neural networks are fully characterized by these spikes. By precisely analyzing the improvement in the training and test errors, we illustrate how these non-linear features can enhance the learning process. Through this comprehensive analysis, we shed light on the intricate dynamics of feature learning and its crucial role in the performance of neural networks.

ISBN: 9798382830889Subjects--Topical Terms:

2122814
Applied mathematics.
Subjects--Index Terms:

Deep learning theory

Statistical and High-Dimensional Perspectives on Machine Learning.
LDR:03844nmm a2200409 4500 001 2399322
005 20240909103816.5
006 m o d
007 cr#unu||||||||
008 251215s2024 ||||||||||||||||| ||eng d
020 $a 9798382830889
035 $a (MiAaPQ)AAI31237254
035 $a AAI31237254
040 $a MiAaPQ $c MiAaPQ
100 1 $a Lee, Donghwan. $3 3343928
245 1 0 $a Statistical and High-Dimensional Perspectives on Machine Learning.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 270 p.
500 $a Source: Dissertations Abstracts International, Volume: 85-12, Section: A.
500 $a Advisor: Dobriban, Edgar;Hassani, Hamed.
502 $a Thesis (Ph.D.)--University of Pennsylvania, 2024.
520 $a In the first chapter, we consider the problem of calibration. While the accuracy of modern machine learning techniques continues to improve, many models exhibit mis-calibration, wherein the probability scores produced by the models fail to align with the actual frequencies of the labels. This discrepancy can lead to unreliable predictions and hinder the practical application of these models. To address this issue, we frame the task of detecting mis-calibration as a hypothesis testing problem. Drawing inspiration from nonparametric hypothesis testing, we propose T-Cal, a minimax optimal test for calibration based on a debiased plug-in estimator of the ℓ2-Expected Calibration Error (ECE). T-Cal offers a principled and statistically sound approach to assess the calibration of machine learning models.The second chapter focuses on out-of-distribution performance estimation. Evaluating model performance under distribution shift is particularly challenging when we are only given unlabeled data from the target domain. Recent work suggests the notion of disagreement, the degree to which two models trained with different randomness differ on the same input, can be used as a proxy for the accuracy. We establish a theoretical foundation for analyzing disagreement in high-dimensional random features regression. Our analysis shows that there is a linear relationship between source and target disagreement, which we can leverage to estimate the out-of-distribution performance.The third chapter studies feature learning in two-layer neural networks, which is considered one of the fundamental reasons behind the success of deep neural networks. Despite its significance, existing theoretical frameworks do not fully explain the mechanism of feature learning, even in the simplest case of two-layer neural networks. In this work, we enrich our understanding of feature learning by considering a general setting where the learning rate grows with the sample size. Under this setting, we demonstrate that a single step of gradient descent introduces multiple rank-one components to the feature matrix, each corresponding to a specific polynomial feature. Furthermore, we prove that the limiting training and test errors of the updated neural networks are fully characterized by these spikes. By precisely analyzing the improvement in the training and test errors, we illustrate how these non-linear features can enhance the learning process. Through this comprehensive analysis, we shed light on the intricate dynamics of feature learning and its crucial role in the performance of neural networks.
590 $a School code: 0175.
650 4 $a Applied mathematics. $3 2122814
650 4 $a Statistics. $3 517247
650 4 $a Computer science. $3 523869
650 4 $a Information science. $3 554358
653 $a Deep learning theory
653 $a Machine learning theory
653 $a Hypothesis testing
653 $a Two-layer neural networks
653 $a Out-of-distribution performance
690 $a 0364
690 $a 0463
690 $a 0984
690 $a 0800
690 $a 0723
710 2 $a University of Pennsylvania. $b Applied Mathematics and Computational Science. $3 2094759
773 0 $t Dissertations Abstracts International $g 85-12A.
790 $a 0175
791 $a Ph.D.
792 $a 2024
793 $a English
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31237254