東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Convex Optimization Formulation of N...

Wang, Yifei,

Linked to FindBook

Google Book

Amazon

博客來

Convex Optimization Formulation of Neural Networks: Theories, Applications and Beyond /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Convex Optimization Formulation of Neural Networks: Theories, Applications and Beyond // Yifei Wang.
Author:	Wang, Yifei,
Description:	1 electronic resource (372 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-12, Section: B.
Contained By:	Dissertations Abstracts International86-12B.
Subject:	Phase transitions. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31974609
ISBN:	9798311950862

Convex Optimization Formulation of Neural Networks: Theories, Applications and Beyond /
Wang, Yifei,

Convex Optimization Formulation of Neural Networks: Theories, Applications and Beyond /Yifei Wang. - 1 electronic resource (372 pages)

Source: Dissertations Abstracts International, Volume: 86-12, Section: B.

Deep neural networks (DNNs) have revolutionized numerous fields, including computer vision, natural language processing (NLP), and recommendation systems, demonstrating remarkable capabilities in representation learning and generalization. Their empirical success is driven by highly expressive architectures, large-scale datasets, and sophisticated training strategies. However, despite these advancements, a complete theoretical understanding of their optimization and generalization properties remains an open challenge. The intrinsic nonlinearity of neural networks, coupled with over-parameterization and the highly nonconvex nature of their training landscapes, poses significant difficulties in theoretical analysis.Large language models (LLMs), a specialized class of deep networks, have further pushed the boundaries of artificial intelligence by achieving unprecedented performance across a wide range of tasks. However, training such models is computationally intensive, requiring massive datasets and substantial computational resources. Even fine-tuning pretrained LLMs for specific tasks remains a costly and resource-demanding process, limiting their accessibility and scalability. These challenges underscore the need for alternative optimization frameworks that can enhance training efficiency without compromising performance.One promising avenue in this regard is convex optimization, which offers a principled approach to designing efficient training algorithms for neural networks. By leveraging convex formulations, it may be possible to mitigate the computational challenges associated with deep learning while preserving the expressiveness and generalization capabilities of neural networks. This thesis explores how convex optimization techniques can provide new theoretical insights and practical improvements in neural network training, paving the way for more efficient and scalable learning paradigms.Let X ∈ R nxdand y ∈ R nbe the data matrix and the label vector. Given a number of neurons m⩾ 1 and a regularization parameter β> 0, we consider the regularized optimization problem of the simplist neural network, a two-layer ReLU neural network:where Θm= R dxmx R m, θ = (W1, W2), w1,iis the i-th column of W1∈ R dxmand w2,iis the i-th coefficient of w2∈ R m. Here we focus on the ReLU activation, i.e., σ(z) = max{z, 0} and absorb the label y ∈ R nin the loss function ℓ : R n→ R, which is assumed to be convex (e.g., logistic, hinge, squared loss). The model Σmi=1σ(Xw1,i)w2,iin (1.1) can be easily extended to the one with bias term by adding a column of 1's into the data X. We refer to an element θ ∈ Θmas a neural network and to each pair (w1,i, w2,i) as a neuron. We denote the set of optimal neural network as We denote the best training loss achievable by a neural as P* = infm⩾1 P ∗ m.We now introduce an important concept from combinatorial geometry called hyperplane arrangement patterns, which plays an important role in convex optimization formulations of ReLU network training problems.

English

ISBN: 9798311950862Subjects--Topical Terms:

3560387
Phase transitions.

Convex Optimization Formulation of Neural Networks: Theories, Applications and Beyond /
LDR:04345nmm a22003733i 4500 001 2432680
005 20250731101911.5
006 m o d
007 cr|nu||||||||
008 261011s2025 miu||||||m |||||||eng d
020 $a 9798311950862
035 $a (MiAaPQD)AAI31974609
035 $a (MiAaPQD)Stanfordhh655zv7345
035 $a AAI31974609
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Wang, Yifei, $e author. $3 3821996
245 1 0 $a Convex Optimization Formulation of Neural Networks: Theories, Applications and Beyond / $c Yifei Wang.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2025
300 $a 1 electronic resource (372 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-12, Section: B.
500 $a Advisors: Pilanci, Mert Committee members: Montanari, Andrea; Tse, David.
502 $b Ph.D. $c Stanford University $d 2025.
520 $a Deep neural networks (DNNs) have revolutionized numerous fields, including computer vision, natural language processing (NLP), and recommendation systems, demonstrating remarkable capabilities in representation learning and generalization. Their empirical success is driven by highly expressive architectures, large-scale datasets, and sophisticated training strategies. However, despite these advancements, a complete theoretical understanding of their optimization and generalization properties remains an open challenge. The intrinsic nonlinearity of neural networks, coupled with over-parameterization and the highly nonconvex nature of their training landscapes, poses significant difficulties in theoretical analysis.Large language models (LLMs), a specialized class of deep networks, have further pushed the boundaries of artificial intelligence by achieving unprecedented performance across a wide range of tasks. However, training such models is computationally intensive, requiring massive datasets and substantial computational resources. Even fine-tuning pretrained LLMs for specific tasks remains a costly and resource-demanding process, limiting their accessibility and scalability. These challenges underscore the need for alternative optimization frameworks that can enhance training efficiency without compromising performance.One promising avenue in this regard is convex optimization, which offers a principled approach to designing efficient training algorithms for neural networks. By leveraging convex formulations, it may be possible to mitigate the computational challenges associated with deep learning while preserving the expressiveness and generalization capabilities of neural networks. This thesis explores how convex optimization techniques can provide new theoretical insights and practical improvements in neural network training, paving the way for more efficient and scalable learning paradigms.Let X ∈ R nxdand y ∈ R nbe the data matrix and the label vector. Given a number of neurons m⩾ 1 and a regularization parameter β> 0, we consider the regularized optimization problem of the simplist neural network, a two-layer ReLU neural network:where Θm= R dxmx R m, θ = (W1, W2), w1,iis the i-th column of W1∈ R dxmand w2,iis the i-th coefficient of w2∈ R m. Here we focus on the ReLU activation, i.e., σ(z) = max{z, 0} and absorb the label y ∈ R nin the loss function ℓ : R n→ R, which is assumed to be convex (e.g., logistic, hinge, squared loss). The model Σmi=1σ(Xw1,i)w2,iin (1.1) can be easily extended to the one with bias term by adding a column of 1's into the data X. We refer to an element θ ∈ Θmas a neural network and to each pair (w1,i, w2,i) as a neuron. We denote the set of optimal neural network as We denote the best training loss achievable by a neural as P* = infm⩾1 P ∗ m.We now introduce an important concept from combinatorial geometry called hyperplane arrangement patterns, which plays an important role in convex optimization formulations of ReLU network training problems.
546 $a English
590 $a School code: 0212
650 4 $a Phase transitions. $3 3560387
650 4 $a Convex analysis. $3 3681761
650 4 $a Neurons. $3 588699
650 4 $a Algebra. $3 516203
650 4 $a Neural networks. $3 677449
650 4 $a Computer science. $3 523869
690 $a 0984
690 $a 0800
710 2 $a Stanford University. $e degree granting institution. $3 3765820
720 1 $a Pilanci, Mert $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-12B.
790 $a 0212
791 $a Ph.D.
792 $a 2025
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31974609