東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Convex Neural Networks /

Sahiner, Arda Ege,

Linked to FindBook

Google Book

Amazon

博客來

Convex Neural Networks /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Convex Neural Networks // Arda Ege Sahiner.
Author:	Sahiner, Arda Ege,
Description:	1 electronic resource (229 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
Contained By:	Dissertations Abstracts International85-06B.
Subject:	Neural networks. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30742170
ISBN:	9798381028164

Convex Neural Networks /
Sahiner, Arda Ege,

Convex Neural Networks /Arda Ege Sahiner. - 1 electronic resource (229 pages)

Source: Dissertations Abstracts International, Volume: 85-06, Section: B.

Neural networks have made tremendous advancements in a variety of machine learning tasks across different fields. Typically, neural networks have relied on heuristically optimizing a non-convex objective, raising doubts into their transparency, efficiency, and empirical performance. In this thesis, we show that a wide variety of neural network architectures are amenable to convex optimization, meaning that their non-convex objectives can be reformulated as convex optimization problems using semi-infinite dual formulations. We first show that for two-layer fully connected neural networks with ReLU activations, the optimization problem is convex and demonstrates a unique link to copositive programming, with a regularizer which promotes both sparsity in the number of activation patterns used in the network, and sparsity in the number of neurons that are active for each activation pattern. We show that this formulation admits closed-form solutions in certain data regimes, and use copositive programming to relax the problem into one that is polynomial-time in the problem dimensions for data matrices of a fixed rank. We show that solving the convex reformulation results in a better solution than that found by heuristic algorithms such as gradient descent applied to the original non-convex objective.In the rest of this thesis, we explore different neural network architectures and training regimes which pose new challenges to the convex optimization formulation. We show that for convolutional neural networks and transformer architectures, the optimization problem also admits a convex reformulation. We also show that for neural networks with batch normalization and generative adversarial networks, the same convex reformulation techniques can disentangle uninterpretable aspects of non-convex optimization and admit faster and more robust solutions to practical problems in the field. Finally, we show that these approaches can be scaled to deeper networks using a Burer-Monteiro factorization of the convex objective which maintains convex guarantees but allows for layerwise stacking convex sub-networks in a scalable fashion.

English

ISBN: 9798381028164Subjects--Topical Terms:

677449
Neural networks.

Convex Neural Networks /
LDR:03476nmm a22003853i 4500 001 2400499
005 20250522084139.5
006 m o d
007 cr|nu||||||||
008 251215s2023 miu||||||m |||||||eng d
020 $a 9798381028164
035 $a (MiAaPQD)AAI30742170
035 $a (MiAaPQD)STANFORDzs047ch0365
035 $a AAI30742170
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Sahiner, Arda Ege, $e author. $3 3770516
245 1 0 $a Convex Neural Networks / $c Arda Ege Sahiner.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2023
300 $a 1 electronic resource (229 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 85-06, Section: B.
500 $a Advisors: Pauly, John; Pilanci, Mert; Vasanawala, Shreyas Committee members: Bent, Stacey F.
502 $b Ph.D. $c Stanford University $d 2023.
520 $a Neural networks have made tremendous advancements in a variety of machine learning tasks across different fields. Typically, neural networks have relied on heuristically optimizing a non-convex objective, raising doubts into their transparency, efficiency, and empirical performance. In this thesis, we show that a wide variety of neural network architectures are amenable to convex optimization, meaning that their non-convex objectives can be reformulated as convex optimization problems using semi-infinite dual formulations. We first show that for two-layer fully connected neural networks with ReLU activations, the optimization problem is convex and demonstrates a unique link to copositive programming, with a regularizer which promotes both sparsity in the number of activation patterns used in the network, and sparsity in the number of neurons that are active for each activation pattern. We show that this formulation admits closed-form solutions in certain data regimes, and use copositive programming to relax the problem into one that is polynomial-time in the problem dimensions for data matrices of a fixed rank. We show that solving the convex reformulation results in a better solution than that found by heuristic algorithms such as gradient descent applied to the original non-convex objective.In the rest of this thesis, we explore different neural network architectures and training regimes which pose new challenges to the convex optimization formulation. We show that for convolutional neural networks and transformer architectures, the optimization problem also admits a convex reformulation. We also show that for neural networks with batch normalization and generative adversarial networks, the same convex reformulation techniques can disentangle uninterpretable aspects of non-convex optimization and admit faster and more robust solutions to practical problems in the field. Finally, we show that these approaches can be scaled to deeper networks using a Burer-Monteiro factorization of the convex objective which maintains convex guarantees but allows for layerwise stacking convex sub-networks in a scalable fashion.
546 $a English
590 $a School code: 0212
650 4 $a Neural networks. $3 677449
690 $a 0800
710 2 $a Stanford University. $e degree granting institution. $3 3765820
720 1 $a Pauly, John $e degree supervisor.
720 1 $a Pilanci, Mert $e degree supervisor.
720 1 $a Vasanawala, Shreyas $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 85-06B.
790 $a 0212
791 $a Ph.D.
792 $a 2023
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30742170