東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Open System Neural Networks.

Hatch, Bradley.

Linked to FindBook

Google Book

Amazon

博客來

Open System Neural Networks.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Open System Neural Networks./
Author:	Hatch, Bradley.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2024,
Description:	19 p.
Notes:	Source: Masters Abstracts International, Volume: 85-12.
Contained By:	Masters Abstracts International85-12.
Subject:	Linguistics. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31097653
ISBN:	9798383012642

Open System Neural Networks.
Hatch, Bradley.

Open System Neural Networks. - Ann Arbor : ProQuest Dissertations & Theses, 2024 - 19 p.

Source: Masters Abstracts International, Volume: 85-12.

Thesis (M.Sc.)--Brigham Young University, 2024.

Recent advances in self-supervised learning have made it possible to reuse information-rich models that have been generally pre-trained on massive amounts of data for other downstream tasks. But the pre-training process can be drastically different from the fine-tuning training process, which can lead to inefficient learning. We address this disconnect in training dynamics by structuring the learning process like an open system in thermodynamics. Open systems can achieve a steady state when low-entropy inputs are converted to high-entropy outputs. We modify the the model and the learning process to mimic this behavior, and attend more to elements of the input sequence that exhibit greater changes in entropy. We call this architecture the Open System Neural Network (OSNN). We show the efficacy of the OSNN on multiple classification datasets with a variety of encoder-only Transformers. We find that the OSNN outperforms nearly all model specific baselines, and achieves a new state-of-the-art result on two classification datasets.

ISBN: 9798383012642Subjects--Topical Terms:

524476
Linguistics.

Open System Neural Networks.
LDR:02064nmm a2200349 4500 001 2401896
005 20241022111600.5
006 m o d
007 cr#unu||||||||
008 251215s2024 ||||||||||||||||| ||eng d
020 $a 9798383012642
035 $a (MiAaPQ)AAI31097653
035 $a (MiAaPQ)BrighamYoung11243
035 $a AAI31097653
035 $a 2401896
040 $a MiAaPQ $c MiAaPQ
100 1 $a Hatch, Bradley. $3 3772116
245 1 0 $a Open System Neural Networks.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 19 p.
500 $a Source: Masters Abstracts International, Volume: 85-12.
500 $a Advisor: Fulda, Nancy;Farrell, Ryan;Wingate, David;Jenkins, Porter.
502 $a Thesis (M.Sc.)--Brigham Young University, 2024.
520 $a Recent advances in self-supervised learning have made it possible to reuse information-rich models that have been generally pre-trained on massive amounts of data for other downstream tasks. But the pre-training process can be drastically different from the fine-tuning training process, which can lead to inefficient learning. We address this disconnect in training dynamics by structuring the learning process like an open system in thermodynamics. Open systems can achieve a steady state when low-entropy inputs are converted to high-entropy outputs. We modify the the model and the learning process to mimic this behavior, and attend more to elements of the input sequence that exhibit greater changes in entropy. We call this architecture the Open System Neural Network (OSNN). We show the efficacy of the OSNN on multiple classification datasets with a variety of encoder-only Transformers. We find that the OSNN outperforms nearly all model specific baselines, and achieves a new state-of-the-art result on two classification datasets.
590 $a School code: 0022.
650 4 $a Linguistics. $3 524476
650 4 $a Sentiment analysis. $2 lcstt $3 3266790
650 4 $a Neural networks. $3 677449
650 4 $a Computer science. $3 523869
690 $a 0290
690 $a 0800
690 $a 0984
710 2 $a Brigham Young University. $3 1017451
773 0 $t Masters Abstracts International $g 85-12.
790 $a 0022
791 $a M.Sc.
792 $a 2024
793 $a English
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31097653