東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Automatic Speech Separation for Brai...

Han, Cong.

Linked to FindBook

Google Book

Amazon

博客來

Automatic Speech Separation for Brain-Controlled Hearing Technologies.

Record Type:	Electronic resources : Monograph/item
Title/Author:	Automatic Speech Separation for Brain-Controlled Hearing Technologies./
Author:	Han, Cong.
Published:	Ann Arbor : ProQuest Dissertations & Theses, : 2024,
Description:	174 p.
Notes:	Source: Dissertations Abstracts International, Volume: 85-07, Section: B.
Contained By:	Dissertations Abstracts International85-07B.
Subject:	Electrical engineering. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30990427
ISBN:	9798381445657

Automatic Speech Separation for Brain-Controlled Hearing Technologies.
Han, Cong.

Automatic Speech Separation for Brain-Controlled Hearing Technologies. - Ann Arbor : ProQuest Dissertations & Theses, 2024 - 174 p.

Source: Dissertations Abstracts International, Volume: 85-07, Section: B.

Thesis (Ph.D.)--Columbia University, 2024.

Speech perception in crowded acoustic environments is particularly challenging for hearing impaired listeners. While assistive hearing devices can suppress background noises distinct from speech, they struggle to lower interfering speakers without knowing the speaker on which the listener is focusing. The human brain has a remarkable ability to pick out individual voices in a noisy environment like a crowded restaurant or a busy city street. This inspires the brain-controlled hearing technologies. A brain-controlled hearing aid acts as an intelligent filter, reading wearers' brainwaves and enhancing the voice they want to focus on. Two essential elements form the core of brain-controlled hearing aids: automatic speech separation (SS), which isolates individual speakers from mixed audio in an acoustic scene, and auditory attention decoding (AAD) in which the brainwaves of listeners are compared with separated speakers to determine the attended one, which can then be amplified to facilitate hearing. This dissertation focuses on speech separation and its integration with AAD, aiming to propel the evolution of brain-controlled hearing technologies. The goal is to help users to engage in conversations with people around them seamlessly and efficiently.This dissertation is structured into two parts. The first part focuses on automatic speech separation models, beginning with the introduction of a real-time monaural speech separation model, followed by more advanced real-time binaural speech separation models. The binaural models use both spectral and spatial features to separate speakers and are more robust to noise and reverberation. Beyond performing speech separation, the binaural models preserve the interaural cues of separated sound sources, which is a significant step towards immersive augmented hearing. Additionally, the first part explores using speaker identifications to improve the performance and robustness of models in long-form speech separation. This part also delves into unsupervised learning methods for multi-channel speech separation, aiming to improve the models' ability to generalize to real-world audio. The second part of the dissertation integrates speech separation introduced in the first part with auditory attention decoding (SS-AAD) to develop brain-controlled augmented hearing systems. It is demonstrated that auditory attention decoding with automatically separated speakers is as accurate and fast as using clean speech sounds. Furthermore, to better align the experimental environment of SS-AAD systems with real-life scenarios, the second part introduces a new AAD task that closely simulates real-world complex acoustic settings. The results show that the SS-AAD system is capable of improving speech intelligibility and facilitating tracking of the attended speaker in realistic acoustic environments. Finally, this part presents employing self-supervised learned speech representation in the SS-AAD systems to enhance the neural decoding of attentional selection.

ISBN: 9798381445657Subjects--Topical Terms:

649834
Electrical engineering.
Subjects--Index Terms:

Auditory attention decoding

Automatic Speech Separation for Brain-Controlled Hearing Technologies.
LDR:04184nmm a2200385 4500 001 2401881
005 20241022111555.5
006 m o d
007 cr#unu||||||||
008 251215s2024 ||||||||||||||||| ||eng d
020 $a 9798381445657
035 $a (MiAaPQ)AAI30990427
035 $a AAI30990427
035 $a 2401881
040 $a MiAaPQ $c MiAaPQ
100 1 $a Han, Cong. $3 3772101
245 1 0 $a Automatic Speech Separation for Brain-Controlled Hearing Technologies.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 174 p.
500 $a Source: Dissertations Abstracts International, Volume: 85-07, Section: B.
500 $a Advisor: Mesgarani, Nima.
502 $a Thesis (Ph.D.)--Columbia University, 2024.
520 $a Speech perception in crowded acoustic environments is particularly challenging for hearing impaired listeners. While assistive hearing devices can suppress background noises distinct from speech, they struggle to lower interfering speakers without knowing the speaker on which the listener is focusing. The human brain has a remarkable ability to pick out individual voices in a noisy environment like a crowded restaurant or a busy city street. This inspires the brain-controlled hearing technologies. A brain-controlled hearing aid acts as an intelligent filter, reading wearers' brainwaves and enhancing the voice they want to focus on. Two essential elements form the core of brain-controlled hearing aids: automatic speech separation (SS), which isolates individual speakers from mixed audio in an acoustic scene, and auditory attention decoding (AAD) in which the brainwaves of listeners are compared with separated speakers to determine the attended one, which can then be amplified to facilitate hearing. This dissertation focuses on speech separation and its integration with AAD, aiming to propel the evolution of brain-controlled hearing technologies. The goal is to help users to engage in conversations with people around them seamlessly and efficiently.This dissertation is structured into two parts. The first part focuses on automatic speech separation models, beginning with the introduction of a real-time monaural speech separation model, followed by more advanced real-time binaural speech separation models. The binaural models use both spectral and spatial features to separate speakers and are more robust to noise and reverberation. Beyond performing speech separation, the binaural models preserve the interaural cues of separated sound sources, which is a significant step towards immersive augmented hearing. Additionally, the first part explores using speaker identifications to improve the performance and robustness of models in long-form speech separation. This part also delves into unsupervised learning methods for multi-channel speech separation, aiming to improve the models' ability to generalize to real-world audio. The second part of the dissertation integrates speech separation introduced in the first part with auditory attention decoding (SS-AAD) to develop brain-controlled augmented hearing systems. It is demonstrated that auditory attention decoding with automatically separated speakers is as accurate and fast as using clean speech sounds. Furthermore, to better align the experimental environment of SS-AAD systems with real-life scenarios, the second part introduces a new AAD task that closely simulates real-world complex acoustic settings. The results show that the SS-AAD system is capable of improving speech intelligibility and facilitating tracking of the attended speaker in realistic acoustic environments. Finally, this part presents employing self-supervised learned speech representation in the SS-AAD systems to enhance the neural decoding of attentional selection.
590 $a School code: 0054.
650 4 $a Electrical engineering. $3 649834
650 4 $a Speech therapy. $3 520446
650 4 $a Audiology. $3 537237
653 $a Auditory attention decoding
653 $a Brain-computer interface
653 $a Smart hearing aid
653 $a Speech separation
690 $a 0544
690 $a 0300
690 $a 0460
710 2 $a Columbia University. $b Electrical Engineering. $3 1675652
773 0 $t Dissertations Abstracts International $g 85-07B.
790 $a 0054
791 $a Ph.D.
792 $a 2024
793 $a English
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=30990427