東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Reinforcement Learning for Swarm Sys...

Ghignone, Leo,

Linked to FindBook

Google Book

Amazon

博客來

Reinforcement Learning for Swarm Systems: Tackling Limited Computational Resources and Decentralised Learning Under Communication Constraints /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Reinforcement Learning for Swarm Systems: Tackling Limited Computational Resources and Decentralised Learning Under Communication Constraints // Leo Ghignone.
Author:	Ghignone, Leo,
Description:	1 electronic resource (254 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-08, Section: B.
Contained By:	Dissertations Abstracts International86-08B.
Subject:	Deep learning. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31863341
ISBN:	9798302896056

Reinforcement Learning for Swarm Systems: Tackling Limited Computational Resources and Decentralised Learning Under Communication Constraints /
Ghignone, Leo,

Reinforcement Learning for Swarm Systems: Tackling Limited Computational Resources and Decentralised Learning Under Communication Constraints /Leo Ghignone. - 1 electronic resource (254 pages)

Source: Dissertations Abstracts International, Volume: 86-08, Section: B.

The advances in technology of the last decades have brought about the diffusion of small-scale computing devices that are cheap to make and can cooperate with each other. Coupled with the theoretical advances in swarm intelligence, the advent of autonomous robotic swarms is now a very realistic possibility for the near future. Reinforcement Learning is among the best techniques available to train these swarms to accomplish arbitrary tasks, however most of the current research in Deep Reinforcement Learning focuses on simulations requiring centralized computations carried on through the use of a very high amount of computing resources.This thesis tries to fill some of the gaps between the current state of the art in Deep Reinforcement Learning and what we will need to successfully apply it to distributed systems of simple units. The first part of the thesis focuses on analysing the memory and time costs of several key state-of-the-art Reinforcement Learning algorithms, in order to provide useful information to direct the choice of algorithms and parameters for applications affected by resource constraints. A theoretical study is presented, with the analysis of several algorithms focusing on the factors affecting memory and time costs, resulting in the definitions of formulas to describe memory costs and asymptotic time complexity for each algorithm; then, an empirical study is performed to validate the formulas and provide additional insight into the trade-offs between memory and time costs, task performance, and reliability of the algorithms.The second part of the thesis focuses on improving decentralized Reinforcement Learning, by proposing a framework to share experience between agents without requiring a central coordinating unit nor perfect communication between the agents. This framework is implemented into two different state-of-the-art algorithms (DQN and PPO), and analysis are performed to evaluate the impact of loss of communication on the performance of the agents and compare this to fully centralized and fully independent versions.The results of the first study show that the experience replay buffer, used by several state-of-the-art Reinforcement Learning algorithms, is by far the main factor increasing the memory cost of Deep Reinforcement Learning, followed by the size of the neural network; the time costs can be more difficult to tune, depending mostly on the choice of algorithm and task, while being less significantly affected by the size of the neural network. Another interesting result is the much higher reliability provided by algorithms such as PPO and DQN compared to other algorithms, which show both a lower robustness to perturbation of parameters and a higher variability between multiple runs with the same parameters. The second study shows the efficacy of experience sharing as a simple decentralized framework for multiagent cooperative Reinforcement Learning, with improved results when applied to both DQN and PPO. Experience sharing resulted in faster convergence on a 2-agents coordinated delivery task than both the centralized and independent DQN versions; it also improved the performance of PPO in a gathering task more than the use of multiple parallel simulations, which is a common technique used in simulated experiments that would not be easily applicable to real-world implementations. The analyses of performance when communication availability is reduced, both by introducing a limited communication distance and a probability of communication failure, show that this framework is still effective when communication is not perfect, and can provide good performance with up to 90% of communication failures.

English

ISBN: 9798302896056Subjects--Topical Terms:

3554982
Deep learning.
Subjects--Index Terms:

Swarm systems

Reinforcement Learning for Swarm Systems: Tackling Limited Computational Resources and Decentralised Learning Under Communication Constraints /
LDR:05097nmm a22003973i 4500 001 2437707
005 20251124055000.5
006 m o d
007 cr|nu||||||||
008 261011s2024 miu||||||m |||||||eng d
020 $a 9798302896056
035 $a (MiAaPQD)AAI31863341
035 $a AAI31863341
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Ghignone, Leo, $e author. $3 3828781
245 1 0 $a Reinforcement Learning for Swarm Systems: Tackling Limited Computational Resources and Decentralised Learning Under Communication Constraints / $c Leo Ghignone.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2024
300 $a 1 electronic resource (254 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-08, Section: B.
500 $a Advisors: Barlow, Michael; Abbass, Hussein.
502 $b Ph.D. $c University of New South Wales (Australia) $d 2024.
520 $a The advances in technology of the last decades have brought about the diffusion of small-scale computing devices that are cheap to make and can cooperate with each other. Coupled with the theoretical advances in swarm intelligence, the advent of autonomous robotic swarms is now a very realistic possibility for the near future. Reinforcement Learning is among the best techniques available to train these swarms to accomplish arbitrary tasks, however most of the current research in Deep Reinforcement Learning focuses on simulations requiring centralized computations carried on through the use of a very high amount of computing resources.This thesis tries to fill some of the gaps between the current state of the art in Deep Reinforcement Learning and what we will need to successfully apply it to distributed systems of simple units. The first part of the thesis focuses on analysing the memory and time costs of several key state-of-the-art Reinforcement Learning algorithms, in order to provide useful information to direct the choice of algorithms and parameters for applications affected by resource constraints. A theoretical study is presented, with the analysis of several algorithms focusing on the factors affecting memory and time costs, resulting in the definitions of formulas to describe memory costs and asymptotic time complexity for each algorithm; then, an empirical study is performed to validate the formulas and provide additional insight into the trade-offs between memory and time costs, task performance, and reliability of the algorithms.The second part of the thesis focuses on improving decentralized Reinforcement Learning, by proposing a framework to share experience between agents without requiring a central coordinating unit nor perfect communication between the agents. This framework is implemented into two different state-of-the-art algorithms (DQN and PPO), and analysis are performed to evaluate the impact of loss of communication on the performance of the agents and compare this to fully centralized and fully independent versions.The results of the first study show that the experience replay buffer, used by several state-of-the-art Reinforcement Learning algorithms, is by far the main factor increasing the memory cost of Deep Reinforcement Learning, followed by the size of the neural network; the time costs can be more difficult to tune, depending mostly on the choice of algorithm and task, while being less significantly affected by the size of the neural network. Another interesting result is the much higher reliability provided by algorithms such as PPO and DQN compared to other algorithms, which show both a lower robustness to perturbation of parameters and a higher variability between multiple runs with the same parameters. The second study shows the efficacy of experience sharing as a simple decentralized framework for multiagent cooperative Reinforcement Learning, with improved results when applied to both DQN and PPO. Experience sharing resulted in faster convergence on a 2-agents coordinated delivery task than both the centralized and independent DQN versions; it also improved the performance of PPO in a gathering task more than the use of multiple parallel simulations, which is a common technique used in simulated experiments that would not be easily applicable to real-world implementations. The analyses of performance when communication availability is reduced, both by introducing a limited communication distance and a probability of communication failure, show that this framework is still effective when communication is not perfect, and can provide good performance with up to 90% of communication failures.
546 $a English
590 $a School code: 0423
650 4 $a Deep learning. $3 3554982
650 4 $a Success. $3 518195
650 4 $a Communication. $3 524709
650 4 $a Neural networks. $3 677449
653 $a Swarm systems
653 $a Deep learning
690 $a 0800
690 $a 0459
710 2 $a University of New South Wales (Australia). $e degree granting institution. $3 3815488
720 1 $a Barlow, Michael $e degree supervisor.
720 1 $a Abbass, Hussein $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-08B.
790 $a 0423
791 $a Ph.D.
792 $a 2024
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31863341