東華大學圖書館 |

Language: English

Help

回圖書館首頁

手機版館藏查詢

Back

Switch To: Labeled | MARC Mode | ISBD

Policy Learning for High-Precision C...

Burns, Kaylee Carissa,

Linked to FindBook

Google Book

Amazon

博客來

Policy Learning for High-Precision Control /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Policy Learning for High-Precision Control // Kaylee Carissa Burns.
Author:	Burns, Kaylee Carissa,
Description:	1 electronic resource (102 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-12, Section: B.
Contained By:	Dissertations Abstracts International86-12B.
Subject:	Robots. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31974607
ISBN:	9798311957410

Policy Learning for High-Precision Control /
Burns, Kaylee Carissa,

Policy Learning for High-Precision Control /Kaylee Carissa Burns. - 1 electronic resource (102 pages)

Source: Dissertations Abstracts International, Volume: 86-12, Section: B.

The frontier of difficult and interesting manipulation policies requires precision. This thesis studies the challenge of learning high-precision manipulation tasks with modern policy learning methods and develops methods to handle high precision tasks to push forward the horizon of control. We analyze the inadequacies of existing policy learning frameworks up the entire stack of control.First, we note that a precise policy requires high frequency control. Precise tasks create tight success tolerances and, if a policy is not queried at a high enough frequency, it cannot achieve the reactivity necessary to handle those tolerances. Standard offline reinforcement learning (RL) does not accommodate variable frequencies out-of-the-box, so there is no way to train with high frequency data from precise tasks and low frequency data from large-scale robotics datasets. We study how well offline RL algorithms can accommodate data with a mixture of frequencies during training. We observe that the Q-value propagates at different rates for different discretizations, leading to a number of learning challenges for off-the-shelf offline RL algorithms. We present a simple yet effective solution that enforces consistency in the rate of Q-value updates to stabilize learning. By scaling the value of Q in Q-step returns with the discretization size, we effectively balance Q-value propagation, leading to more stable convergence. On three simulated robotic control problems, the resulting policy is capable of generalizing across discretization frequencies.Then, we discuss how to modify imitation learning algorithms to regress very precise action values. While imitation learning has emerged as a promising approach to data-driven robot control, its successful application to fine-manipulation tasks has remained elusive. Complicating this, there is little consensus around what makes fine control difficult for imitation learning. In this work, we first conduct tests in simulation to study whether vision or control bottlenecks progress in fine-manipulation tasks. Concretely, by decoupling the physical geometry of an object from its visual appearance, we find that precise control is the limiting factor in successful imitation learning. From this motivation, we analyze a prototypical instantiation of an imitation learning training pipeline and identify a series of hurdles that hinder the policy from precisely matching demonstrated actions. These include i) how typical loss functions behave in different regions of error magnitude, ii) how insufficient batch size results in gradient noise exceeding the level of precision required by the task, and iii) how a large number of gradient steps is generally required to reach a suitably low loss value. To overcome these hurdles, we propose \\opt{}, a simple training recipe that combines a piecewise L1 loss function, gradient accumulation for an increased virtual batch size, and over-training of the output layer. Together, these modifications enable the regression of very precise actions. We test this on six challenging real-world manipulation tasks and find that Output Precision Tuning improves performance by 75% on average for precise tasks.Finally, we discuss how to design primitives for LLM-based planners to adapt code-generation methods to high-precision tasks. Large Language Models (LLMs) have been successful at generating robot policy code, but so far these results have been limited to high-level tasks that do not require precise movement.

English

ISBN: 9798311957410Subjects--Topical Terms:

529507
Robots.

Policy Learning for High-Precision Control /
LDR:04811nmm a22003733i 4500 001 2438304
005 20251210114004.5
006 m o d
007 cr|nu||||||||
008 261011s2025 miu||||||m |||||||eng d
020 $a 9798311957410
035 $a (MiAaPQD)AAI31974607
035 $a (MiAaPQD)Stanfordgx537mw1273
035 $a AAI31974607
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Burns, Kaylee Carissa, $e author. $3 3829562
245 1 0 $a Policy Learning for High-Precision Control / $c Kaylee Carissa Burns.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2025
300 $a 1 electronic resource (102 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-12, Section: B.
500 $a Advisors: Finn, Chelsea; Bent, Stacey F. Committee members: Hausman, Karol; Liu, Karen.
502 $b Ph.D. $c Stanford University $d 2025.
520 $a The frontier of difficult and interesting manipulation policies requires precision. This thesis studies the challenge of learning high-precision manipulation tasks with modern policy learning methods and develops methods to handle high precision tasks to push forward the horizon of control. We analyze the inadequacies of existing policy learning frameworks up the entire stack of control.First, we note that a precise policy requires high frequency control. Precise tasks create tight success tolerances and, if a policy is not queried at a high enough frequency, it cannot achieve the reactivity necessary to handle those tolerances. Standard offline reinforcement learning (RL) does not accommodate variable frequencies out-of-the-box, so there is no way to train with high frequency data from precise tasks and low frequency data from large-scale robotics datasets. We study how well offline RL algorithms can accommodate data with a mixture of frequencies during training. We observe that the Q-value propagates at different rates for different discretizations, leading to a number of learning challenges for off-the-shelf offline RL algorithms. We present a simple yet effective solution that enforces consistency in the rate of Q-value updates to stabilize learning. By scaling the value of Q in Q-step returns with the discretization size, we effectively balance Q-value propagation, leading to more stable convergence. On three simulated robotic control problems, the resulting policy is capable of generalizing across discretization frequencies.Then, we discuss how to modify imitation learning algorithms to regress very precise action values. While imitation learning has emerged as a promising approach to data-driven robot control, its successful application to fine-manipulation tasks has remained elusive. Complicating this, there is little consensus around what makes fine control difficult for imitation learning. In this work, we first conduct tests in simulation to study whether vision or control bottlenecks progress in fine-manipulation tasks. Concretely, by decoupling the physical geometry of an object from its visual appearance, we find that precise control is the limiting factor in successful imitation learning. From this motivation, we analyze a prototypical instantiation of an imitation learning training pipeline and identify a series of hurdles that hinder the policy from precisely matching demonstrated actions. These include i) how typical loss functions behave in different regions of error magnitude, ii) how insufficient batch size results in gradient noise exceeding the level of precision required by the task, and iii) how a large number of gradient steps is generally required to reach a suitably low loss value. To overcome these hurdles, we propose \\opt{}, a simple training recipe that combines a piecewise L1 loss function, gradient accumulation for an increased virtual batch size, and over-training of the output layer. Together, these modifications enable the regression of very precise actions. We test this on six challenging real-world manipulation tasks and find that Output Precision Tuning improves performance by 75% on average for precise tasks.Finally, we discuss how to design primitives for LLM-based planners to adapt code-generation methods to high-precision tasks. Large Language Models (LLMs) have been successful at generating robot policy code, but so far these results have been limited to high-level tasks that do not require precise movement.
546 $a English
590 $a School code: 0212
650 4 $a Robots. $3 529507
650 4 $a Success. $3 518195
650 4 $a Computer science. $3 523869
690 $a 0984
710 2 $a Stanford University. $e degree granting institution. $3 3765820
720 1 $a Finn, Chelsea $e degree supervisor.
720 1 $a Bent, Stacey F. $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-12B.
790 $a 0212
791 $a Ph.D.
792 $a 2025
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=31974607