東華大學圖書館 |

Extrapolating AI Agents to Open-World Tasks With Natural Language /

Record Type:	Electronic resources : Monograph/item
Title/Author:	Extrapolating AI Agents to Open-World Tasks With Natural Language // Yu Gu.
Author:	Gu, Yu,
Description:	1 electronic resource (185 pages)
Notes:	Source: Dissertations Abstracts International, Volume: 86-11, Section: A.
Contained By:	Dissertations Abstracts International86-11A.
Subject:	Computer science. -
Online resource:	https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=32111843
ISBN:	9798314889794

Extrapolating AI Agents to Open-World Tasks With Natural Language /
Gu, Yu,

Extrapolating AI Agents to Open-World Tasks With Natural Language /Yu Gu. - 1 electronic resource (185 pages)

Source: Dissertations Abstracts International, Volume: 86-11, Section: A.

Enabling machines to take actions and fulfill diverse human goals constitutes the most long-standing objective of Artificial Intelligence (AI) research. This thesis addresses the problem of developing AI agents in open-world environments characterized by unbounded states, actions, and broad goal specifications the ultimate objective in Al agent development. A core challenge for developing such versatile agents in open-world tasks lies in effective goal transfer (i.e., extrapolating to diverse unseen goals): Classic Al agents operate exclusively within structured logic worlds, while mapping environment dynamics and goals into logic symbols in open-world tasks is extremely challenging, if not impossible; Agents based on reinforcement learning (RL) specify goals as reward functions and are typically only optimized to achieving a fixed goal (e.g., playing Go or cleaning a table). Extrapolating to new goals would require specifying new reward functions and optimization against them. We advocate for explicitly modeling goals in natural language in AI agents. This not only creates a friendly interface between humans and agents, but more importantly, it also allows agents to extrapolate to new goals more easily: imagine an agent that can "water plants in the kitchen" and "fold laundry in the bedroom"-ideally, the agent should understand and execute "water plants in the bedroom" by composing these concepts without requiring additional training for this new goal. Particularly, there are two core components that contribute to extrapolation with natural language modeling: 1) Natural language modeling produces a proper structure for the goal space, including similarity measurements, compositions, etc. 2) Procedural knowledge encoded in natural language can facilitate the mapping from goal to policy. We detail our discussions in two parts: Part I focuses on providing deeper insights into how modeling natural language goals can help in extrapolation. Specifically, leveraging the knowledge base (KB) environment as a testbed, we first establish a benchmark that systematically defines extrapolating to new goals at three levels: i.i.d., compositional, and zero-shot extrapolation. These levels are defined based on the relationship between the training data and test data. Based on our observations using this benchmark, we propose two methods that further enhance extrapolation within the KB environment. However, agents in the KB environment operate with a strong assumption that the goal description and available actions are of a similar abstraction level. This strong assumption does not always hold. For example, in a household setting, a robot instructed to "make a pizza" probably does not come with an atomic action called make_pizza. To handle this, we go one step further in Part II, in which we investigate ways to leverage procedural knowledge encoded in large language models (LLMs) to boost the mapping from goal to policy, without using any training data. We discuss two works. First, we introduce how to use natural language prior to enhance policy greedily in the KB and database environments. Second, we shift to the more complex real-world website environment and show that we can also use natural language prior for model-based planning, i.e., using LLMs to simulate a world model for the Internet and conduct systematic exploration within the simulation. With the work presented in this thesis, we demonstrate the critical role of natural language modeling in developing extrapolative agents in open-world environments. Beyond serving as a communication interface, natural language enables effective modeling of the goal space and provides a prior for the mapping between goals and policies. However, this thesis is just a beginning, more future breakthroughs to incorporate natural language capability more seamlessly into the RL or classic agent framework are yet to come. Finally, the more generalizable statement of this thesis is that policies should be grounded in a proper concept space (i.e., concept-grounded policies), whether derived from language learning or not. The fundamental challenge in Al research remains how to effectively form concept spaces that mirror human cognitive structures.

English

ISBN: 9798314889794Subjects--Topical Terms:

523869
Computer science.
Subjects--Index Terms:

AI agents

Extrapolating AI Agents to Open-World Tasks With Natural Language /
LDR:05657nmm a22004333i 4500 001 2433363
005 20250825104058.5
006 m o d
007 cr|nu||||||||
008 261011s2025 miu||||||m |||||||eng d
020 $a 9798314889794
035 $a (MiAaPQD)AAI32111843
035 $a (MiAaPQD)OhioLINKosu1744888108745188
035 $a AAI32111843
040 $a MiAaPQD $b eng $c MiAaPQD $e rda
100 1 $a Gu, Yu, $e author. $3 3822944
245 1 0 $a Extrapolating AI Agents to Open-World Tasks With Natural Language / $c Yu Gu.
264 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2025
300 $a 1 electronic resource (185 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 86-11, Section: A.
500 $a Advisors: Su, Yu Committee members: Sun, Huan; Brent, Jonathan; Zhu, Zhihui; Chao, Wei-Lun.
502 $b Ph.D. $c The Ohio State University $d 2025.
520 $a Enabling machines to take actions and fulfill diverse human goals constitutes the most long-standing objective of Artificial Intelligence (AI) research. This thesis addresses the problem of developing AI agents in open-world environments characterized by unbounded states, actions, and broad goal specifications the ultimate objective in Al agent development. A core challenge for developing such versatile agents in open-world tasks lies in effective goal transfer (i.e., extrapolating to diverse unseen goals): Classic Al agents operate exclusively within structured logic worlds, while mapping environment dynamics and goals into logic symbols in open-world tasks is extremely challenging, if not impossible; Agents based on reinforcement learning (RL) specify goals as reward functions and are typically only optimized to achieving a fixed goal (e.g., playing Go or cleaning a table). Extrapolating to new goals would require specifying new reward functions and optimization against them. We advocate for explicitly modeling goals in natural language in AI agents. This not only creates a friendly interface between humans and agents, but more importantly, it also allows agents to extrapolate to new goals more easily: imagine an agent that can "water plants in the kitchen" and "fold laundry in the bedroom"-ideally, the agent should understand and execute "water plants in the bedroom" by composing these concepts without requiring additional training for this new goal. Particularly, there are two core components that contribute to extrapolation with natural language modeling: 1) Natural language modeling produces a proper structure for the goal space, including similarity measurements, compositions, etc. 2) Procedural knowledge encoded in natural language can facilitate the mapping from goal to policy. We detail our discussions in two parts: Part I focuses on providing deeper insights into how modeling natural language goals can help in extrapolation. Specifically, leveraging the knowledge base (KB) environment as a testbed, we first establish a benchmark that systematically defines extrapolating to new goals at three levels: i.i.d., compositional, and zero-shot extrapolation. These levels are defined based on the relationship between the training data and test data. Based on our observations using this benchmark, we propose two methods that further enhance extrapolation within the KB environment. However, agents in the KB environment operate with a strong assumption that the goal description and available actions are of a similar abstraction level. This strong assumption does not always hold. For example, in a household setting, a robot instructed to "make a pizza" probably does not come with an atomic action called make_pizza. To handle this, we go one step further in Part II, in which we investigate ways to leverage procedural knowledge encoded in large language models (LLMs) to boost the mapping from goal to policy, without using any training data. We discuss two works. First, we introduce how to use natural language prior to enhance policy greedily in the KB and database environments. Second, we shift to the more complex real-world website environment and show that we can also use natural language prior for model-based planning, i.e., using LLMs to simulate a world model for the Internet and conduct systematic exploration within the simulation. With the work presented in this thesis, we demonstrate the critical role of natural language modeling in developing extrapolative agents in open-world environments. Beyond serving as a communication interface, natural language enables effective modeling of the goal space and provides a prior for the mapping between goals and policies. However, this thesis is just a beginning, more future breakthroughs to incorporate natural language capability more seamlessly into the RL or classic agent framework are yet to come. Finally, the more generalizable statement of this thesis is that policies should be grounded in a proper concept space (i.e., concept-grounded policies), whether derived from language learning or not. The fundamental challenge in Al research remains how to effectively form concept spaces that mirror human cognitive structures.
546 $a English
590 $a School code: 0168
650 4 $a Computer science. $3 523869
650 4 $a Engineering. $3 586835
650 4 $a Linguistics. $3 524476
653 $a AI agents
653 $a Extrapolation
653 $a Language models
690 $a 0984
690 $a 0800
690 $a 0290
690 $a 0537
710 2 $a The Ohio State University. $b Computer Science and Engineering. $e degree granting institution. $3 3821630
720 1 $a Su, Yu $e degree supervisor.
773 0 $t Dissertations Abstracts International $g 86-11A.
790 $a 0168
791 $a Ph.D.
792 $a 2025
856 4 0 $u https://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=32111843