Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability

Marcus Hutter

Springer, 2005 - Computers - 278 pages

This book presents sequential decision theory from a novel algorithmic information theory perspective. While the former is suited for active agents in known environments, the latter is suited for passive prediction in unknown environments. The book introduces these two different ideas and removes the limitations by unifying them to one parameter-free theory of an optimal reinforcement learning agent embedded in an unknown environment. Most AI problems can easily be formulated within this theory, reducing the conceptual problems to pure computational ones. Considered problem classes include sequence prediction, strategic games, function minimization, reinforcement and supervised learning. The discussion includes formal definitions of intelligence order relations, the horizon problem and relations to other approaches. One intention of this book is to excite a broader AI audience about abstract algorithmic information theory concepts, and conversely to inform theorists about exciting applications to AI.

Contents

• A Short Tour Through the Book: 1
• Simplicity Uncertainty: 29
• 2 Prefix coding of natural numbers and strings: 34
• 14 Kolmogorovs axioms of probability theory: 42
• 25 Posterior convergence of M to i: 48
• 31 MartinL6f random sequences: 54
• Universal Sequence Prediction: 65
• 8 Convergence of random sequences: 71
• 36 Error bound: 83
• 59 Instantaneous loss bound: 91
• 64 Lower error bound: 97
• 65 Pareto optimality: 99
• Agents in Known Probabilistic Environments: 125
• 1 The agent model: 126
• The Universal Algorithmic Agent AIXI: 141
• 12 We expect AIXI to be universally optimal: 146
• 30 Discounted AIp model and value: 159
• Important Environmental Classes: 185
• Computational Aspects: 209
• 1 The fastest algorithm: 211
• 8 Effective intelligence order relation: 226
• Discussion: 231
• Bibliography: 251
• Index: 265

About the author

Marcus Hutter received his masters in computer sciences in 1992 at the Technical University in Munich, Germany. After his PhD in theoretical particle physics he developed algorithms in a medical software company for 5 years. For four years he has been working as a researcher at the AI institute IDSIA in Lugano, Switzerland. His current interests are centered around reinforcement learning, algorithmic information theory and statistics, universal induction schemes, adaptive control theory, and related areas.

IDSIA (Istituto Dalle Molle di Studi sull'Intelligenza Artificiale) is a non-profit oriented research institute for artificial intelligence, affiliated with both the University of Lugano and SUPSI. It focusses on machine learning (artificial neural networks, reinforcement learning), optimal universal artificial intelligence and optimal rational agents, operations research, complexity theory, and robotics. In Business Week's "X-Lab Survey" IDSIA was ranked in fourth place in the category "Computer Science - Biologically Inspired", after much larger institutions. IDSIA also ranked in the top 10 of the broader category "Artificial Intelligence."

Bibliographic information

• Title: Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability; Texts in Theoretical Computer Science. An EATCS Series
• Author: Marcus Hutter
• Edition: illustrated
• Publisher: Springer, 2005
• ISBN: 3540221395, 9783540221395
• Length: 278 pages
• Subjects: Computers / Computer Science; Computers / Information Theory; Computers / Intelligence (AI) & Semantics; Computers / Mathematical & Statistical Software; Mathematics / Logic