Advances in Computation:
Theory and Practice
Volume 8
Machines That Learn to Play Games
Editors: Johannes Fürnkranz and Miroslav Kubat
|
Chapter 1 - Should Machines Learn How to Play
Games?
Chapter 2 - Machine Learning in Games: A
Survey
Johannes Fürnkranz
2.1 Samuel's Legacy
2.1.1 Machine Learning
2.1.2 Game Playing
2.1.3 Chapter overview
2 .2 Book Learning
2.2.1 Learning to choose opening
variations
2.2.2 Learning to extend the opening book
2.2.3 Learning from mistakes
2.2.4 Learning from simulation
2 .3 Learning Search Control
2.4 Evaluation Function Tuning
2.4.1 Supervised learning
2.4.2 Comparison training
2.4.3 Reinforcement learning
2.4.4 Temporal-difference learning
2.4.5 Issues for evaluation function learning
2 .5 Learning Patterns and
Plans
2.5.1 Advice-taking
2.5.2 Cognitive models
2.5.3 Pattern-based learning systems
2.5.4 Explanation-based learning
2.5.5 Pattern induction
2.5.6 Learning playing strategies
2 .6 Opponent Modeling
2.7 Conclusion
Chapter 3 - Human Learning in Game Playing
Fernand Gobet & Herbert A.
Simon
3.1 Introduction
3.2 Research on memory and perception
3.2.1 Memory
3.2.2 Perception
3.2.3 Modeling experts' perception and memory: The
chunking and template theories
3 .3 Research on problem
solving
3.3.1 De Groot's results
3.3.2 Theories and computer models of problem solving
3 .4 Empirical studies of
learning
3.4.1 Short-range learning
3.4.2 Medium-range learning
3.4.3 Long-range learning
3 .5 Human and machine learning
3.5.1 How has human learning
informed machine learning?
3.5.2 What does machine learning tell us about human
learning?
3 .6 Conclusion
Chapter 4 - Toward Opening Book Learning
Michael Buro
4.1 Introduction
4.2 Basic Requirements
4.3 Choosing Book Moves
4.4 Book Extension
4.5 Implementation Aspects
4.6 Discussion and Enhancements
4.7 Outlook
Chapter 5 - Reinforcement Learning and Chess
Johnathan Baxter, Andrew
Tridgell, Lex Weaver
5.1 Introduction
5.2 KnightCap
5.2.1 Broad representation
5.2.2 Search algorithm
5.2.3 Null moves
5.2.4 Search extensions
5.2.5 Asymmetries
5.2.6 Transposition Tables
5.2.7 Move ordering
5.2.8 Parallel search
5.2.9 Evaluation function
5.2.10 Modification for TDLeaf(λ)
5 .3 The TD(λ) algorithm
applied to games
5.4 Minimax Search and TD(λ)
5.5 TDLeaf(λ) and Chess
5.5.1 Experiments with
KnightCap
5.5.2 Discussion
5 .6 Experiment with Backgammon
5.6.1 LGammon
5.6.2 Experiment with LGammon
5 .7 Future Work
5.8 Conclusion
Chapter 6 - Comparison Training of Chess Evaluation
Functions
Gerald Tesauro
6.1 Introduction
6.2 Comparison Training for Arbitrary-Depth Searches
6.3 Tuning the SCP evaluation function
6.3.1 Experimental details
6.3.2 Simple 1-ply training
6.3.3 Training with 1-ply search plus expansions
6 .4 Tuning Deep Blue's
evaluation function
6.4.1 Modified training
algorithm
6.4.2 Effect on the Kasparov-Deep Blue rematch
6 .5 Discussion
Chapter 7 - Feature Construction for Game
Playing
Paul E. Utgoff
7.1 Introduction
7.2 Evaluation Functions
7.3 Feature Overlap
7.4 Construction Overlapping Features
7.4.1 Parameter tuning
7.4.2 Higher order expansion
7.4.3 Quasi-random methods
7.4.4 Knowledge derivation
7 .5 Directions for Constructing
Overlappig Features
7.5.1 Layered learning
7.5.2 Compression
7.5.3 Teachable systems
7 .6 Discussion
Chapter 8 - Learning to Play Expertly: A Tutorial on
Hoyle
Susan L. Epstein
8.1 Introduction
8.2 A Game-Playing Vocabulary
8.3 Underlying Principles
8.3.1 Useful knowledge
8.3.2 The Advisors
8.3.3 The architecture
8.3.4 Weight learning for voting
8 .4 Perceptual Enhancement
8.4.1 Patterns
8.4.2 Zones
8 .5 An Empirical Framework
8.6 Results
8.7 Conclusion: Why Hoyle Works
Chapter 9 - Acquisition of Go Knowledge from Game
Records
Takuya Kojima & Atsushi
Yoshikawa
9.1 Introduction
9.1.1 Purpose
9.1.2 Classification ot Go Knowledge
9.1.3 Two Approaches
9 .2 Rules to Go
9.3 A Deductive Approach
9.3.1 System
9.3.2 Rule Acquisition
9 .4 An Evolutionary Approach
9.4.1 Algorithm
9.4.2 Application to Tsume-Go
9 .5 Conclusions
Chapter 10 - Honte, a G0-Playing Program Using Neural
Nets
Fredrik A. Dahl
10.1 Introduction
10.1.1 Rules
10.1.2 Strength of programs
1 0.2 General Approach in Honte
10.3 Joseki Library
10.4 Shape Evaluating Neural Net
10.5 Alpha-Beta Search
10.6 Influence
10.7 Neural Nets Evaluating Safety and Territory
10.8 Evaluation of Honte
10.9 Conclusions
Chapter 11 - Learning to Play Strong Poker
Darse Billings, Lourdes
Peña, Jonathan Schaeffer, Duane Szafron
11.1 Introduction
11.2 Texas Hold'em
11.3 Requrirements for a World-Class Poker Player
11.4 Loki's Architecture
11.5 Implicit Learning
11.6 Explicit Learning
11.7 Experiments
11.8 Ongoing Research
11.9 Conclusions