Annotation+of+Papers

This page contains some annotation about the read papers for reference.

[[http://www.informatik.uni-trier.de/%7Eley/db/indices/a-tree/o/Onta=ntilde
oacute=n:Santiago.html|Santiago Ontañón]], [|Kinshuk Mishra], [|Neha Sugandh], [|Ashwin Ram]: **Case-Based Planning and Execution for Real-Time Strategy Games.** [|ICCBR 2007]: 164-178===

The paper introduces an approach composed of two major parts.

The first one is called Behavior acquisition. When the experts play the game "Wargus", the system keeps track of what operation the experts conducted and asks the experts to annotate why they did so. From these information the system build up its case-based knowledge base.

The second part is an execution engine which could play the game with the acquired knowledge base. In this engine, the Plan Expansion Execution module maintain the states gathered from the game world. It also checks whether some goal turns to open state(means that it's ready to be achieved), then query the Behavior Generation module for the appropriate behavior at the current game scenario. The Behavior Generation module would try to match the best case

In the paper the author introduce a Behavior Reasoning Language. The declarative part contains 3 parts {goal, preconditions, alive conditions}; the procedural part contains 4 kind of structures {sequence, parallel, action, subgoal} or their combinations.

They used a partial plan tree. The plan node could be decomposed to sub-nodes. The relation between the sub-nodes could be sequential or parallel etc.

The current game statistical states are gather then used to match the previous cases in the knowledge base. After retrieving the behavior of previous case, the system would adapt the behavior with the current units and coordinate.

===Thomas R. Hinrichs, [|Kenneth D. Forbus]: **Analogical Learning in a Turn-Based Strategy Game.** [|IJCAI 2007]: 853-858===

The paper proposes to use the Hierarchical Task Network (HTN) planner with Simple Hierarchical Ordered Planner (SHOP) algorithm to create a AI player model to play the turn-based game Freeciv (a game similar to the Civilization). It focuses on the economic model of the strategy game.

First it discusses how to build up the relationship between the top-level goal and the sub-goals. Here different sub-goal individually could help the top-level goal to be achieved. Then in practice the author gives some clues on how to choose from these sub-goals and how to achieve these sub-goals. They include some planning techniques and other heuristic conditions.

They also propose to compare the "snapshot" before and after some typical actions to learn the qualitative and quantitative effects of the action. Somehow the learning process might act like the novice human player when he is learning to play the game.

[|Marc J. V. Ponsen], [|Héctor Muñoz-Avila], [|Pieter Spronck], [|David W. Aha]: **Automatically Acquiring Domain Knowledge For Adaptive Game AI Using Evolutionary Learning**. [|AAAI 2005]: 1535-1540

They found a representation of the game score and fitness, then use these to calculate the weight of different tactics. An evolution algorithm would be run on the initial solutions to find the better tactics on different running instances. The next step the system would try to extract the good tactic from the game running instances.

Avi Pfeffer, [|Ya'akov Gal]: **On the Reasoning Patterns of Agents in Games.** [|AAAI 2007]: 102-109

Under the assumption that the agents are all rational, the paper introduces 4 reasoning patterns(based on Bayesian Network) for a particular agent to decide what to do in a multi-agent environment.

[|Stefan J. Johansson]: **On using multi-agent systems in playing board games.** [|AAMAS 2006]: 569-576

The paper proposes a general system model to process as the board game's bot. Then they show this on two board game: Diplomacy and Risk. An extra bidding system for Risk.

The experiment of the system shows good performance. They concludes in the high branching board game, MAS is a good candidate solution.

[|Megan Smith], [|Stephen Lee-Urban], Hector Muñoz-Avila: **RETALIATE: Learning Winning Policies in First-Person Shooter Games.** [|AAAI 2007]: 1801-1806

The paper introduces a typical reinforcement learning implementation on First-Person Shooting Game (Domination mode). With some modification, the RL algorithm runs smoothly and efficiently. The state and action are combined directly as S x A. Each node (s,a) in S x A is attached a Q-value and then is updated by the algorithm. They show that the algorithm outperforms the HTNbot. But I think the experiment itself is not convincing enough… Maybe the result is concerning to the game winning mode somehow...

[|Manu Sharma], [|Michael Holmes], [|Juan Carlos Santamaria], [|Arya Irani], [|Charles Lee Isbell Jr.], Ashwin Ram: **Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL.** [|IJCAI 2007]: 1041-1046

The system is divided to three levels, which represent different granularities of the planning. The upper layer's action would be treated as goal by the layer immediately below. The system uses both the RL and CBR to do the online and offline learning. The implementation works well on Level 3 and Level 4 of Transfer Learning. They use the score as the measure.

[|Bikramjit Banerjee], [|Peter Stone]: **General Game Learning Using Knowledge Transfer**. [|IJCAI 2007]: 672-677

In the deterministic turn-based game, the author suggests a hierarchy to transfer general knowledge between different kind of game. It relies on the structure and the rewards(deterministic) to figure out similar condition between different kind of game. Then use the former knowledge to initialize the Q-value in the RL.

go back