Petter Ögren

Title of the project:
Using Behavior Trees to combine the efficiency of reinforcement learning with performance guarantees regarding safety and goal convergence

Background and summary of fellowship:
Behaviour Trees (BTs) represent a hierarchical way of combining low-level controllers for different tasks into high-level controllers for more complex tasks. The key advantages of BTs have been shown to include the following:

• Recursive structure: The BT is a rooted tree and at every edge of that tree, the interface between the parent and the subtree is the same, centred around a return status of either Success, Failure or Running.
• Modularity: Due to the recursive structure, a complex subtree can be seen as a single leaf, and vice versa. This enables the encapsulation of complexity.
• Transparency: The recursive structure of BTs makes them human-readable. You can always look at a BT and see why it is executing a particular behaviour. This fact in combination with the modularity described above enables a user to understand complex BTs by analyzing one subtree at a time.
• An efficient tool for human system design: BTs were created by computer game programmers to make their life easier when creating complex AI designs. Modularity is a well-known tool to handle complexity, and transparency is vital in any human design.
• An efficient tool for automated design. The modular recursive structure simplifies automated design.
• A structure that enables formal analysis of safety and convergence. Formal analysis of convergence and region of attraction is enabled by the modular recursive structure.

In this project, we will use the properties of BTs listed above to synthesize controllers that combine the efficiency of reinforcement learning with formal performance guarantees such as safety and convergence to a designated goal area.