.
Genewein T, Leibfried F, Grau-Moya J, Braun DA (2015). Bounded rationality, abstraction and hierarchical decision-making: an information-theoretic optimality principle. Front. Robot. AI 2:27. doi: 10.3389/frobt.2015.00027
Our paper on an information-theoretic optimality principle that leads to the emergence of abstractions and also hierarchies of abstractions was published in Frontiers in Robotics and AI as part of the research topic on Theory and Applications of Guided Self-Organisation in Real and Synthetic Dynamical Systems.
At the core of the paper is the consequent trade-off between large expected utility and low information processing cost (as measured by the mutual information). The optimization of this trade-off leads to a quantitative framework for bounded rational decision-making, that is optimal decision-making under information processing limitations. In the paper we present an extension of the basic trade-off to more complex architectures that involve more computational nodes. We find that this extension leads to non-trivial solutions that allow to design bounded-optimal decision-making hierarchies. In particular we present two architectures in the paper
-
Serial hierarchies: the serial hierarchy consists of a perceptual channel and an action channel both of which are limited in their information processing rate. The perceptual channel maps a world state to an observation or internal percept. Since the rate on the channel is limited, the mapping must be lossy or bijective, implying that a one-to-one mapping between world states and percepts is not possible. Classically the perceptual stage is designed independently of the action stage and the goal of perception is to represent the true world-state as faithfully as possible (given the perceptual limitations). By applying our optimality principle we find that perception should be tightly coupled to action and that the goal of the perceptual part is to extract the most relevant information about the world-state such that the action-part of the system can work most efficiently (given its computational limitations). The optimality principle yields a well-defined likelihood function for bounded-optimal perception and also answers the question how to design the action-part of the system in a bounded-optimal fashion.
-
Parallel hierarchies: In a parallel hierarchical architecture there are two levels of computation: the upper level partially processes information about the world-state and then narrows down the search space over possible actions. The lower level searches within the narrowed down search space for “good” actions in response to the world-state. For instance, imagine that the optimal response is a certain parameter-setting of the movement-controller. The upper level narrows down uncertainty over good parameter settings with a distribution over the parameters - this distribution over parameters is often called a model. The lower level of the hierarchy uses the distribution imposed by the model (that is the reduced search space) and computes a policy that maximizes expected utility given the computational limitations of the lower level. The important question is: who designs the models (the distributions over parameters imposed by the models) and given some observations, which model should be used? By applying our optimality principle to parallel hierarchies we find principled, bounded-optimal answers to both questions. Importantly, the emerging two-level architecture is shaped by the utility-function and the computational constraints of the agent. Additionally, since computational limitations are an integral component of the optimality principle, computational tractability of the solutions can be guaranteed. In the paper we also show how the two-level hierarchy can be used to split up information processing load between both levels of the hierarchy - in the model-parameter example this means that uncertainty over the optimal parameter(s) is reduced in two steps: first on the upper level by selecting the correct model and then on the lower level by processing more information about the world-state.
The paper also contains a unifying general case that includes the serial and parallel hierarchies as special cases and is a promising starting-point for generalizing towards more complex architectures.
See the Research pages for more information on the information-theoretic principle for bounded rational decision-making and its extension to decision-making hierarchies. The work presented in the paper is a follow-up on our earlier work on the formation of abstractions through information processing limitations and was partially presented in a talk at the seventh workshop on GSO.
The document under the download button is protected by copyright and was first published by Frontiers. All rights reserved. It is reproduced with permission.