.
Genewein T, Leibfried F, GrauMoya J, Braun DA (2015). Bounded rationality, abstraction and hierarchical decisionmaking: an informationtheoretic optimality principle. Front. Robot. AI 2:27. doi: 10.3389/frobt.2015.00027
Our paper on an informationtheoretic optimality principle that leads to the emergence of abstractions and also hierarchies of abstractions was published in Frontiers in Robotics and AI as part of the research topic on Theory and Applications of Guided SelfOrganisation in Real and Synthetic Dynamical Systems.
At the core of the paper is the consequent tradeoff between large expected utility and low information processing cost (as measured by the mutual information). The optimization of this tradeoff leads to a quantitative framework for bounded rational decisionmaking, that is optimal decisionmaking under information processing limitations. In the paper we present an extension of the basic tradeoff to more complex architectures that involve more computational nodes. We find that this extension leads to nontrivial solutions that allow to design boundedoptimal decisionmaking hierarchies. In particular we present two architectures in the paper

Serial hierarchies: the serial hierarchy consists of a perceptual channel and an action channel both of which are limited in their information processing rate. The perceptual channel maps a world state to an observation or internal percept. Since the rate on the channel is limited, the mapping must be lossy or bijective, implying that a onetoone mapping between world states and percepts is not possible. Classically the perceptual stage is designed independently of the action stage and the goal of perception is to represent the true worldstate as faithfully as possible (given the perceptual limitations). By applying our optimality principle we find that perception should be tightly coupled to action and that the goal of the perceptual part is to extract the most relevant information about the worldstate such that the actionpart of the system can work most efficiently (given its computational limitations). The optimality principle yields a welldefined likelihood function for boundedoptimal perception and also answers the question how to design the actionpart of the system in a boundedoptimal fashion.

Parallel hierarchies: In a parallel hierarchical architecture there are two levels of computation: the upper level partially processes information about the worldstate and then narrows down the search space over possible actions. The lower level searches within the narrowed down search space for “good” actions in response to the worldstate. For instance, imagine that the optimal response is a certain parametersetting of the movementcontroller. The upper level narrows down uncertainty over good parameter settings with a distribution over the parameters  this distribution over parameters is often called a model. The lower level of the hierarchy uses the distribution imposed by the model (that is the reduced search space) and computes a policy that maximizes expected utility given the computational limitations of the lower level. The important question is: who designs the models (the distributions over parameters imposed by the models) and given some observations, which model should be used? By applying our optimality principle to parallel hierarchies we find principled, boundedoptimal answers to both questions. Importantly, the emerging twolevel architecture is shaped by the utilityfunction and the computational constraints of the agent. Additionally, since computational limitations are an integral component of the optimality principle, computational tractability of the solutions can be guaranteed. In the paper we also show how the twolevel hierarchy can be used to split up information processing load between both levels of the hierarchy  in the modelparameter example this means that uncertainty over the optimal parameter(s) is reduced in two steps: first on the upper level by selecting the correct model and then on the lower level by processing more information about the worldstate.
The paper also contains a unifying general case that includes the serial and parallel hierarchies as special cases and is a promising startingpoint for generalizing towards more complex architectures.
See the Research pages for more information on the informationtheoretic principle for bounded rational decisionmaking and its extension to decisionmaking hierarchies. The work presented in the paper is a followup on our earlier work on the formation of abstractions through information processing limitations and was partially presented in a talk at the seventh workshop on GSO.
The document under the download button is protected by copyright and was first published by Frontiers. All rights reserved. It is reproduced with permission.