.
Genewein T (2014) An Information-theoretic Optimality Principle for the Formation of Abstractions. Seventh international workshop on guided self-organization, Freiburg.
I had the chance to present my recent work on an information-theoretic optimality principle for the formation of abstractions with a talk at the seventh workshop on guided self-organization in Freiburg. In the talk I presented the information-theoretic framework for bounded rationality that trades off large expected utility against low information processing cost (see the Research pages for more information or alternatively this paper).
The central idea of the optimality principle is that any change in behavior incurs computation and in an agent with limited computational capacity this computation is costly. The cost of computation should thus be traded off against achieving a high expected utility. Interestingly, the resulting principle is identical to the rate distortion problem (the information-theoretic framework for lossy compression) and has strong formal ties to free energy minimization in thermodynamics. The problem in lossy compression is essentially the same as when forming abstractions: relevant information must be separated from irrelevant information (noise). The rate distortion principle for bounded rational decision-making has a parameter that governs the trade off between utility and information processing cost and we could show that changing this parameter leads to the emergence of natural levels of abstraction.
At the end of the talk there is a short outlook on how to extend the basic trade off to more complex systems - in particular perception-action systems. The results of this extension are published here.