Inverse Temperature | Tim Genewein

Research

For an up-to-date publication list see my Google Scholar profile, and all my research-related blog posts are tagged with #research. A short summary of my current research topics is below on this page.

Keywords: (amortized) Bayesian inference, meta-learning, AI safety, information theory, agent analysis, bounded rationality, rate-distortion theory.

Blog highlights

For these topics I have written overview posts or series of posts on my blog:

Current research topics

Two problems that I am very interested in are:

Both questions draw on a rich and complex discourse in philosophy, and more recently also in mathematics, psychology, and AI research. The first question mainly falls into the realms of epistemology, logic, and induction; with compelling formalizations via Bayesian inference, Solomonoff induction, and sequential decision-making. The second question is about the philosophy of language, ethics, and perhaps also about induction. Formalizations are less clear (thanks Wittgenstein), but the theory of computation, (algorithmic) information theory, and decision-theory seem like helpful tools in this context.

More concretely I am currently exploring how implicit meta-training leads to amortized Bayesian inference and more general algorithm distillation, and how this may be the main (computational) mechanism behind in-context learning. How to drive this mechanism to reliably specify tasks and desired behavior to a pre-trained model is an open question, including reliable knowledge of when the system "understands" these instructions (meaning that the instructions are received without ambiguity). Related is the question of how to deal with ambiguity and epistemic uncertainty, and how to modify meta-learning schemes and bring in other approaches (such as active learning on the agent's side) to do so.

Past research

In the context of AI Safety I have worked on analysis (gaining a computational understanding) and interpretability of current AI agents. The emphasis was on fundamental questions and methodology that are likely to be relevant for future, and more advanced AI systems. Work that came out of this research was heavily influenced by Pedro Ortega's forward-looking thinking and the members of the Safety Analysis team and includes a formal understanding of meta-learning via amortized Bayesian inference. We empirically confirmed this understanding in prediction and decision-making tasks and (more recently) on non-stationary distributions ("switching" sources) as well as general algorithmic sources. Using this understanding, we formulated alternative meta-learning schemes that lead to amortized systems that can naturally deal with epistemic uncertainty. We advocated for using causal interventions, rather than purely observational analysis of agent behavior; and formulated the delusions problem - a problem of causality when predictors are used for decision-making and falsely take their own actions as evidence for belief updates. Finally, we were interested in fundamental limitations of today's AI models, and found that formal language complexity (the Chomsky hierarchy) is highly predictive of what models (including transformers) can and cannot learn.

During my time at Bosch I was working on neural network compression (reducing the computational footprint of neural nets) and Bayesian Deep learning. Interestingly, the prior in a Bayesian neural net can be used to favor weight configurations that are easily compressible. We also found that ensembles of networks produced superior epistemic uncertainty estimates for active learning compared to computationally cheaper methods.

In my PhD I was interested in computational mechanisms that allow for structure learning, that is learning of higher-order statistical structure that is shared across a family of tasks and facilitates learning of new task instances. I investigated such learning in the human sensorimotor system (via virtual reality experiments) and formulated an information-theoretic optimality principle based on hierarchical lossy compression as a theoretical computational mechanism. Our lab was working on thermodynamic-inspired theories for bounde-rational decision-making and Bayesian models of human sensorimotor learning. See more on my PhD research pages.
📥 Download Thesis