WTF is the FEP? A short explainer on the free energy principle

Setting the scene 

In this blog post, we discuss the free energy principle (FEP): What it is, what it is not, what promise it holds, why it can be extremely useful, and why it has yet to live up to the hype. The FEP has received a great deal of attention recently. Building on musings by Erwin Schrödinger, it was proposed by neuroscientist Karl Friston as a computational theory of cortical responses, and then a bit later, as a unified theory of brain function. The FEP has grown in popularity in academia and, increasingly, also in industry and the popular press. Its domain of application has accordingly expanded, from its origins in computational neuroscience, to areas as varied as narrative psychology, cognitive anthropology, niche construction theory, immunology, and artificial intelligence. 

The FEP has many zealous proponents, as well as many detractors — and has been quite polarizing. To its adherents, the FEP is the best thing since sliced bread. For instance, in a WIRED article from 2019 by Shaun Raviv, the FEP was described as possibly the single most important and expansive scientific development since Charles Darwin’s theory of evolution via natural selection. The FEP is often touted as a solution to many of the outstanding problems in the empirical study of nature, the philosophy of science, and even metaphysics. The FEP has been proposed variously as a unifying and uniformly applicable empirical principle, a mathematical theory of “every” “thing”, as the foundation of a new class of physical theories, but also as a metaphysics of mind, as a theory of consciousness, as a formal theory of semantics. Obviously, it is difficult to see how all this can be true at the same time. Accordingly, to its detractors, it is an overhyped, messy body of work that provides post hoc explanations of data and does not live up to its lofty promises. 

So what is the truth of the matter? Is the FEP too good to be true — or too true to be good? 

An overview of the FEP

At its core, the FEP is a theoretically sound mathematical characterization of the flow of information in the presence of a boundary within a dynamical system. In other words, it falls under the rubric of mathematics or mathematical physics, in that it makes precise statements about the necessary properties of dynamical systems that can be sensibly partitioned into sub-systems or objects. In this respect, distilled to its core elements, the FEP has a scientific status much like that of statistical physics. That is, the FEP is a mathematical framework, not an empirically testable theory making specific predictions. It can be used to generate testable hypotheses, but is not an empirical theory per se. 

More precisely, the FEP uses information theory to describe the manner in which separable sub-systems of an overall system couple to one another and synchronize over time. The broad applicability of the FEP comes from this information theoretic formulation. This affords a compact, scale-free description of the microscopic dynamics of a system, i.e., the rules or laws of motion that govern its behavior over time, in terms of the behavior of the macroscopic objects that emerge to form both agents and the elements of environments. Indeed, the original formulation of the FEP was explicitly built from a consideration of statistical mechanics in the presence of boundaries that define macroscopic objects.

The FEP gets its name from a mathematical quantity called the free energy. This is a quantity that can be defined for any dynamical system and has the useful property that it monotonically decreases over time and can be linked to information theoretic quantities that are equivalent to generalized notions of heat, work, action, and entropy production. It can also be linked to message passing schemes for inference and learning. This is what allows the FEP to express the properties of interacting dynamical systems in information theoretic terms, recasting the interaction itself as a special kind of inferencing. This also allows us to interpret a dynamical system as if it was performing a computation, in the same sense that a computer is a dynamical system that performs computations.

Before reviewing it in more detail, we motivate the FEP conceptually, beginning with a few simple observations about observing and modelling objects. The first is that any collection of parts that can be usefully identified as a ‘thing’, ‘object’, or ‘sub-system’ must necessarily exhibit specific statistical regularities when observed, which are particular to that type of ‘thing’. For example, we designate a collection of particles (called quarks) as a “proton” because that set of particles has specific properties, such that they behave in specific, stereotyped ways that differ from collections that we label "neutrons" and "electrons". This is a truism.

The second concerns the use of the term “statistical regularity.” Generically, statistical regularities are associated with a reduction in the entropy of our observations — and usually are modeled via dimensionality reduction. That is, if a given system that we observe has some statistical regularity, then the entropy of observations of that system is lower than it would be without any regularity. In other words, the reason that we label something as a thing is that it reduces our surprise when observing its behavior. This too is a truism. By correctly labeling a particle as an electron or proton, we enhance our ability to predict its behavior relative to an unlabeled particle — good labels enhance prediction and hence lower the entropy of future observations.  

The third is perhaps less obvious — but just as essential, and forms a core part of systems identification theory and reinforcement learning. It is that an object, in a particular environment, can be formally defined entirely in terms of an input-output relationship — or equivalently a “boundary” that comprises both inputs and outputs. This is because a description of how an object affects its environment, and how the environment affects it, provides a complete environment-specific description of the typical behavior of such objects. (This dovetails nicely with Rovelli’s relational approach to empirical enquiry and empiricism.)

This leads naturally to the conclusion that one can meaningfully define object types in terms of the statistical regularities that obtain at their boundary and that good descriptions are those that maximally reduce entropy. In a nutshell, the FEP provides a mathematical treatment of this conclusion.  

Deriving the FEP from first principles

How does the machinery of the FEP work, more specifically? In its original formulation, the information theoretic quantities and relationships that characterize the FEP were derived from a consideration of the equations of statistical physics in the presence of a boundary. More generally, the FEP can be derived solely from information theoretic considerations by starting from Jaynes’s principle of maximum entropy or maximum caliber, under the constraint that there are boundaries of certain kinds that segregate the system into sub-systems. The principle of maximum entropy (or maximum caliber, when applied to trajectories) is a canonically sound framework for modelling, as well as generating, testing, and iterating on hypotheses. It is canonically sound in the sense that it can be used to generate the least biased model consistent with the set of constraints imposed by observation of some data.  

An explicit consideration of the boundaries of sub-systems is the main difference between standard applications of maximum entropy/caliber and the FEP. In fact, one might even simply say that applying the FEP is just maximum caliber with explicit (and perhaps multiple) boundaries. In a more traditional application of the methods of statistical physics, one only ever considers a single system, and everything else — “the rest of the universe” — is modeled as a heat bath with which the system exchanges energy and entropy. While useful in many settings, this idealization is clearly inadequate when we want to formally describe many separate and interacting sub-systems in terms of the information flow between them. The problem with a heat bath assumption is that, by construction, heat contains no information, so the transfer of heat from environment to object does not transfer any information about the environment to the object. 

We believe that the best way to appreciate the generality of the FEP is to derive it directly from information theory — that is, by starting from well established first principles in information theory, and deriving the FEP as a consequence of those first principles when boundaries are present. Note that this is not how the FEP is typically derived or motivated. As we said, in more standard treatments of the FEP, the derivation proceeds in the other direction: One starts from the equations of statistical physics (e.g., from the Langevin or Fokker-Plank equation) and from there, one recovers an equivalent description in information theoretic terms (as a gradient flow over a free energy functional). That is, FEP derivations typically assume that a dynamical system can be described via the Langevin or Fokker-Plank formalism, and then derive an equivalent information theoretic description of that system. 

In contrast, we leverage the fact that all principal physical theories — classical, statistical, quantum — can be derived from a single information theoretic principle, namely, maximum caliber, and all lead directly to a characterization via a gradient flow on free energy. That is, we can begin from the maximum entropy formulation, applied to paths of a system (i.e., from the maximum caliber formulation). From there, it can be shown that we can easily recover a free energy functional that can be optimized to simulate the time evolution of a dynamical system.

This responds to the criticism that the FEP is contingent on the validity of applying specific physical theories, and accordingly is limited by the assumptions that they make. The situation is far simpler because one can go about the equivalence in the other direction by deriving the FEP starting from a standard information theory and adding boundaries. This highlights the generality of the FEP, but also opens it up to the standard criticism that it is not an empirical theory. We want to be clear. This is a valid criticism. The FEP is not an empirical theory. It is a widely, perhaps even uniformly, applicable mathematical framework armed with an objective function for optimization. It is called a principle precisely because it comes with a modeling conjecture, like the principle of least action or Fermat’s principle that light takes the shortest path. 

From objects to constraints

With these preliminaries in place, the FEP itself can be derived by considering the minimal information theoretic description of an object via a minimal description of its boundary. The general idea is that a sub-system or object of a certain type exists in a dynamical system if the system evolves over time such that it satisfies type specific constraints on the statistics of the boundary, i.e., constraints on the composition and time evolution of the boundary. 

In this approach, the existence of a boundary with a specified set of statistical regularities is translated into a set of optimization constraints that, once imposed, determine the dynamics of the states inside and outside the boundary. In a maximum entropy setting, constraints are usually implemented by setting the expectation value of the function that describes the system’s dynamics to be equal to a specific quantity or have a specific relationship, which is usually determined directly from the data that we are trying to model. For example, a pendulum is constrained to always be a particular distance from the origin. These constraints are then imposed via Lagrange multipliers.  Constraint functions and Lagrange multipliers uniquely determine the energy function of the system as a whole, which can then be used to generate its temporal evolution. Applying maximum caliber, the end result is a probabilistic model of the system that is as agnostic as it is possible to be with respect to mechanism, while also satisfying the constraints that are imposed by the data. 

What kinds of constraints are these? Different types of objects emerge from different kinds of constraints which, in turn, lead to different energy functions. For example, static geometric constraints on subsets generate what we interpret as solid objects, while stationary (time independent) constraints lead to systems that conserve energy. In the FEP, these constraints are constraints on the composition and statistics of the boundary itself and specify subsystems or objects and their types. 

From constraints to energy-based models 

The nice thing about defining objects in terms of constraints on dynamics is that we can easily derive energy functions describing the time evolution of such objects. The reason this all works is precisely because the principle of maximum entropy or caliber can be used to derive Hamiltonian and Langevin dynamics from a given set of constraints. What comes out of this theoretical formulation is a new set of physical laws that characterize the interactions between internal and external states (mediated by the boundary) that are minimally sufficient to give rise to the particular kind of object characterized by the boundary constraints. 

So the idea is that if we can characterize a sub-system with a free energy functional, then all we have to do is minimize the associated free energy — and call it a day. This is mathematically true. The questions then become: What specific free energy functional should I use? Here, the FEP is largely agnostic and the choice of constraints is left to the modeler. This is why we prefer to characterize it as a framework and not a theory.

So the idea behind the FEP is that we can define this special mathematical quantity that has very useful properties when used as an objective function. Machine learning requires an objective function, and free energy functionals constitute a class of objective functions that are both grounded in information theory and applicable to seemingly any physical system. But free energy is not the only quantity with this property. What’s so special about free energy as an objective function, anyway?

Ultimately, the answer is that even though free energy was initially defined as an information theoretic quantity, it is also a measurable physical quantity that has an intimate relationship to thermodynamics, the production of entropy, the dissipation of heat by work, and (via the FEP) object identity. It is important to note that many of these relationships were not explicitly derived within the FEP literature. Physicists working at the boundary of information theory and statistical mechanics have done amazing work elucidating these relationships for isolated systems. In particular, Niven (2010) derived a generalization of the free energy functional used in physics and its relationship to generalized notions of heat and work. González and Davis (2016) showed how Jaynes’s principle of maximum caliber can be used to derive Hamiltonian and Langevin dynamics. Their work directly connects the information theoretic notion of free energy from information theory with the thermodynamic notion of free energy and explains why self organizing systems can be described in terms of a conversion of energy into entropy. This is precisely because constraints lead to energy functions. Heuristically, the idea is that in order to maintain a constraint, we have to be pulling in energy (typically from the heat bath — more generally, from the environment) for the purposes of maintaining the constraint, at the cost of exporting entropy to the heat bath or environment. It is also worth noting that the idea that free energy minimization is effectively the force that drives self-organization can be traced back all the way to Erwin Schrödinger’s seminal book What Is Life? (As an aside, that curious little book is worth a read, if for no other reason than to enjoy his rationalization for why he chose to frame it in terms of free energy instead of negative entropy.)

In summary, the starting point for the FEP is equivalent to maximum caliber with boundary constraints. This formulation enables us to answer the question: How do internal and external states have to interact in order to make this kind of object occur — in other words, what kinds of macroscopic forces have to be in play? What kinds of energy and information flows need to be in place to enable the emergence of a specific type of object? 

A “theory” of “every” “thing”?  

So where does this leave us? Does the FEP live up to the hype? In our view, some of the criticism of the FEP is valid, and has to do with how the FEP has been presented in the literature — as well as a certain penchant for rhetorical excess on the part of its proponents. 

The FEP has been described in many, sometimes contradictory ways. The FEP has even been hyped up as a “theory” of “every” “thing”. In this telling, the FEP is decidedly not a theory, in the same way that Jaynes’ principle of maximum entropy is not a theory. It is what it states: A principle. It is a mathematical modeling framework that comes equipped with an objective function. Like any other mathematical principle, it is used to generate theories via the imposition of data driven constraints. The FEP has been criticized for failing to make specific empirical predictions. This criticism is warranted. Despite how it was originally presented in the literature, the FEP is simply not the kind of thing that makes quantitative predictions about data on its own. Indeed, the FEP is not an empirical theory at all — let alone one that explains everything. 

Rather, the FEP is what it says on the tin: it is a mathematical principle. It is a principle because it specifies an objective function — free energy — to optimize in order to generate a model of the time evolution of a system composed of specified sub-systems. Taken as the source of this optimization objective, the FEP is the basis of a nearly universally applicable mathematical modeling method for stochastic dynamical systems where the goal is to partition the system into sub-systems. In this respect, the status of the FEP is far less problematic than might seem if one was under the impression that it was meant to be an empirical theory. Its scientific status is unambiguously the same as that of other mathematical principles used in statistical and physics modeling, such as the principle of stationary action or the principle of maximum caliber. 

So what is the value add of the FEP? This deflationary take on the FEP might seem a bit dissatisfying, since it seems to imply that the FEP doesn’t add much to maximum entropy modeling. If they’re equivalent, then why bother with the FEP at all?

At the end of the day, the FEP provides a unique (and uniquely useful) recipe for defining objects and object types. In this setting, as we discussed, an object or object type is precisely defined by the statistics of its blanket or boundary. This arguably provides the only sensible definition of an object and object type available within the maximum entropy framework. Given this definition, the FEP tells us the specific conditions that allow for the emergence of a specific type of object in a given environment and to characterize the flow of information between them. The utility of this approach is to significantly reduce the space of all possible models under consideration to an infinitesimal subset of the original set. 

Moreover, because the FEP is derived from information theory, it allows us to ask and answer additional questions like: What is the minimal amount of information that an object or organism must have about its environment in order to function? This naturally pops out of the combination of a boundary-based definition of object type and the maximum caliber objective that underwrites the FEP. From a computational modelling perspective, this property has great value because it establishes the nature of the computations that an organism must perform or what it must know about its environment in order to survive. Indeed, it not only tells us what an agent has to know about its environment, it also tells us how an organism will go about seeking out the information that it needs in order to continue its existence. 

Finally, because the FEP comes equipped with an information theoretic objective function, it provides a normative replacement for the reward function in reinforcement learning. In reinforcement learning, reward motivates behavior and must be specified by the user. In the FEP, the imperative to optimize an arbitrary reward function is replaced by the imperative to minimize free energy under the constraint that has a particular kind of boundary. This eliminates the need for an explicit reward function. This is great, because the notion of “reward” makes no sense for a physical system like a pendulum or an electron. It is true that we often talk about physical objects as “seeking a low energy state”  — but of course, they do not really seek anything. They just behave. Accordingly, in physics there are no reward functions. Instead, there are (free) energy functions that can, at least in the ideal case, be defined for objects, enabling us to model their behavior over time. This is how the FEP allows us to apply the tools of normative physical modeling to “goal-directed” behavior generally. 

So, if we were to engage in a little rhetorical excess of our own, we could say that the upshot is a naturalization of teleology, allowing us to overcome the anthropocentric metaphors of reinforcement learning by identifying an “ontological potential function” that defines an object type in the same way constraints define a given physical system. 

In closing, what is perhaps most unique about the FEP is that it is nearly universally applicable. The FEP is in some sense “inevitable” in that it is a direct consequence of describing the interactions between sub-systems of a physical system using first principles from information theory. That is, the FEP ultimately provides statements about our descriptions of dynamical systems that can be partitioned into subsystems and interpreted as performing a computation. Namely, it says that we can always model a dynamical system that has a boundary as estimating the parameters of the sub-systems to which it is coupled. This nearly universal scope of the FEP follows from its grounding in information theory and the indubitable mathematical fact that it is always possible to re-describe any dynamical system using this framework, that has the appropriate blanket structure using this framework.