The Hamiltonian is a function used to solve a problem of optimal control for a dynamical system.
It can be understood as an instantaneous increment of the Lagrangian expression of the problem that is to be optimized over a certain time period.
[1] Inspired by—but distinct from—the Hamiltonian of classical mechanics, the Hamiltonian of optimal control theory was developed by Lev Pontryagin as part of his maximum principle.
maximizes or minimizes a certain objective function between an initial time
defined at each point in time, subject to the above equations of motion of the state variables.
The solution method involves defining an ancillary function known as the control Hamiltonian
which combines the objective function and the state equations much like a Lagrangian in a static optimization problem, only that the multipliers
The goal is to find an optimal control policy function
, which by Pontryagin's maximum principle are the arguments that maximize the Hamiltonian, The first-order necessary conditions for a maximum are given by Together, the state and costate equations describe the Hamiltonian dynamical system (again analogous to but distinct from the Hamiltonian system in physics), the solution of which involves a two-point boundary value problem, given that there are
differential equations for the state variables), and the terminal time (the
differential equations for the costate variables; unless a final function is specified, the boundary conditions are
[4] A sufficient condition for a maximum is the concavity of the Hamiltonian evaluated at the solution, i.e. where
[5] Alternatively, by a result due to Olvi L. Mangasarian, the necessary conditions are sufficient if the functions
[6] A constrained optimization problem as the one stated above usually suggests a Lagrangian expression, specifically where
compares to the Lagrange multiplier in a static optimization problem but is now, as noted above, a function of time.
, the last term on the right-hand side can be rewritten using integration by parts, such that which can be substituted back into the Lagrangian expression to give To derive the first-order conditions for an optimum, assume that the solution has been found and the Lagrangian is maximized.
obeys For this expression to equal zero necessitates the following optimality conditions: If both the initial value
The latter is called a transversality condition for a fixed horizon problem.
Thus the Hamiltonian can be understood as a device to generate the first-order necessary conditions.
From Pontryagin's maximum principle, special conditions for the Hamiltonian can be derived.
[12] William Rowan Hamilton defined the Hamiltonian for describing the mechanics of a system.
, the so-called "conjugate momentum", relates to it as Hamilton then formulated his equations to describe the dynamics of the system as The Hamiltonian of control theory describes not the dynamics of a system but conditions for extremizing some scalar function thereof (the Lagrangian) with respect to a control variable
The associated conditions for a maximum are This definition agrees with that given by the article by Sussmann and Willems.
Sussmann and Willems show how the control Hamiltonian can be used in dynamics e.g. for the brachistochrone problem, but do not mention the prior work of Carathéodory on this approach.
[14] In economics, the objective function in dynamic optimization problems often depends directly on time only through exponential discounting, such that it takes the form where
represent current-valued shadow prices for the capital goods
In economics, the Ramsey–Cass–Koopmans model is used to determine an optimal savings behavior for an economy.
is the social welfare function, to be maximized by choice of an optimal consumption path
The maximization problem is subject to the following differential equation for capital intensity, describing the time evolution of capital per effective worker: where
yields Inserting this equation into the second optimality condition yields which is known as the Keynes–Ramsey rule, which gives a condition for consumption in every period which, if followed, ensures maximum lifetime utility.