Variational Calculus

Type of physical science: Mathematical methods

Field of study: Calculus

The calculus of variations addresses the problem of finding extreme values of expressions called functionals. It is an extension of the differential calculus, which seeks the extreme values of functions. Many problems in classical mechanics, including the motion of pendulums and the shape of soap bubbles, can be considered by means of the variational calculus, as can problems in electromagnetic theory, quantum mechanics, and relativity.

Overview

The calculus of variations is an extension of the ideas of the differential calculus. While the differential calculus is primarily concerned with the determination of extreme values (either maxima or minima) of functions, the variational calculus addresses the question of finding the extreme values of functionals. In its simplest form, a function can be viewed as a set of points that can be represented graphically as a curve or a surface. The words "function" and "curve" are here used interchangeably, although there are technical differences between the two. Similarly, a functional, in its simplest form, is an association between a function and a real number. For example, if one considers two points, A and B, in the plane, one can draw a continuous, unbroken curve that connects them. This curve is the graph of a function. A measure of the length of this curve is a functional; that is, associated with any such curve is a real number that represents its length.

It is reasonable to ask which, of all possible curves joining A and B, has the shortest length. The obvious answer is the straight line segment joining the two points, but a change in the functional under scrutiny may give no hint about the required optimal value. An example is to find the path that a particle would follow from point A to point B so that the elapsed time of descent is minimal when the only external force is gravity. The calculus of variations provides a framework for solving problems such as these.

The basic tool of the differential calculus is the derivative, which gives a measure of the rate of change of a function. The extreme values of a function can be determined by examining these points where the function either is not changing (the derivative is zero there) or is undergoing a possible abrupt change (the derivative does not exist there). A fundamental result in the calculus says that if a function has an extreme value, then it must occur at these critical values. Critical values where the derivative is zero are called stationary points, and the terminology originates with the idea that if the function represents distance as a function of time, then the derivative is a velocity (rate of change of distance with respect to time). When the velocity is zero, the particle is motionless, or stationary. An example is a ball thrown vertically, which has zero velocity at the peak of its trajectory. More complicated trajectories require more sophisticated analysis, but the candidates for extreme values are always obtained by means of the derivative.

The calculus of variations has an analogue of the derivative that is called the first variation of the functional. The concept is based upon the notion that although the explicit evaluation of the functional for many, if not all, of the admissible functions may not be possible, a comparison between the optimal solution and others can be made. An attempt to minimize the difference or variation between the possible candidate functions and the optimal solution can be made, and the ultimate goal is to make this difference zero. This optimal candidate function is said to be a stationary value for the functional, in analogy with calculus. An ingenious use of a technique from the integral calculus, called integration by parts, is then employed to shift the emphasis from evaluating the functional to obtaining a necessary condition for this variation to be zero. This latter condition is the Euler-Lagrange differential equation, which, when solved, identifies a candidate for the optimal solution for the functional under study. Solving the Euler-Lagrange equation is usually very difficult by itself, but there are at least three factors that complicate the situation.

The first of these is the variety of external conditions that must be satisfied. These usually arise not so much from the mathematics of the problem as from physical restrictions.

These are sometimes called constraints, and they can be as simple as a specification of where the endpoints must be or as complicated as allowing the endpoints to move about freely. Other constraints may even involve a limitation on the solution that itself takes the form of a functional.

A second factor is the identification of which type of functions are admissible candidates for the solution. If one places a priori requirements that are too severe on the candidates, solutions may not even exist. For example, the admissible functions must always be continuous (unbroken) curves, but it may be desirable to have solution curves that have no corners. A case in point is a hanging rope suspended from two points, which physically must be a smooth curve. A restriction like this is equivalent to prohibiting abrupt changes in the velocity of a particle. Higher dimensions, complicated geometries, and systems of functionals pose other problems.

The third factor--usually the most difficult problem--is showing that the candidate yields an optimal value for the functional under study. The analogue here is that of using the second derivative test in the calculus to distinguish maximum and minimum values. In the calculus of variations, there are tests that can identify weak extrema (simply continuous functions) and strong extrema (smooth functions). These are quite sophisticated, and research on this and other problems in the calculus of variations continues.

Applications

The variational calculus provides a powerful and elegant framework for analyzing many of the physical problems in optics, electromagnetic theory, and mechanics, both classical and modern. The foundation for these applications is Pierre de Fermat's principle that nature behaves in ways that are "easiest and fastest." Easiest implies that a minimum amount of energy is expended, and fastest indicates a minimal time problem. A few examples of how the variational calculus is used to explain physical phenomena follow.

The simplest minimal distance problem is to find the path that joins two points A and B in the plane so that the total length of the path is as small as possible. The arc length of a curve can be represented as an integral (essentially, a sum of infinitely many small quantities) of an expression involving square roots and the derivative of the curve. For plane curves, the minimal path is a straight line. If the points are on some surface, the minimal paths are called geodesics. If the surface is a sphere, the arcs of minimal length are great circle routes. As expected, the functional representing the distance in these latter cases will be more complicated, to accommodate the geometry of the surface.

An extension of this problem is obtained if one considers the rotation of a curve around some fixed line and seeks the curve that minimizes the surface area of the resulting solid of revolution. The shape of a soap bubble and the shape of a red blood cell can be ascertained from the solution of such minimal surface problems.

Other applications arise when the effects of external physical forces, such as gravity, are included in specifying the functional. Examples here include the minimum time problem for a particle in the plane descending from point A to point B under the influence of gravity alone.

This is the famous brachistochrone problem, and the solution curve is part of a cycloid. Other examples include the path that a ray of light follows through media with different indices of refraction. The crucial observation here is that the quantity that is minimized is not the total length of the path, but the time of transit. As a consequence, Snell's law for refraction of light can be obtained.

Further examples in physics can be found when the functional represents the energy of a system. A hanging, flexible chain, suspended from two points A and B in the plane influenced by gravity alone, takes the shape of a catenary. The quantity minimized here is the potential energy of the chain, or equivalently, the center of gravity is found at the lowest possible point. It is also possible to ascertain motions when external requirements of the system are given in advance. For example, if a pendulum is required to swing in an arc so that the elapsed time of each cycle is the same, then one has the solution to the tautochrone problem, which was actually used to design many large-amplitude pendulum clocks.

All the previous examples illustrate the elegance of explaining physical phenomena by means of minimizing an important parameter such as distance, time, or energy. One of the crowning achievements of mathematical physics is a much more sophisticated application of the ideas discussed above. Let the functional take the form of an integral (or summing up) of an expression called the action of the system. The action is defined to be the difference between the kinetic and the potential energy, and it is usually referred to as the Lagrangian function for the system. Fermat's idea of "easiest and fastest" becomes the problem of minimizing this action integral. In order to obtain such a minimum, the Euler-Lagrange differential equation must hold, and this dictates the form of the Lagrangian, which identifies the motion of the system. The resulting Euler-Lagrange equation is referred to as Lagrange's equation of motion.

In the absence of any external constraints, Lagrange's equations reduce to Sir Isaac Newton's laws of motion, and the analysis by the action integral does not present a new theory, since each set of equations can be derived from the other. It is in the presence of external constraints and in complicated geometries that the action integral method proves to be superior.

A significant impediment occurs if frictional forces are present, since these are not accounted for in the action integral method. Problems in physics (but not engineering) adapt easily to this approach.

In many problems in physics, it is advantageous to change the coordinate system to suit the geometry of the problem. For example, a change from rectangular to polar or cylindrical coordinates when studying fluid flow in a circular pipe is desirable, and a change to spherical coordinates in gravitational, electromagnetic, and atomic physics simplifies the mathematics.

Each problem generally has a natural setting in which the mathematics is most transparent, and the preferred coordinate system is called the set of canonical variables for the system. The surprising feature is that spatial coordinates are not always the best choice for describing the motion of a system. Sometimes it is more reasonable to use "generalized" coordinates, such as the components of momentum (which is a product of the mass times a velocity component).

While Newton's equations of motion depend on the choice of the coordinate system, the value of the Lagrangian, which is found from the action integral, is the same in all coordinate systems. In fact, the form of the Euler-Lagrange equation remains coordinate-free. This is of crucial importance in quantum mechanics and in relativity, where the usual rectangular coordinates do not even exist.

If the generalized coordinates include those of momentum, it is possible to define a new function, the Hamiltonian, which is a combination of the Lagrangian and these coordinates and which represents the total energy of the system. When considering the Euler-Lagrange equation in minimizing the action integral in this form, it can be shown that certain expressions must remain constant. These are mathematical results, yet the quantities that remain fixed are the physical quantities of energy, momentum, and angular momentum. Thus, the conservation laws that form the basis for much of Newtonian mechanics follow logically from the calculus of variations. In fact, there are many other conserved quantities that do not have direct physical meaning.

A stable equilibrium point occurs for a physical system when the potential energy is minimized, and the variational calculus provides a mechanism for identifying motions near such equilibria. The motions of vibrating strings, rods, and membranes, as well as buckling beams with the subsidiary conditions of how the ends of edges are fixed, are suited to analysis by the calculus of variations. The ability to incorporate conservation laws, external constraints originating from the geometry, and energy restrictions allows one to formulate and solve problems in elasticity and electromagnetism in a natural way. Relativity problems can be discussed by making appropriate changes in the masses when velocities approach the speed of light. Using complex variables and probability theory, it is possible to minimize an energy functional in the sense of quantum mechanics and obtain the Schrodinger equation for wave mechanics.

Finally, Newton's laws of motion can be derived from the Lagrangian and Hamiltonian functions with the calculus of variations, and vice versa. The latter are more general, since the coordinates need not be lengths, and they are much better adapted to complicated systems, even in the presence of external constraints.

Context

The earliest appearance of a problem in the variational calculus was the isoperimetric problem of Dido in Greek mythology, but the first mathematical treatment was most likely by Galileo in 1638, when he conjectured (incorrectly) the solution of the brachistochrone problem.

Pierre de Fermat in 1662 made the first attempt to use the calculus to solve a problem in optics, stating the principle of least action. Later, in 1696, Johann Bernoulli challenged others to solve the brachistochrone problem, not mentioning that he had already found a solution that used a variational approach. Solutions were submitted by Newton, Gottfried Wilhelm Leibniz, Johann's brother Jakob Bernoulli, and Guillaume F. A. L'Hopital. All except L'Hopital gave the correct solution as a cycloid.

It was the Bernoullis' solutions, however, that essentially initiated the subject, giving a general method for solving problems of this type. Prior to this, questions had always focused on finding the equations of motion for the trajectory of some object, such as the earth around the sun or the flight of a cannonball. Once the unique trajectory was found, questions concerning maximum height, maximum range, or total elapsed time of transit were considered. The difference now was in the nature of the problem. The task was to choose, from the infinitely many possible trajectories, the one that minimized (or maximized) a quantity such as the time of travel, the length of the path, or the total energy used.

Leonhard Euler, a student of Johann Bernoulli, realized the mathematical significance of a problem of this type and, in 1744, gave a necessary condition that a solution must satisfy.

Later in 1775, Joseph Lagrange introduced the concept of "variations," and Euler deferred to Lagrange's more elegant approach and named the subject the calculus of variations. Lagrange, however, saw more than the mathematical importance of this problem; he realized its physical significance as well. His monumental work, MECANIQUE ANALYTIQUE, introduced in 1788 the notion of generalized coordinates and exploited the concept that nature acts in such a way that the action of a system is minimized.

In 1835, William R. Hamilton extended Lagrange's ideas even further and developed Hamiltonian mechanics. By minimizing, or at least making stationary, an expression for the total energy, he derived the equations of motion for complicated systems. Both Lagrange and Hamilton made significant contributions because of their abilities to see intimate interconnections between theoretical physics and the mathematics in the variational calculus.

In the late 1800's, Karl Weierstrass gave conditions that guaranteed an extreme value for functionals, which gives a structure that is somewhat parallel to the differential calculus. In the twentieth century, David Hilbert essentially completed the analytical structure of the variational calculus and opened the way for more extensions using geometric approaches. One direction for such extension was by L. S. Pontryagin and his coworkers, who let the constraints take the form of differential equations. This area is known as control theory, and Pontryagin gave constructive methods for finding optimal solutions.

Principal terms

ADMISSIBLE FUNCTION: a curve that meets certain requirements, such as being unbroken or having no corners

BRACHISTOCHRONE PROBLEM: the problem of finding the path that minimizes the time of descent for a particle under the influence of gravity alone

CONSTRAINT: an external condition placed on a solution curve

DERIVATIVE: the rate of change of a function, or of one variable with respect to another

EULER-LAGRANGE EQUATION: an equation that places a necessary restriction on the solution of a variational calculus problem

FUNCTION: a relationship between two sets of numbers, graphically represented as a curve or surface

FUNCTIONAL: a relationship between a function and a real number, such as the length of its graph or the difference between its starting and its ending value

VARIATION: an approximate measure of the change in the value of a functional when the underlying function is changed

Bibliography

Bliss, Gilbert Ames. CALCULUS OF VARIATIONS. The Carus Mathematical Monographs. Chicago: Mathematical Association of America, 1925. A readable account of the variational calculus with emphasis on minimal distance, time, and area. The discussions of these problems and extensions are accessible to the layperson.

Courant, Richard, and Herbert Robbins. WHAT IS MATHEMATICS? London: Oxford University Press, 1967. Chapter 7 is a particularly well-written account of the minimal-surface and soap-bubble problems, along with other extreme problems in mathematics.

Gelfand, I. M., and S. V. Fomin. CALCULUS OF VARIATIONS. Englewood Cliffs, N.J.: Prentice-Hall, 1963. An introductory textbook on the subject with an explanation of problems along with physical motivations concerning Lagrangian mechanics.

Goldstine, Herman H. A HISTORY OF THE CALCULUS OF VARIATIONS FROM THE SEVENTEENTH THROUGH THE NINETEENTH CENTURY. New York: Springer-Verlag, 1980. The definitive account of the subject from its earliest conception to the twentieth century. Original discussions, interconnections, and historical anecdotes make this the best source for the subject. Extensive bibliography.

Struik, D. J. A SOURCE BOOK IN MATHEMATICS, 1200-1800. Princeton, N.J.: Princeton University Press, 1986. Originals and translations of the earliest problems of the subject and fascinating footnotes that contain excellent references. (See, especially, pages 391f.).