Orthogonal Functions And Expansions

Type of physical science: Mathematical methods

Field of study: Calculus

Equations that describe how a physical system's physical parameters vary are often differential in nature. One of the most powerful methods of solving either ordinary or partial differential equations involves expressing the most general solution in terms of a linear combination of orthogonal functions.

Overview

"Orthogonality" and "expansion" are two mathematical terms that are used in a number of different fashions. First, in a simple geometric sense, one can define orthogonality for vectors.

In a plane, if two vectors are orthogonal to each other, there is a right angle between them. The orientation of the pair of vectors in the plane is immaterial apart from the requirement of a right angle between them. In a simple plane, coordinate axes are defined in such a way as to make advantageous use of orthogonality. In Cartesian coordinate systems, the x-axis and y-axis are perpendicular to each other. (Perpendicular is equivalent to orthogonal in the geometric sense, for spaces of dimension less than or equal to three, that is, one-, two-, or three-dimensional space.) In rectangular polar coordinate systems, a unit vector indicating the direction of increasing radial distance from the origin and a unit vector indicating the direction of increasing angle are always mutually perpendicular, although they are allowed to rotate about in the plane relative to stationary Cartesian x- and y-axes. In a three-dimensional space, geometric interpretation is still applicable for orthogonality of a pair of vectors, although the requirement for a right angle between the vectors is no longer restricted to one special plane.

The x-, y-, and z-axes are mutually perpendicular in a three-dimensional Cartesian coordinate system. In spherical coordinates, the radial unit vector and a pair of angular unit vectors are mutually perpendicular at all points in space.

Vector spaces can be generalized to spaces of dimension n, which can be greater than three. (In physics and abstract mathematics, one often employs an infinite dimensional space in problem solving.) Orthogonality can also be generalized to be of value in such an n-dimensional space, but the concept loses its geometric interpretation. Human beings do not have the sensory apparatus to conceptualize angles in spaces of dimensionality higher than three. In a generalized definition, a pair of n-dimensional vectors are said to be orthogonal if the sum of the products of their similar components (that is, the first component of the first vector times the first component of the second vector plus the second component of the first vector times the second component of the second vector, and so forth) vanishes. This is referred to as the "scalar" or "dot product" of two vectors, as defined both in a geometrical and analytical sense. A vector in three-dimensional space has a magnitude and a perceivable direction. The scalar product of two vectors A and B can be shown to be equivalent to the product of the magnitudes of both vectors and the cosine of the angle between them. That angle is often calculated by independently determining the scalar product in terms of the analytic definition involving the sum of products of like components and then dividing by both magnitudes to identify the angle's cosine.

Functions can be said to be orthogonal in a generalized sense in the same way as vectors. Orthogonality of functions arises quite naturally in the definition of a Sturm-Liouville problem. A Sturm-Liouville problem includes an ordinary linear differential equation of second order that can be written in a form that identifies three special functions of the same independent variable and a parameter known as the eigenvalue. Solutions to the differential equation depend upon the values that the eigenvalue parameter may assume. If the solution to a Sturm-Liouville problem is Y(x), where x is the independent variable, then the set {Y₁(x), Y₂(x), . . . , Y_m(x), . . .} contains eigenfunctions that correspond in a one-to-one mapping to the eigenvalues {n₁, n₂, . . . , n_m}, respectively. One of the specially identified functions, often referred to as w(x), is called the weight function. The eigenfunctions {Y₁(x), Y₂(x), . . . , Y_m(x), . . .} are said to be orthogonal with respect to the weight function w(x) on the interval of x for which the Sturm-Liouville problem is defined. Orthogonality in this case is mathematically expressed as the vanishing of the integral (over the interval in x) of the product of the weight function and Y_m(x) and Y_n(x) for all possible pairs of m and n, where m and n are not equal. This characteristic arises from the distinguishability of the eigenvalues {n₁, n₂, . . . ,n_m, . . .} for all m. With the eigenfunctions specified as orthogonal solutions, the most general solution to the Sturm-Liouville problem is a linear combination, summed over all m, of the individual eigenfunctions weighted accordingly to match the boundary conditions.

Like orthogonality, the term "expansion" can connote several somewhat different meanings. For example, the expression 1/- x) can be shown, by direct long division, to be equivalent to the infinite sum (1 + x + x² + . . . + xⁿ +. . .). This series expansion will converge for absolute value of x less than unity and diverge for absolute value of x greater than or equal to unity. This is an example of an expansion of a function of x about a chosen point, zero in this case. Brook Taylor demonstrated that any infinitely differentiable function of an independent variable x, f(x), can be expanded about a point a, in terms of polynomial powers of (x - a) and derivatives of f(x) evaluated at the point a. Such expansions are referred to as the Taylor series. If the point a is zero, then the series is called a Maclaurin series expansion. An example of a Maclaurin series is the exponential series, e^x or exp(x), which can be shown to be an infinite sum over n of terms of the form xⁿ/n! (which converges for all x).

Another mathematical meaning for expansion involves expressing any well-behaved function, defined on a particular interval, in terms of a set of functions that are orthogonal on the same interval. Such expansions can take on a variety of forms, such as polynomial series (finite or infinite sums of terms such as c_nxⁿ), exponential series (finite or infinite sums of terms of the form c_nexp(inπx/L)), where L is a characteristic length and n is a positive or negative integer), trigonometric series (finite or infinite series of the form a_ncos(nπx/L), and b_nsin(nπx/L), where L is a characteristic length and n is a positive integer or zero), or series in special functions such as the Bessel functions or the Legendre polynomials.

The coefficients of these series (listed as either a_n, b_n, or c_n) can be said to represent or define the function that was expanded over the set of orthogonal functions on the specified interval. It must be stressed that the coefficients (a_n, b_n, and c_n) will vary depending upon the interval over which the function to be expanded is defined.

Another meaning of expansion is related to the most general solution to an ordinary or partial differential equation. Any ordinary or partial differential equation has a number of linearly independent solutions dictated by the characteristics of the equation. Each linearly independent solution satisfies the differential equation. Thus, a linear combination of these linearly independent solutions is also a solution of the differential equation and as such represents the most general solution of the differential equation. The coefficients in such a linear combination are determined by the initial or boundary conditions.

Applications

The trigonometric set of functions {1, cos nx, sin nx (where n = 1, 2, 3, . . .)} on the interval [-π,π] forms the basis of the standard Fourier series representation.

Such representations can be generalized to include the interval [-c,c] and [0,c] upon which the set of sine and cosine functions are orthogonal. These trigonometric expansions are particularly useful in physical science. Large classes of physical phenomena can be understood in terms of superposition of trigonometric terms such as those found in a standard Fourier series.

In physics, the superposition principle is very powerful. For example, if a harmonic oscillator is subjected to a driving force F(t) that can be analyzed into a Fourier expansion in terms of sines and cosines of varying frequencies, the net response of the oscillator to the driving force is the superposition of the responses that the oscillator would have if the individual Fourier components of F(t) acted alone. As another example, consider wave motion in a material medium. No two particles can exist in the same space at the same time, but the net displacement of a material medium is the superposition of all wave disturbances that pass through the medium at any given instant. If the disturbance can be analyzed into orthogonal sets of functions, such as sines and cosines in the case of vibrations of a one-dimensional wave propagation or Bessel functions in the case of a circular membrane, then the net displacement is the superposition of the displacements resulting from each individual Fourier component. In both examples, a complicated wave motion is described as a combination of simpler wave forms (of different frequencies) with well-established properties.

The boundary value problem that described the transfer of thermal energy in a solid, from regions of warmer temperature to regions of cooler temperature via conduction, relates changes in temperature as a function of variations in time to changes in temperature as a function of variations in spatial position. Separation of variables--time and position--yields a pair of Sturm-Liouville problems connected by a shared set of eigenvalues. The spatial variation will be sinusoidal or cosinusoidal depending on the boundary conditions, and the temporal variation will be a decaying exponential function. Each mode, represented by the product of a spatial term and a temporal term sharing the same eigenvalue, is an independent solution of the partial differential equation and represents a potential simple mechanism for heat transfer in the solid. Any complex heat transfer can be expressed as a linear combination over all possible modes of heat transfer, the coefficients being determined by the nature of the complex heat transfer.

The transverse wave displacement of a string is governed by a differential equation that relates changes in transverse position as a result of variations in position along the length of the string to changes in transverse position as a result of changes in time. Separation of variables in this instance also leads to a pair of Sturm-Liouville problems that share the same eigenvalues.

The spatial problem will have eigenfunctions that are sinusoidal or cosinusodial in distance along the string. The temporal problem will have eigenfunctions that are sinusoidal or cosinusoidal in time. The product of the spatial eigenfunction and temporal eigenfunction for a given eigenvalue represents a potential mode of vibration that can be excited in the string. The response of a string to a complex-shaped pluck will be a linear sum over all possible modes of vibration. The coefficients must be determined in conjunction with the nature of the initial pluck and the boundary conditions.

In quantum mechanics, a given system will be allowed to exist in any of a set of finite or infinite discrete states specified by a special state vector or wave function. The wave function of the system at any one moment in time can be expressed as an expansion over all possible states. The coefficients in the expansion can be related to the probability that the system will be observed to be in a particular state.

Context

Mathematical innovation and progress in physical science have often gone hand-in-hand. Mathematics is the language of physical science. It was the need for expression of motion and its causes in the appropriate language that guided Sir Isaac Newton (1642-1727) to invent the theory of calculus. Gottfried Wilhelm Leibniz (1646-1716) independently derived the methods of basic calculus. European mathematicians began to apply the new techniques of calculus to interesting physical problems of their day. Problems in wave motion, sound vibration, and vibrations of strings, springs, and beams were shown to have similar characteristics. Efforts by some of the greatest mathematicians of the eighteenth century led to the understanding that, to explain wave motion, a boundary value problem, consisting of a partial differential equation and special initial or boundary conditions, had to be solved. The French mathematician Jean Le Rond d'Alembert (1717-1783), Swiss mathematician Daniel Bernoulli (1700-1782), English mathematician Brook Taylor (1685-1731), and Swiss mathematician Leonhard Euler (1707-1783) were instrumental in deriving the wave equation for vibrating strings and providing a framework for finding allowed modes of vibration.

Any arbitrary vibration was shown to be expressible as a linear combination of basic modes of vibrations by the principle of superposition. Using the orthogonality of special functions such as the trigonometric series, a formalism for expansions as the solutions to boundary value problems was developed. The French mathematician Joseph Fourier (1768-1830) extensively studied the use of the trigonometric series (when orthogonal on finite intervals) in boundary value problems of special significance to physics. Fourier's work was later generalized for use with any set of orthogonal functions, and that formalism is referred to as "Fourier analysis." Special functions, forming orthogonal sets, arose as solutions to Sturm-Liouville problems and certain partial differential equations from physics. Examples of such functions are the Legendre polynomials and Bessel functions.

Solution by expansion of orthogonal sets of functions has become standard practice in physics in problems where closed form solutions in terms of elementary functions are not possible.

Principal terms

FOURIER ANALYSIS: a transformation of functions that permits expression of any sectionally continuous function in terms of orthogonal functions such as the trigonometric sequence {sin nx} and {cos nx}

LINEAR INDEPENDENCE: a set of functions have linear independence if all members of the set cannot be expressed as a sum of multiples of the other functions in the set

ORDINARY LINEAR DIFFERENTIAL EQUATION: an equation whose solution must satisfy the differential equation and its initial conditions

ORTHOGONAL: geometrically, referring to a pair of vectors that are perpendicular to each other; can be extended beyond three dimensions and to vector functions that can represent physical quantities

PARTIAL DIFFERENTIAL EQUATION: an equation involving derivatives of a function of more than one independent variable; a solution must satisfy the partial differential equation and its boundary conditions

SUPERPOSITION: if a set of functions satisfies either an ordinary or partial differential equation, a linear combination of these functions also satisfies the ordinary or partial differential equation

VECTOR: geometrically, a mathematical quantity having magnitude and direction; can be abstracted to include definition in physical or mathematical spaces of a dimension higher than ordinary 3-D

Bibliography

Arfken, George. MATHEMATICAL METHODS FOR PHYSICISTS. Orlando, Fla.: Academic Press, 1985. A superb handbook of higher-level applied mathematics available to the theoretical physicist. Mathematically rigorous. Discussions of orthogonality of functions and superposition of functions are available to the advanced reader. Contains many examples of physics problems and their solution.

Butkov, Eugene. MATHEMATICAL PHYSICS. Reading, Mass.: Addison-Wesley, 1968. An excellent condensation of major applied mathematical techniques used by physicists in the solution of theoretical problems. Mathematically rigorous. Descriptions of orthogonality of functions available to the layperson with close attention to detail.

Churchill, Ruel V. FOURIER SERIES AND BOUNDARY VALUE PROBLEMS. New York: McGraw-Hill, 1969. Classic text on the solution to partial differential equations through the use of orthogonal functions and superposition of linearly independent functions. Contains several physics examples, such as heat conduction and wave propagation.

Halliday, David, and Robert Resnick. FUNDAMENTALS OF PHYSICS. 3d ed. New York: John Wiley & Sons, 1988. This version of a classic undergraduate physics text contains material beyond the introductory level. Contains a lucid explanation of the principle of superposition as applied to waves.

Pettofrezzo, Anthony J. ELEMENTS OF LINEAR ALGEBRA. Englewood Cliffs, N.J.: Prentice-Hall, 1970. Explains the concepts of orthogonality for vectors and functions. Explains linear independence of solutions in the general solution of eigenvalue problems.

Rainville, Earl D., and Phillip E. Bedient. ELEMENTARY DIFFERENTIAL EQUATIONS. New York: Macmillan, 1974. A classic text for a first investigation of ordinary linear differential equations. Thorough demonstration of the properties of orthogonality and linear independence of vectors and functions. Amenable to self-study for serious investigators.

Spiegel, Murray R. ADVANCED MATHEMATICS FOR ENGINEERS AND SCIENTISTS. New York: McGraw-Hill, 1971. An excellent workbook for self-study of higher applied mathematics of physics. Includes hundreds of solved problems.

Many problems at the end of chapters contain answers.

The Effect of Electric and Magnetic Fields on Quantum Systems

Sets and Groups