Numerical Analysis

Summary

Numerical analysis studies how to design, implement, and optimize algorithms that provide approximate values to variables in mathematical expressions. Numerical analysis has two broad subareas: first, finding roots of equations, solving systems of linear equations, and finding eigenvalues; and second, finding solutions to ordinary and partial differential equations (PDEs). Much of this field involves using numerical methods (such as finite differences) to solve sets of differential equations. Examples are the Brownian motion of polymers in solution, the kinetics of phase transition, the prediction of material microstructures, and the development of novel methods for simulating earthquake mechanics.

Definition and Basic Principles

Most phenomena of science have discrete or continuous models that use a set of mathematical equations to represent the phenomena. Some equations have exact solutions as a number or set of numbers, but many do not. Numerical analysis provides algorithms that, when run a finite number of times, produce a number or set of numbers that approximate the actual solution of the equation or set of equations. For example, since π is transcendental, it has no finite decimal representation. Using English mathematician Brook Taylor's series for the arctangent, however, one can easily find an approximation of π to any number of digits. One can also do an error analysis of this approximation by looking at the tail of the series to see how closely the approximation came to the exact solution.

Finding roots of polynomial equations of a single variable is an important part of numerical analysis, as is solving systems of linear equations using Gaussian elimination (named for German mathematician Carl Friedrich Gauss) and finding eigenvalues of matrices using triangulation techniques. Numeric solution of ordinary differential equations (using simple finite difference methods, such as Swiss mathematician Leonhard Euler's formula, or more complex methods, such as German mathematicians C. Runge and J. W. Kutta's Runge-Kutta algorithm) and partial differential equations (using finite element or grid methods) are the most active areas in numerical analysis.

Background and History

Numerical analysis existed as a discipline long before the development of computers. By 1800, Lagrange polynomials, named for Italian-born mathematician Joseph-Louis Lagrange, were being used for general approximation, and by 1900, the Gaussian technique for solving systems of equations was in common use. Ordinary differential equations with boundary conditions were being solved using Gauss's method in 1810, English mathematician John Couch Adams's difference methods in 1890, and the Runge-Kutta algorithm in 1900. Analytic solutions of partial differential equations (PDEs) were being developed by 1850, finite difference solutions by 1930, and finite element solutions by 1956.

The classic numerical analysis textbook Introduction to Numerical Analysis (1956), written by American mathematician Francis Begnaud Hildebrand, had substantial sections on numeric linear algebra and ordinary differential equations, but the algorithms were computed with desktop calculators. In these early days, much time was spent finding multiple representations of a problem to get a representation that worked best with desktop calculators. For example, a great deal of effort was spent on deriving Lagrange polynomials to be used for approximating curves. The early computers, such as the Electronic Numerical Integrator And Computer (ENIAC), built by John Mauchly and John Presper Eckert in 1946, were immediately applied to the existing numerical analysis algorithms and stimulated the development of many new algorithms.

Modern computer-based numerical analysis got started with John von Neumann and Herman Goldstine's 1947 paper “Numerical Inverting of Matrices of High Order,” which appeared in the Bulletin of the American Mathematical Society. Many new and improved techniques were developed for numerical analysis, including cubic-spline approximation, sparse-matrix packages, and the finite element method for elliptic PDEs with boundary conditions.

How It Works

Numerical analysis has many fundamental techniques. Below are some of the best-known and most useful.

Approximation and Error. It is believed that the Greeks and others developed the earliest examples of numerical analysis to find numerical quantities that were approximations of values of variables in simple equations. For example, the hypotenuse length of a 45-degree right triangle with a side of length 1 is √2, which has no exact decimal representation but has rational approximations to any degree of accuracy. The first three elements of the standard sequence of approximations of √2 are 1.4, 1.414, and 1.4142. Another fundamental idea is error (a bound of the absolute difference between a value and its approximation); for example, the error in the approximation of √2 by 1.4242 to ten digits is 0.000013563.

There are many examples of approximation in numerical analysis. The Newton-Raphson method, named for English physicist Sir Isaac Newton and English mathematician Joseph Raphson, is an iterative method used to find a real root of a differentiable function and is included in most desktop math packages. A function can also be approximated by a combination of simpler functions, such as representing a periodic function as a Fourier series (named for French mathematician Jean-Baptiste Joseph Fourier) of trigonometric functions, representing a piecewise smooth function as a sum of cubic splines (a special polynomial of degree 3), and representing any function as a Laguerre series (a sum of special polynomials of various degrees, named for French mathematician Edmond Laguerre).

Solution of Systems of Linear Equations. Systems of linear equations were studied shortly after the introduction of variables; examples existed in early Babylonia. Solving a system of linear equations involves determining whether a solution exists and then using either a direct or an iterative method to find the solution. Gauss developed the earliest iterative solutions of linear systems; newer iterative algorithms are still published. Many science problems are expressed as systems of linear equations, such as balancing chemical equations, and are solved when the linear system is solved.

Finite Differences. Many of the equations used in numerical analysis contain ordinary or partial derivatives. One of the most important techniques used in numerical analysis is to replace the derivatives in an equation, or system of equations, with equivalent finite differences of the same order and then develop an iterative formula from the equation. For example, in the case of a first-order differential equation with an initial value condition, such as f′(x) = F[x, f(x)], y₀ = f(x₀), one can replace the derivative by using equivalent differences, f′(x₁) = [f(x₁) − f(x₀)]/(y₁ − x₀) and solve the resulting equation for y₁. After getting y₁, one can use the same technique to find approximations for yi given xi for i greater than 2. There are many examples of using finite differences in solving differential equations. Some use forward differences, such as the example above; others use backward differences, much like antiderivatives, and others use higher-order differences.

Grid Methods. Grid methods provide a popular technique for solving a partial differential equation with n independent variables that satisfies a set of boundary conditions. A grid vector is a vector of the independent variables of a partial differential equation, often formed by adding increments to a boundary point. For example, from (t, x), one could create a grid by adding t to t and x to x systematically. The basic assumption of the grid method is that the partial differential equation is solved when a solution has been found at one or more of the grid points. While grid methods can be very complex, most follow a fairly simple pattern. First, one or more initial vectors are generated using the boundary information. For example, if the boundary is a rectangle, one might choose a corner point of the rectangle as an initial value. Otherwise, one might have to interpolate from the boundary information to select an initial point. Once the initial vectors are selected, one can develop recursive formulas (using techniques like Taylor expansions or finite differences) and from these, generate recursive equations over the grid vectors. Adding in the information for the initial values yields sufficient information to solve the recursive equations and thus yields a solution to the partial differential equation. For example, given a partial differential equation and a rectangle, the lower-left corner point of which satisfies the boundary condition, one often generates a rectangular grid and set of recursive equations that, when solved, yield a solution of the partial differential equation. Many grid methods support existence and uniqueness proofs for the PDE and error analysis at each grid vector.

Applications and Products

The applications of numerical analysis, including developing new algorithms and packages within the field itself, are numerous. Some broad categories are listed below.

Packages to Support Numerical Analysis. One of the main applications of numerical analysis is developing computer software packages that implement sets of algorithms. The best-known and most widely distributed package is LINPACK, a software library for performing numeric linear algebra developed at Stanford University by students of American mathematician George Forsythe. Originally developed in FORTRAN for supercomputers, it is now available in many languages and can be run on large and small computers. However, it has largely been superseded by LAPACK (Linear Algebra Package). Another software success story is the development of microcomputer numerical analysis packages. In 1970, a few numerical analysis algorithms existed for muMath (the first math package for microcomputers). By the twenty-first century, almost all widely used math packages incorporated many numerical analysis algorithms.

Astronomy, Biology, Chemistry, Geology, and Physics. Those in the natural sciences often express phenomena as variables in systems of equations, whether differential equations or PDEs. Sometimes, a symbolic solution is all that is necessary, but numeric answers are also sought. Astronomers use numeric integration to estimate the volume of Saturn a million years ago. Entomologists can use numeric integration to predict the size of the fire ant population in Texas twenty years from the present. In physics, the solutions of differential equations associated with dynamic light scattering can be used to determine the size of polymers in a solution. In geology, some of the Earth's characteristics, such as fault lines, can be used as variables in models of the Earth, and scientists have predicted when earthquakes will occur by solving these differential equations.

Medicine. Many of the phenomena of medicine are represented by ordinary or partial differential equations. Some typical applications of numerical analysis to medicine include estimating blood flow for stents of different sizes (using fluid flow equations), studying diaphragmatic hernias across many physicians (using statistical packages), and determining the optimal artificial limb for a patient (solving some differential equations of dynamics).

Engineering. Engineers apply the natural sciences to real-world problems and often use numerical analysis. For example, civil and mechanical engineering have structural finite element simulations and are among the biggest users of computer time at universities. In industry, numerical analysis is used to design aerodynamic car doors and high-efficiency air-conditioner compressors. Electrical engineers at universities and in industry always build a computer model of their circuits and run simulations before they build the real circuits. These simulations use numeric linear algebra, numeric solution of ordinary differential equations, and numeric solution of PDEs.

Atmospheric Modelling and Weather Forecast. Numerical analysis of fundamental atmospheric parameters, such as temperature, velocity, pressure, and concentration of various atmospheric gases at a given point in time, helps in weather forecasting. It helps create a climate model for the entire globe. This makes it an important tool for combatting the ever-growing concern for climate change. The atmospheric modeling generally includes computational fluid mechanics and numerical solution for differential equations.

Finite Element Packages. For some problems in numerical analysis, a useful technique for solving the problem's differential equation is first to convert it to a new problem, which agrees with the solution of the original equation. One of the most popular uses of the finite element has been to solve elliptic partial PDEs. In most versions of finite element packages, the original equations are replaced by a new system of equations that agree at the boundary points. The new system of differential equations is easier to solve “inside” the boundary than is the original system and is proved to be close to the solution of the original system. Much care is taken in selecting the original finite element grid and approximating functions. Examples of finite elements abound, including modeling car body parts, calculating the stiffness of a beam that needs to hold several different weights, and simulating the icing of an airplane wing.

Stock Market Analysis. Successfully predicting a future stock price for a company on a stock exchange can yield significant profits. Such algorithms can be broadly classified as single or double-source methods. The source can be a news article or raw numerical data. In the double-source method, a correlation between two sources is established to predict a stock's performance, such as retrieving context from a news article and correlating it to the company's numerical data.

Careers and Course Work

Students who enter one of the careers that use numerical analysis typically major in mathematics or physics. One needs substantial coursework in computer science, mathematics, and physics. A Bachelor's degree is sufficient for those seeking positions as programmers, although a Master's degree is helpful. For a position involving the development of new algorithms, one generally needs a Master's or Doctoral degree. A formal minor in an area regularly using numerical analysis, such as biology, is recommended.

Those seeking careers involving numerical analysis take a wide variety of positions. These include programmers, algorithm designers, managers of scientific labs, and computer science instructors.

Social Context and Future Prospects

Advances in numerical algorithms have made many advances in science possible, such as improved weather forecasting, and have improved life for everyone. If scientific theories are to live up to their full potential in the future, ways of finding approximations to values of the variables used in these theories are needed. The increased complexity of these scientific models is forcing programmers to design, implement, and test new and more sophisticated numerical analysis algorithms.

In the twenty-first century, many new and emerging fields use numerical analysis. Numerical analysis creates and optimizes artificial intelligence and model algorithms for machine learning and blockchain technology. In this way, numerical analysis plays a role in cryptography and cybersecurity. The modern uses of numerical analysis are vast and include creating climate models for environmentalists or financial models for those in the economic sector. Numerical analysis continues to play a role in the medical field, enhancing healthcare technology and allowing for the creation of predictive models of disease progression. Finally, numerical analysis can be used in virtual and augmented reality. 

Bibliography

Liu, Guang, and Xiaojie Wang. "A Numerical-Based Attention Method for Stock Market Prediction with Dual Information." IEEE Access, vol. 7, 2019, pp. 7357–7367.

Burden, Richard L., and J. Douglas Faires. Numerical Analysis. 9th ed. Boston: Brooks, 2011.

Hildebrand, F. B. Introduction to Numerical Analysis. 2nd ed. New York: McGraw, 1974.

Iserles, Arieh. A First Course in the Numerical Analysis of Differential Equations. 2nd ed. New York: Cambridge UP, 2009.

Kalra, Isha. “Why Do We Need Numerical Analysis In Everyday Life?” Science ABC, 1 Oct. 2020, www.scienceabc.com/eyeopeners/why-do-we-need-numerical-analysis-in-everyday-life.html. Accessed 5 June 2024.

Moler, Cleve B. Numerical Computing with MATLAB. Philadelphia: Soc. for Industrial and Applied Mathematics, 2008.

Overton, Michael L. Numerical Computing with IEEE Floating Point Arithmetic. Philadelphia: Soc. for Industrial and Applied Mathematics, 2001.

Orlando, S., et al. "Virtual Reality for the Analysis and Visualization of Scientific Numerical Models." ArXiv, 2023, doi.org/10.36116/MEMSAIT‗94n1.2023.13. Accessed 5 Jun. 2024.

Ralston, Anthony, and Philip Rabinowitz. A First Course in Numerical Analysis. 2nd ed. Mineola: Dover, 2001.

Strauss, Walter A. Partial Differential Equations: An Introduction. 2nd ed. Hoboken: Wiley, 2008.