Bayes' theorem

Bayes’ theorem, developed in the 1700s by English mathematician Thomas Bayes, is a fundamental concept in the field of probability theory. Probability theory is a branch of mathematics that examines the likelihood that certain events will occur. Probability theorists use this theorem to examine a situation with possible outcomes and try to calculate how likely it is that one particular outcome will happen.

Brief History

Bayes’ theorem is named after its creator, Thomas Bayes, a noted English theologian and mathematician. Born in 1702, Bayes embarked on a life of both religion and science that led him to become one of the major founders of the mathematical discipline of statistics.

His most impactful work in this area was “Essay Towards Solving a Problem in the Doctrine of Chances,” which was published posthumously in Philosophical Transactions of the Royal Society, two years after Bayes’ 1761 death. In his work, Bayes demonstrates a mathematical technique to estimate the likelihood of a particular event occurring based on events that have occurred previously.

His work revolutionized math studies of the era, although later generations of mathematicians found some weak points in his theories and revised them into new forms. However, even in the twenty-first century, mathematicians acknowledge the important role Bayes played in bringing about the field of statistics and modernizing understanding of probability.

Overview

Bayes’ theorem is an important concept in probability theory. Probability theory is an area of mathematics that deals with probability, or the likelihood, that a phenomenon will lead to a particular outcome. This phenomenon, usually a random event, will have more than one, and sometimes many, possible outcomes.

In probability theory, mathematicians use rules and equations to determine the chance that a possible outcome will occur. This chance is described in a numerical way, typically the amount of a particular kind of outcome divided by the total number of potential outcomes. For example, flipping a coin may have two possible outcomes: heads or tails. The probability of getting heads is therefore one out of two, which may be written mathematically as 1/2 or 0.5.

This calculation always leads to a numeric ranking between zero and one. Zero indicates that a result has no likelihood of occurring, and one indicates that a result is definitely expected to occur. Results cannot be higher than one because the particular outcome being calculated cannot exceed the total number of possible outcomes.

Mathematicians may use probability theory in different ways, the main two of which are theoretical and experimental. In theoretical probability, mathematicians reach their conclusions mainly through logical reasoning. In experimental probability, mathematicians use experiments, usually several repeated tests, to reach their conclusions about results and likelihoods.

Probability theory has several different areas of specialization. The area most related to Bayes’ theorem is known as conditional probability. This area focuses on events that may occur after an earlier event has already taken place. For example, it might study the possible outcomes of new government policies, such as calculating the likelihood of property values rising in a city if the mayor decides to bring in a new business.

Mathematicians use the equation P(A|B) to show conditional probability. The vertical line used in math and logic represents the phrase “such that” or “it is true that.” In this equation, P represents probability. A represents the possible outcome being calculated. B represents the important motivating event that has already taken place. Here, both A and B are independent events, meaning that the probability of one outcome is not dependent on the probability of the other outcome.

Using the previous example, A might be described as “property values rising” and B might be described as “the mayor brings in a new business.” Therefore, the equation might be read as: “the probability that property values will rise if it is true that the mayor brings in a new business.”

Bayes’ theorem expands upon this basic equation, using the same elements of P, A, and B. This theorem may be represented as (see equation 1). On the left is the original equation, “the probability of A occurring if B is true.” This equation is equal to the more complex equation on the right. The equation on the right shows “the probability of B occurring if A is true,” multiplied by “the probability of A occurring,” with the result divided by “the probability of B occurring.”

Complications may arise in the application of Bayes’ theorem that require adjustments to the equation. One complication is that one of the events may be a binary variable, meaning it can only have one of two results instead of several or many possible outcomes. Some common examples of binary variables are “on or off,” or “true or false.” If the A event in the equation were a binary variable, a new and more complicated expression of the equation becomes necessary: (see equation 2).

In this new equation, the ⁺ and ^– symbols denote the two possible results of the binary variable. To refer back to the earlier example, an A⁺ event might be “property values will rise.” In that case, the A^- event might be a single opposing result such as “property values will fall.”

Bayes’ theorem and related concepts in probability theory and statistical mathematics may be important in many industries and fields of study. Experts often apply the theorem in various medical fields, such as pharmacology, in which it might be used to study the effects of different drugs or treatments on people with different symptoms. Alternately, it might study the effects of various conditions on people’s health, such as studying whether a particular type of factory is likely to increase cancer rates among the people living nearby.

These concepts are also very useful in fields involving money or business deals. In finance, experts might use Bayes’ theorem or related models to analyze the chances that an investment will lead to profit, or that borrowers will repay their loans on time. Businesses may use these equations to analyze potential failures in new products and correct them before the products go to market. Casinos and other gambling establishments may use these concepts to set a performance level for slot machines so they can keep gamblers entertained but still create profits for the casino.

Bibliography

“Basic Probability.” Seeing Theory, seeing-theory.brown.edu/basic-probability/index.html. Accessed 3 Apr. 2023.

Brownlee, Jason. “A Gentle Introduction to Bayes Theorem for Machine Learning.” Machine Learning Mastery, 4 Dec. 2019, machinelearningmastery.com/bayes-theorem-for-machine-learning/. Accessed 3 Apr. 2023.

Hayes, Adam. “Bayes’ Theorem: What It Is, the Formula, and Examples.” Investopedia, 1 Mar. 2022, www.investopedia.com/terms/b/bayes-theorem.asp. Accessed 3 April 2023.

Joyce, James. “Bayes’ Theorem.” Stanford Encyclopedia of Philosophy, Sept. 30, 2003, plato.stanford.edu/entries/bayes-theorem/. Accessed 3 April 2023.

Loredo, Thomas J. and Robert L. Wolpert. "Bayesian Inference: More Than Bayes's Theorem." Frontiers in Astronomy and Space Sciences, vol. 11, 21 Oct. 2024, doi.org/10.3389/fspas.2024.1326926. Accessed 13 Nov. 2024.

O’Connor, J.J., and E.F. Robertson. “Thomas Bayes.” School of Mathematics and Statistics, University of St. Andrews, Scotland, June 2004, mathshistory.st-andrews.ac.uk/Biographies/Bayes/. Accessed 3 Apr. 2023.

“Probability Theory.” Cue Math, www.cuemath.com/data/probability-theory/. Accessed 3 Apr. 2023.

“Probability Theory.” Science Direct / Elsevier B.V., 2023, www.sciencedirect.com/topics/mathematics/probability-theory. Accessed 3 April 2023.

Taylor, Sebastian. “Bayes’ Theorem.” Corporate Finance Institute, 5 Mar. 2023, corporatefinanceinstitute.com/resources/data-science/bayes-theorem/. Accessed 3 Apr. 2023.