Statistics

Type of physical science: Mathematical methods

Field of study:Probability and statistics

Statistics is the field concerned with methods based on mathematical theory and probability that allow scientists to summarize many observations concisely. It also tests hypotheses about conditions within or between populations or universes from which these data were drawn.

Overview

As a plural noun, "statistics" refers to numbers, such as the number of individuals living in a certain city. When used as a singular noun, statistics refers to the methods used to summarize, analyze, and draw conclusions from numerical data. These methods make sense out of the apparent chaos of many individual observations. This article discusses the singular definition of statistics.

Some consider the field of statistics to be both an art and a science, similar to the practice of medicine. As in medicine, the statistician often must draw conclusions from a set of data or diagnostic symptoms that are often incomplete, vague, or represent "too small a sample."

Statistical methods can be used by all who deal with numerical data. Statistical theory and techniques are the same for all fields of study.

Statistical activity can be divided into two areas: mathematical statistics or theoretical statistics, and applied or practical statistics. Theoretical statistics deals with the mathematical foundations of methodology. This study of abstract mathematics provides the tools used to attack practical problems. Theoretical or abstract statistics is often an end in itself.

To the applied or practical statistician, statistics is a means to an end. It is a kit of tools or the technology of the scientific method. Solving problems in statistics involves three steps: definition of the problem, collection of the data, and data analysis. How the data are collected and the units of measure used for these data determine how these observations are analyzed and summarized by the statistician.

Five sampling designs are available to use to collect data from a population. Random sampling is the most common type used to collect data. It assumes that all individuals within a population or universe have an equal chance of being selected in a sample. If one hundred iron rods were present and ten were to be selected to measure their tensile strength, each rod should have a one-in-ten chance of being selected for testing. Stratified random sampling is the procedure used when the population being sampled is not uniform. The population is then separated into strata, which have similar characteristics, and a portion of the total sample to be taken from the entire population is randomly taken from within each strata. If strata cannot be established and then stratified, random sampling cannot be used. If the strata are properly drawn, stratified random sampling will provide a more precise estimate of the population parameters than will simple random sampling with the same sampling effort. Perhaps the iron rods of the example come from two different foundries, one of which produces better iron than the other.

The rods would be separated, based on which foundry they were produced by, into two strata.

Stratified random sampling would provide a better estimate of the overall tensile strength of the average iron rod for all the iron rods than, perhaps, would simple random sampling.

Systematic sampling is another way one might collect information on the tensile strength of the iron rods. Assume all one hundred rods are in a pile; every tenth rod would be selected that was picked up as the one that would be used to determine tensile strength. Although easier to do, systematic sampling does not meet the mathematical assumptions needed to provide unbiased descriptive statistics and should not be used in sampling. Cluster sampling requires that many samples are taken once the initial area or sample site is located. This type of sampling, used when getting to where the data are to be collected, such as on a mountain top, is very difficult. Once on top of the mountain, many samples of whatever was being measured would be taken. Double sampling is the type of sampling used when there exists a linear relationship between previous data that have been collected and data that are now needed. An example of double sampling would be estimating the population of the United States in 1990 based on data collected from 1980.

Four classes or types of measurement are used to describe data. Nominal measurement classifies only the observations into groups or categories. For example, objects could be classified by color and counted; data could be reported as 34 red, 49 blue, and 115 white crystals in a growth chamber. Ordinal data are those grouped by size into groups, but no absolute value is placed on the differences between the groups. Snow crystals could be counted and placed into one of three groups: small, medium, and large. Crystals in the small group are clearly smaller, but are not differentiated by actual, measurable units from those placed in the medium or large categories.

Interval measurement units have real differences between the units. Commonly, things such as length, 10 millimeters versus 20 millimeters, indicate a real difference of 10 millimeters between the two measurements. Ratio measurements are created by dividing one number by another to create a single number.

The assumptions or parameters required of each statistical method must be satisfied for the statistical procedure to provide an unbiased analysis of the collected data. Descriptive statistics organizes the individual observations and provides a concise summary of their character. Any set of data may provide descriptive statistics. The arithmetic mean and standard deviation are examples of descriptive statistics that may be compiled from a set of data.

Inferential statistics test hypotheses about the population or universe from which the sample was drawn and often contrast that universe with other populations or universes. Statistical techniques allow scientists to measure the strength of their conclusions in terms of probability. The methods chosen to test hypotheses are often called either parametric or nonparametric statistical methods.

Nonparametric statistical methods can be used on all measurement categories. Parametric statistical tests can be used only on those data having interval or ratio measurements, unless the nominal or ordinal measurements are mathematically transformed. If the data are collected and analyzed properly (that is, the assumptions of collection and analysis by a specific statistical method are met), then the statistics cannot lie. If the parameters are not met, then the statistician can make a mistake regarding what the data really mean.

Perhaps the most unique feature of twentieth century applied mathematics has been the invention and development of electronic computers. The growth of statistics, particularly in the applied area, has been phenomenal, and the development of statistical programs or packages has taken the drudgery out of solving problems. Statistical packages often available on mainframe and personal computers include BMPD, Minitab, SAS, and SPSS-X.

Applications

The notation of statistical works is often difficult to follow, but convention provides the reader with some guidelines. Usually, true value for the universe is represented by a Greek letter; the population parameter by a capital letter; and the sample or estimated value by a lowercase letter. For example, the Greek letter mu (μ) represents the actual mean (arithmetic average) for the universe, capital X with a bar over its top (X̅) as the mean for a population, and a lowercase x with a bar over its top (x̅) as an estimate of the mean based on data collected by random sampling from that population or universe.

Any set of data will have descriptive statistics computed for it. For interval data, these regularly include a mean, median, mode, range, standard deviation, standard error of the mean, a confidence interval for the estimate of the mean, and a coefficient of variation. The individual data points may be graphed to form a scatter diagram or a histogram to show the general shape of the distribution or frequency of the data points. Assumptions may be made about the shape of this frequency distribution, and these hypotheses can be tested using different statistical methods to see if, in fact, the graphed data do meet the hypothetical distribution proposed for that sample of data.

Besides the type of measurement that determines what statistical method is appropriate for use, the number of populations or universes being compared also determines the selection of a specific statistical "test" for use. If the assumptions for a parametric statistical method are thought not to be met, then an alternative nonparametric test may be used to compare these sets of data. Nonparametric tests generally require a greater difference among the sets of data being compared to reject the null hypothesis than are necessary for the difference needed by a parametric statistical test to reject it at any given probability level.

For the comparison of two samples, the t-test is often used. The t-test may compare a mean calculated from sample data with either the universe or a population mean, with another independent mean, or with related means. Related means are those that produce sample values from the same unit in a pre- and postmeasurement situation. If the calculated t-value exceeds the tabular value expected by random chance given in the student's t-table, then the null hypothesis of equality will be rejected. The Mann-Whitney U-test is a nonparametric statistical test that may be used if the assumptions required of the t-test are not met. Often, the t-test requirement of equal variability among samples is not met, so the Mann-Whitney test is often preferred over the t-test. Interval measurement must be used to use the t-test.

Nominal data—those observations placed in different categories—may be tested using the chi-square or G-test procedures. Theoretical distributions can be tested against actual observed frequencies of some category. Ratios of occurrence of some character may be tested or ratios between two groups may be also compared to determine if the ratios are the same. This latter is usually known as row by column (R x C) contingency table analysis.

If more than two sets of data are to be compared, again assuming interval-type measurements or proper transformation of the measurement data, then a field of statistical testing known as analysis of variance is often used. The calculated test statistic is the F-value, and its calculation involves some complex calculations, unless a suitable computer program is available for use. The actual application of treatment levels to the experimental groups is called experimental design. The experimental design used determines the actual procedures used to calculate the F-value. Some types of experimental designs that may be used are: completely randomized design, block design, latin square, and split-plot applications. The nonparametric alternative for the randomized design is the Kruskall-Wallis test, while the Friedman test is a possible nonparametric alternative for the randomized block experimental design.

Analysis-of-variance statistical methods are often used where the investigator can control what is happening, either in the laboratory or in field plots, such as agricultural test stations.

The relationship between two variables, usually designated x and y, is often used by statisticians in statistical procedures known as regression and correlation analysis.

The x variable is often called the independent variable, while the y variable is the dependent variable. In other words, the x variable governs what value y may have.

If only one x variable is involved in the equation, the regression is termed "simple"; if more than one x value is used to estimate y, then the relationship is known as multiple regression work. Regression is prediction; how the variable(s) x relate to y determine the predicted value of y. The relationship of the x value to the y value is referred to as "correlation." If an x:y relationship is positively correlated, then as x increases or decreases, the value of y also increases or decreases. If the relationship between the x and y variables is a straight line or linear, then the correlation coefficient r that is calculated when the regression equation is solved is also valid. If the relationship between x and y is not linear, then the calculated correlation coefficient r has no valid meaning and cannot be used. Nonparametric correlation coefficients, such as Kendall's tau (τ) or Spearman's rho (ρ), may be used to show the strength and shape of the relationship between x and y.

Context

Statistics began with the study of probability as it applied to gambling and life insurance. Life insurance, first issued in 1583, utilized probability to determine the odds of the insured's death. Modern insurance rates today are based on the same approach to probability.

Statistics have been used for more than 250 years, but only since the mid-1900s has its use become widespread in every area of study and investigation. Explosive development in statistical theory and methods since the 1950s has made this area of mathematical statistics particularly exciting and fruitful.

The mathematical theory of probability began in 1654 when Chevalier de Mere, a wealthy French nobleman who liked gambling, asked Blaise Pascal to solve a problem involving the distribution of money in an unfinished game of chance. The Dutch scientist Christiaan Huygens published a book on games of chance in 1667, but the first book devoted entirely to probability was Jakob Bernoulli's Ars Conjectandi (1713; Art of Conjecture). In 1662, John Graunt applied statistical probability in his study of death in London.

Thomas Bayes's 1763 essay is well known as the first attempt to use the theory of probability as an instrument of inductive reasoning. Pierre-Simon Laplace in his Theorie Analytique Des Probabilites (1812; Analytic Theory of Probability) stated that the most important questions of life were really only problems of probability. Laplace was unrivaled for his mastery of analytic technique and was perhaps the eighteenth century's best applied statistician. During the middle nineteenth century, only the Soviet mathematicians Pafnuty Lvovich Chebychev and Andrey Andreyevich Markov made significant advances in probability theory.

After 1900, interest in probability and statistics soared, and it became one of the most important and fruitful areas of mathematical investigation. In 1900, Karl Pearson determined the theoretical distribution of chi-square. In 1908, W. S. Gogget calculated the distribution of t, the probable error of a mean, and this began the study of exact sampling distributions in statistics. Sir Ronald A. Fisher developed much of the theory and applications of experimental design and statistical analysis in the early 1900's.

The true beginning of nonparametric statistics was in 1936 when H. Hotelling and M. R. Pabst published their paper on rank correlation. I. R. Savage's 1962 monumental Bibliography of Nonparametric Statistics had more than three thousand references dealing with nonparametric statistics in it, indicating the growth and importance of this area of applied statistics. It is now one of the most successful branches of modern statistics.

Analysis of variance and regression theory and applications have also developed since the mid-1940s. In the twenty-first century statistical analysis has been frequently applied to the study of big data.

Principal terms

ACCURATE: referring to how close to the real or actual value an estimator is; an unbiased estimator gives an accurate estimate

DATA: the individual real number or count of information that one is measuring for use in statistical calculations

HYPOTHESIS: what it is that one wishes to test; always written as an either-or choice; the null hypothesis (H naught or Ho) is the first one stated, the alternative hypothesis (Ha) is the second one listed

POPULATION OR UNIVERSE: the entire group or area for which descriptive and inferential statistical statements are made based on samples (n) from within these areas

PRECISION: how close together in value repeated estimates from the same population or universe actually are

SAMPLE: the total number of individual observations or pieces of data (=n) drawn from the population or universe on which the estimates of parameters for that population or universe are based

Bibliography

Cochran, William G. Sampling Techniques. 3rd ed. New York: Wiley, 2005. Print.

Draper, Norman, and Harry Smith. Applied Regression Analysis. 3rd ed. New York: Wiley, 2014. Print.

Siegel, Sidney. Nonparametric Statistics For the Behavioral Sciences. 2nd ed. New York: McGraw-Hill, 2002. Print.

Snedecor, George W., and William G. Cochran. Statistical Methods. 8th ed. Ames: Iowa State UP, 1996. Print.

Sokal, Robert R., and F. James Rohlf. Biometry: The Principles and Practice of Statistics in Biological Research. 4th ed. New York: Freeman, 2012. Print.

Steel, Robert G. D., and James H. Torrie. Principles and Procedures of Statistics, with Special Reference to the Biological Sciences. 3rd ed. New York: McGraw-Hill, 1997. Print.

Weatherburn, C. E. A First Course in Mathematical Statistics. Cambridge: Cambridge UP, 1968. Print.

Zar, J. H. Biostatistical Analysis. 5th ed. Harlow: Pearson, 2014. Print.