Ability testing and bias

SIGNIFICANCE: Ability tests are sometimes described as objective and fair tools for identifying individuals’ strengths; however, many people have criticized them as being biased against minority groups. Because of the increasing use of ability tests in the decisions made about people’s daily lives, it is critical to understand how they might be unfair.

Ability tests play a significant role in American society. Intelligence and achievement tests are used for many purposes in education: to evaluate a student’s progress, to identify areas of weakness, and to help define teaching goals. They are a major component in the decision to place students in special classes and college entrance examinations. Some occupations also use ability tests to determine qualifications for employment, entrance into training programs, and promotions. With such widespread use, it is imperative that these tests be fair and accurately measure the construct they are purported to measure.

Test bias is a systematic error that disadvantages the performance of one group over another. Although lower scores are not always indicators of bias, various aspects of a test can affect performance and lead to incorrect beliefs about the abilities, knowledge, and characteristics of an individual or group.

Sources of Bias

Critics identify several sources of bias in test items. Most importantly, they note that tests show cultural bias in favor of the culture in which they are developed. Those developed in the United States, for example, generally have been written by middle-class males who have had Anglo-American life experiences. Their attitudes, beliefs, and values are subtly built into their tests. The information that these individuals have acquired and used within their own cultural experiences will determine what they think is important to know. Therefore, the content presented in the test may be culturally inappropriate for someone who has not been reared in the same cultural situation as the test developer.

Psychologist Robert Williams believes that items on ability tests do not measure general abilities, aptitude, or intelligence for everyone who takes them but rather demonstrate the test taker’s knowledge of the test producer’s culture. Not only must the test takers know the “correct” answer; they also must respond in a specific way in order to get credit for their knowledge. The scores on these tests, then, will be as much measures of acculturation and income as they will be measures of ability or skills.

Language has been identified as a major source of bias in ability test items. Test takers may not adequately understand the wording of items because they are not written in their native language, or because items may be phrased in an unfamiliar way. Psychologist Ernesto Bernal points out that even with translated tests or the use of interpreters, the language bias is not always eliminated, because the meaning of a word in one language, dialect, or region may not directly correspond to a word or phrase in another language, dialect, or region. A related type of bias results from the differences in meanings of words, which may be related to cultural background, gender, or income.

Sometimes, the type of question asked or task required may interfere with a test taker’s performance. Researchers have discovered that people have different cognitive, information-processing, or learning styles. That is, they have preferences for the ways they learn, remember, and solve problems most effectively. Some do well with understanding bits of information that have meaning by themselves. Others will learn and remember things if they are related to other information. Researcher Asa Hilliard refers to these as analytical and relational styles, respectively. If the ability test item is written in such a way that it restricts certain test takers from using their preferred ways of processing information, they are more likely to do poorly on the test. In fact, most tests assume that everyone will or should use the style that is seen primarily among middle-class or upper-middle-class White males. If they do not, they are penalized for using a “nonstandard” problem-solving approach.

Research also suggests that test takers tend to score better when they identify with the situations and subjects covered by the test items. Researcher Paula Selkow found that women and men sometimes respond differently to male-oriented and female-oriented content. The timed format of some tests may also be a source of bias.

An example of bias can be found in the section designed to measure social reasoning and knowledge of socially acceptable ways of behaving contained in some standardized intelligence quotient (IQ) tests. A child who has been taught different cultural values or has life experiences that do not mirror those of the test developer will be at a serious disadvantage when taking this part of the test. For example, consider the question, “What is the thing to do if you lose a ball that belongs to one of your friends?” The “best” answer, according to the scoring guidelines in the test manual, is to buy a new one and pay for it. A child from a low-income family, however, may not be able to buy a new ball, and may not see that as an option. The “next-best” answer is to look all over for the ball. Yet, if the ball is really lost, would it not be a waste of time to continue to look for it? If it were possible to find it, however, it might very well be better to look for it than to buy a new one. A child may have been taught that it would be best to apologize for losing the ball. The test taker would not receive credit for this answer.

When value-based questions such as the one above are scored, an assumption is made that the test taker lacks the knowledge to answer the question correctly. It would not be clear, however, without further questioning, why an individual actually missed the question. Scoring procedures do not generally allow for the possibility of alternative correct answers. The test assumes a common cultural perspective that may not be shared by all test takers.

Some tests require that the examinee respond to or respond with information that is decontextualized. That is, the information is taken out of the situation that makes it relevant. This is consistent with an analytical cognitive style and is the type of training that is likely to be emphasized by middle-class White individuals and the traditional education system. For example, many vocabulary tests require a strong memory for specific facts and details. Relational learners may have difficulty defining words that are presented to them outside a meaningful context, such as a picture, a sentence, or a paragraph. They may “talk around” the definition and receive no credit for their knowledge.

Concerns about test bias have existed since the beginning of the psychological testing movement. At various points in history, European immigrants to the United States, African Americans, and Latinos have been discriminated against by those using results from ability tests as evidence of their supposed inferiority. Indigenous, Asian, and Jewish Americans have also been affected by the misuse of tests. By the 1970s, some researchers became interested in finding ways to measure cognitive ability with minimal or no bias, leading to the development of new tests, administrative procedures, and means of interpreting test results. Other psychologists have suggested that even biased tests can be useful, not for establishing a single indicator of intelligence but for revealing what educational opportunities the test taker may need and best respond to. However, employers must use caution and select ability testing measures with care to ensure they do not adversely impact potential hires, which can lead to legal action.

IQ Tests

Psychologist Alfred Binet developed the first usable IQ test in 1905, as a way to diagnose low-achieving French students. He argued against the notion that intelligence is a fixed quantity. Lewis Terman, however, who provided the major adaptation of Binet’s intelligence test from French to English in 1916, stated that the questions on the test did indeed provide a measure of innate intelligence. He believed that his revised test proved that the Eastern European immigrants of the day, as well as people of African descent and low-income individuals of all races, were genetically inferior to White individuals. He did not include test items that were consistent with the language and other cultural experiences of these groups.

The personal and political views of social scientists have contributed to the biased use of tests. In the 1960s, psychologist Arthur Jensen published articles and books supporting the view that intelligence is an inherited trait tied to genetics. Most often cited as evidence were the large differences between the average IQ scores of White and other groups. Jensen opposed the notion that these differences are attributable to item bias.

Most psychologists view the large differences in average IQ scores between White individuals and almost every other group as an indication of test bias. They note that many individuals who are not proficient in English have been subjected to IQ tests in that language. Also problematic for many psychologists are the social consequences for groups that have already suffered educational and vocational disadvantages, as well as discrimination. According to these psychologists, IQ testing often leads to further negative political and educational consequences.

Further research concerning the biased nature of ability testing found that IQ scores correlate highly with socioeconomic status. This means these tests score answers based on a "privileged" baseline. Individuals of higher socioeconomic status with the same true IQ score or level of ability as someone in a lower socioeconomic class will have a score that falsely reflects a greater ability. The way questions are worded, cultural references, language used, and many other factors contribute to this problem, and researchers are continually working to limit the bias found in ability testing.

Bibliography

Benson, Etienne. "Intelligent Intelligence Testing." Monitor on Psychology, vol. 34, no. 2, 2003, p. 48, www.apa.org/monitor/feb03/intelligent. Accessed 10 Dec. 2024.

Friedrich, Kat. "Why IQ Tests Are Bad Science." Popular Mechanics, 11 May 2023, www.popularmechanics.com/science/math/a43862561/why-iq-testing-is-biased. Accessed 10 Dec. 2024.

Hilliard, Asa G. III. "IQ Testing as the Emperor's New Clothes: A Critique of Jensen's Bias in Mental Testing." Perspectives on Bias in Mental Testing, edited by Cecil R. Reynolds and Robert T. Brown, Plenum, 1984.

Holden, LaTasha R., and Gabriel J. Tanenbaum. “Modern Assessments of Intelligence Must Be Fair and Equitable.” Journal of Intelligence, vol. 11, no. 126, 2023, p. 126, doi.org/10.3390/jintelligence11060126. Accessed 10 Dec. 2024.

Jensen, Arthur. Bias in Mental Testing. Free, 1980.

Jensen, Arthur. "Test Bias: Concepts and Criticisms." Perspectives on Bias in Mental Testing, edited by Cecil R. Reynolds and Robert T. Brown, Plenum, 1984.

Kamin, Leon J. The Science and Politics of IQ. Halsten, 1974.

Martinkova P., et al. “Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments.” CBE Life Sciences Education, vol. 16, no. 2, 2017, doi.org/10.1187/cbe.16-10-0307. Accessed 10 Dec. 2024.