Educational Research Overview

This article presents an overview of educational research methods and processes. Educational research encompasses the scientific method as related by John Dewey. The educational researcher begins with identifying the research problem or question, forming a hypothesis, gathering data, forming conclusions, and making a reasonable decision to reject or to validate the hypothesis based on the conclusions. Quantitative research methods yield quantifiable results as supporting evidence for the hypothesis. Qualitative research methods provide observational or narrative evidence in support of the hypothesis and previous quantitative studies. Some examples of qualitative studies include case studies, descriptive, ethnographic, and historical research.

Keywords Dewey, John; Educational Research; Hypothesis; Qualitative Research; Quantitative Research; Research Design; Research Problem; Review of Literature; Scientific Method

Research in Education > Educational Research Overview

Overview

The goal of educational research is to acquire and arrange educational data that will serve to explain, predict, and control behaviors, events, and educational programs. The scientific method, as developed by John Dewey in the early twentieth century, continues as the primary model for educational research. The scientific method involves identifying the main research problem or question, forming a hypothesis, gathering data, forming conclusions, and making a reasonable decision to reject or to validate the hypothesis based on the conclusions.

John Dewey, an American philosopher and educator, described his concept of the scientific method as it applied to educational research. Dewey believed that the scientific method could be used by educators to provide an objective means for organizing and interpreting educational programs and progress. Dewey identified five steps in the research process that remain the canons of educational research (Charles & Mertler, 2002).

1. Identify the main research problem or question.

• 2. State the potential answer or solution to the problem or question as the hypothesis statement.
• 3. Collect, analyze, and interpret data.
• 4. Form conclusions.
• 5. Verify and reject hypothesis based on conclusions. (include validity, reliability, measures, correlations, etc.)

The main question or research problem may be a single case or problem (case study) or it may involve evaluating the effectiveness of a program (program evaluation). For example, we might ask the question, Do Johnny's post-test scores improve after he has participated in the school breakfast program? This is an individual case study. We are only concerned with Johnny's scores. If we ask the question, Do the post-test scores for the fourth graders at Riverside Elementary show statistically significant improvement after participating in the school breakfast program for three months? this is a program evaluation. One individual's scores tell us very little about the effectiveness of the program overall. We want to know how this program has benefited the entire fourth grade.

The hypothesis statement is a formal statement of the researcher's prediction of the relationship that exists among the variables in a quantitative study. The hypothesis is stated as the opposite of the intended outcome. This is often confusing to the beginning researcher. The reason that the hypothesis is stated in null or opposite terms is because the researcher should not have any biases or preconceived notions about the outcome of the study. We should refer back to the first example about the school breakfast program: Johnny's post test scores do not show any improvement after participating in the school lunch program for three months. The researcher may have seen previous data that would indicate that eating a good breakfast helps memory retention. However, Johnny's case could be different.

Collecting data may involve any number of methods. The review of literature involves reading scholarly journal articles and books on a topic, synthesizing the results, and reporting on current research and trends in the proposed area of research. A review of literature also involves knowing what key terms to use in that search (Marken & Morrison, 2013). The researcher develops a research problem or question based on the information that they have gathered and considers a method for answering this question. The researcher might develop a pretest to administer to a sample group, might develop a survey questionnaire, or might outline a plan for a descriptive or historical analysis. The reseacher also could design a longitudinal study or correlational study, a case study, or a multiple methods study that includes the use of several methods. The most effective predictors of a successful research study involve multivariate or multiple methods research.

Data analysis for statistical significance, validity, and reliability is an important step for gauging the success of the program and the success of the research method. Data analysis involves providing supporting evidence to prove the hypothesis in quantitative research. The quantitative researcher uses the canons of validity and reliability to make certain that the instrument and the results are accurate. Validity refers to the instrument and the ability of the instrument to measure what it is supposed to measure. Reliability refers to the consistency of the measure. For example, a personality test might be developed to indicate what personality traits are best suited for high school cheerleaders. After a number of high school cheerleading sponsors have used this test and reported amazingly successful results, the company may begin to market their cheerleading inventory to high schools across the country. However, if a motivational expert decides to use the cheerleading inventory with a group of Fortune 500 businesses for selecting CEO's, the results may not be reliable.

Qualitative researchers provide data such as behavioral descriptions, observations, impressions, recordings, photographs, and other documents. Even though qualitative research does not involve statistical analysis, qualitative researchers will often ground their theories by using multiple methods. For example, a teacher involved in an action research project may incorporate a case study, attendance records, disciplinary records, and other data to support theories or conclusions.

Data interpretation can be a complex process. Often surveys and test scores generate another set of questions or problems. Sometimes these ambiguities are caused by a poor instrument and sometimes they simply indicate the need for further research. For example, if 90% of the survey respondents indicate that the classroom is not comfortable, what does that mean? Does it mean that the temperature is not at an acceptable level? Does it mean that the chairs are not comfortable? Even if there is a section for comments, respondents may not have time to write comments. The best strategy is to initially conduct the survey or test with a pilot group to pinpoint any weaknesses in the instrument.

Interpreting the final data should involve a narrative that indicates the implications from the research study as they relate to previous research, specifically to the review of literature as well as implications for the individual or program. The interpretation should indicate levels of statistical significance, and the constructs of reliability and validity.

Conclusions should point to clear solutions that may be substantiated by the data. The concluding comments will indicate whether or not the researcher is able to accept or to reject the initial hypothesis. Conclusions are reasonable assumptions based on the data presented in the research.

Applications

Quantitative Research

Quantitative research utilizes the scientific method as Dewey originally described for use in educational research. The interpretation should indicate levels of statistical significance, and the constructs of reliability and validity. Quantitative research may involve some of the following:

• Statistical Analysis

• Sampling

• Scales Of Measurement

• Measures Of Central Tendency

• Measures Of Variability

• Standard Scores

• Correlational Research

• Meta-Analyses

Statistically significant results occur when the evidence suggests that the results were probably not caused by chance. The process of selecting a smaller representative group from a larger group for the research project is known as sampling. The sampling group should be representative of the entire group.

The process by which observations are translated into numbers is called scales of measurement. The most widely used measurement taxonomy is the Stevens scales of measurement, in which measurement is classified as nominal, ordinal, interval, and ratio (Ary, Jacobs, & Razavieh, 1996). Nominal scales or numbers are used to represent categories. For example, males might be assigned the number 1 in a survey and females the number 2. The numbers are only significant as another name for a given category. Ordinal scales are used to rank objects or individuals. For example, if a physical education teacher wanted to rank student's performance on the 50-yard dash, they might list first the name of the student who had the best time, followed by the student who came in second, and finally the student who came in last. The individual scores would not need to be listed nor would the intervals between scores. Interval scores are more precise, indicating the exact intervals between units of measure. Interval scores are often used when recording weather statistics. Ratio scales are perhaps the most common in educational research. Ratios can be formed between any two given values. We hear educators talk about the teacher to student ratio, for example. The ratio statistic is expressed with a colon (:). For example, the teacher to student ratio might be 20:1, meaning that there are on the average 20 students per teacher in a given school. It is important to know what specific individuals are included in a ratio statistic. For example, does the teacher ratio include all faculty or just teaching faculty? Does the teacher ratio include both general and special education faculty?

Measures of central tendency summarize or show averages for a whole set of numbers. The mean (or average) score is the most commonly used measure of central tendency. Calculating the mean is done by adding all of the scores and dividing them by the total number of scores. For example if six students have taken a math test and the scores are 71, 89, 92, 94, 94, and 94, then the average score would be 89. The mode is the numerical value considered most typical of the values of a quantitative variable. In the previous example, three students scored 94 on the math test, so the modal score would be 94. There can be more than one modal score. For example, in a larger class, if three students scored a 92 and three scored a 94 and three scored a 96, then we would have trimodal scores. The median is the number in the middle of the distribution. If the median falls between two numbers, as it will with an even number of scores, the two middle scores are divided by 2. In our previous math test example, the 92 and 94 would be in the middle of the distribution, so we would add these two numbers and divide by two, stating the median score as 93. The mean is typically considered to be the least stable of the measures of central tendency because of possible extremes. For example, in our math test scores example, the student that scored a 71 on the math test would bring down the average score.

While measures of central tendency help describe average or typical measures, measures of variability provide information about the range of variation that is present or deviations between distributions in multiple studies. The range is the simplest statistic to calculate. The range in a given set of numbers is found by subtracting the lowest number from the highest. The range is generally used in combination with other statistics, as it means little without supporting data. The quartile difference is half the difference between the upper and lower quartiles of a distribution. The quartile range functions much like the median: It is the central score. The quartiles are the three points that divide a frequency distribution into four quarters with an equal number of scores in each. The second quartile is the median (Ary, Jacobs, & Razavieh, 1996).

While measures of central tendency and measures of variability are useful statistics, variance and standard deviation scores help to give a more accurate or stable picture of score differences. The variance is the score average or mean of the squared deviation scores. The variance score shows the interval measure of dispersion of scores around the mean. The standard deviation is a measure that shows the extent to which individual scores deviate from the mean of the distribution: the square root of the variance, a measure of dispersion used with interval data (Ary, Jacobs, & Razavieh, 1996). Generally, a score that is two standard deviations from the mean is considered to be statistically significant. For example, if Johnny scores two standard deviations below the mean on a standard math test, this would probably indicate that he has a learning disability in math and that he may be considered eligible for special education services.

Standard scores are scores that have been converted from one scale to another to have a particular mean and standard deviation. The validity of a score indicates the accuracy of the inferences, interpretations, or actions made on the basis of test scores (Johnson & Christensen, 2007).

Correlational research is nonexperimental research in which the primary independent variable of interest is a quantitative variable (Johnson & Christensen, 2007). Correlational research seeks to determine the extent and the direction of the relationship between two or more variables. For example, a kindergarten teacher might ask the question, Is there a relationship between early vocabulary mastery for kindergarten students that have attended day care and those students who did not attend day care? The teacher would administer a test including a list of common vocabulary to all students in the kindergarten class. The test would be the independent quantitative variable. The teacher would group the scores in group A (the students that attended day care) and in group B (the students that did not attend day care). The teacher may compare the scores using measures of central tendency, measures of variability and variance, and standard deviation scores to determine if there is a statistically significant correlation between the vocabulary scores of kindergarten students in either group.

Meta-analysis is an integration of data from a large number of studies. For example, if we were to extend the correlational example to include 1,000 kindergarten classes across the United States, we could do a meta-analysis to determine how the results might apply to a national sample.

Causal-comparative research is another form of nonexperimental research. Causal comparative research explores a primary independent variable of interest as a categorical variable (Johnson & Christensen, 2007). For example, a researcher might wish to explore the relationship between crime and the number of public libraries in a community. The researcher might conclude that there are more public libraries in larger cities and there is also more crime in larger cities. The two variables, public libraries (X) and crime (Y) would have a spurious relationship, meaning that there is no way that we could prove that the number of public libraries in a community has an adverse effect on crime. We would have to use a more rigorous approach to establish a direct causal-comparative relationship between two variables. Specifically, the researcher would need to establish that there is a statistical relationship between X and Y and that other factors are not responsible for the change in the Y statistic.

Qualitative Research

Qualitative research may enhance a previous quantitative study by providing descriptive comments that explain statistical results. Qualitative research does not necessarily include statistical measures. Qualitative research is differentiated from quantitative research in that it provides rich descriptions. These descriptions may be based on the followin:

• Observations

• Interviews

• Case Studies

• Descriptive Research

• Historical Research

• Survey Research

• Action Research

• Ethnographic Research

and other methods that provides extensive details about an individual, group, or program.

Observations involve unobtrusively viewing activities or phenomena and recording these activities for the stakeholders. Interviews consist of gathering information from the interviewee for the research project. Case studies provide a detailed account and analysis of one or more cases that are of interest to the stakeholder. Descriptive research is intended to provide an accurate description of the status or characteristics of a situation or phenomenon (Johnson & Christensen, 2007). Historical research is the systematic process of examining past events or combinations of events to arrive at an account of what happened in the past (Johnson & Christensen, 2007). Survey research is nonexperimental research based on questionnaires or interviews. Action research focuses on solving a local problem in which the researcher is usually involved. Ethnographic research focuses on describing group culture.

Most qualitative researchers agree that for a qualitative study to be considered trustworthy it must address the issues of internal validity, reliability, and external validity or generalizability. These issues include triangulation, member checks, peer review or examination, researcher's position or reflexivity, adequate engagement to data collection, maximum variation, audit trail, and rich, thick descriptions (Merriam, 2002).

Triangulation involves using different sources of data and collection methods to confirm findings. The researchers should discuss the data and tentative conclusions with the stakeholders to illicit member checks about the plausibility of their findings. The researchers should also discuss or compare their findings with peers that have conducted similar studies to check for congruency. The researcher's position, biases, and theoretical background will invariably have some bearing on their interpretation of the findings, and it is important for the researcher to be aware of these influences. The researcher must allow adequate time for engagement with the data collection phase to make sure that they have collected a sufficient sampling of representative cases of the whole.

Maximum variation or diversity in sample selection to allow for a greater range or generalizability is critical. The researcher should keep a detailed record or audit trail of the methods, procedures, and conclusions used in the development of the study. The researcher should provide a sufficient description to conceptualize the study but should not include superfluous details that detract from the study.

Viewpoints

Traditionally, educational researchers have argued that only results that can be quantified are reliable or valid. However, qualitative research has gained increasing respect among researchers. While quantitative research results may be more statistically sound, they often leave more questions than answers. The purpose of qualitative research is to substantiate or enhance the findings of statistical studies with observational data. Many researchers highly recommend a mixed methodology using both quantitative and qualitative methods. Another researcher has recommended a mixed methodology in which “mixed researchers should strive for what is the radical middle, which should not be a passive and comfortable middle space wherein the status quo among quantitative and qualitative epistemologies is maintained, but rather a new theoretical and methodological space in which a socially just and productive coexistence among all research traditions is actively promoted, and in which mixed research is consciously local, dynamic, interactive, situated, contingent, fluid, strategic, and generative (Onwuegbuzie, 2012).

The type of methodology that a researcher chooses to use is not as important as the fit with that researcher's abilities and training and the accessibility that they will have to the particular individual or group that is to be studied. Merriam (1998), Stake (1995), and Creswell (1998) all recommend that the researcher should select a method that suits their personality type and abilities. Both Merriam (1998) and Lincoln and Guba (1985) observe that the ideal list of attributes or characteristics of a good researcher are inquisitiveness, sensitivity, and good communication skills. Others argue for “research that works.” That is, research “must pay serious attention to ethical issues,” must “employ different strategies and techniques to enhance reliability and validity of research findings,” and should “be accessible to its intended user group(s)” (Du, 2012).

Terms & Concepts

Conclusions: Reasonable assumptions based on the data presented in the research.

Educational Research: Research designed to acquire dependable and useful information about the educative process. Its goal is to discover general principles or interpretations of behavior that can be used to explain, predict, and control events in educational situations or to formulate scientific theory (Ary, Jacobs, & Razavieh, 1996).

Hypothesis Statement: A formal statement of the researcher's prediction of the relationship that exists among the variables in a quantitative study.

Qualitative Research: Research that relies primarily on observational data.

Quantitative Research: Research that utilizes the scientific method as Dewey originally described for educational research. The interpretation should indicate levels of statistical significance, and the constructs of reliability and validity.

Research Design: Design that encompasses the overall research strategy, including the hypothesis and methods.

Research Problem: An issue or problem that the researcher seeks to solve using a research design.

Review of Literature: An analysis and synthesis of current literature on the proposed research topic.

Scientific Method: Involves identifying a research problem, forming a hypothesis, collecting and analyzing data, forming conclusions, and verifying or rejecting the hypothesis based on the conclusions.

Bibliography

Ary, D., Jacobs, L., & Razavieh, A. (1996). Introduction to research in education, 5th ed. Ft. Worth, TX: Harcourt Brace.

Charles, C. & Mertler, C. (2002). Introduction to educational research, 4th ed. Boston: Allyn & Bacon.

Creswell, J. (1998). Qualitative inquiry and research design choosing among five traditions. Thousand Oaks, CA: Sage.

Du, N. (2012). Educational research: Purpose, quality and effectiveness. Annual Review of Education, Communication & Language Sciences, 91-20. Retrieved December 11, 2013, from EBSCO Online Database Education Research Complete. http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=88924967&site=ehost-live

Johnson, B. & Christensen, L. (2007). Educational research: Quantitative, qualitative, and mixed approaches, 3rd ed. Thousand Oaks, CA: Sage. Retrieved from http://www.southalabama.edu/coe/bset/johnson/2glossary.htm#a

Lincoln, Y. & Guba, E. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

Marken, J., & Morrison, G. (2013). Objectives over time: A look at four decades of objectives in the educational research literature. Contemporary Educational Technology, 4, 1-14. Retrieved December 11, 2013, from EBSCO Online Database Education Research Complete. http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=90645633&site=ehost-live

Merriam, S. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.

Merriam, S. & Associates. (2002). Qualitative research in practice: Examples for discussion and analysis. San Francisco, CA: Jossey-Bass.

Onwuegbuzie, A. (2012). Introduction: Putting the mixed back into quantitative and qualitative research in educational research and beyond: Moving toward the radical middle. International Journal Of Multiple Research Approaches, 6, 192-219. Retrieved December 11, 2013, from EBSCO Online Database Education Research Complete. http://search.ebscohost.com/login.aspx?direct=true&db=ehh&AN=91277401&site=ehost-live

Stake, R. (1995). The art of case study research. Thousand Oaks, CA: Sage.

Suggested Reading

Bogdan, R. & Biklen, S. (2006). Qualitative research for education: An introduction to theories and methods, 5th ed. London: Brown.

Cain, T., Holmes, M., Larrett, A. & Mattock, J. (2007, Feb.) Literature-informed, one-turn action research: three cases and a commentary. British Educational Research Journal, 33 , 91-106.

David, S. (2007, April). Resolving the quantitative-qualitative dilemma: A critical realist approach. International Journal of Research & Method in Education, 30 , 3-17.

Freed, M., Hess, R., & Ryan, J. (2002). The educator's desk reference: A sourcebook of educational information and research, 2nd ed. (ACE/Praeger series on higher education). Westport, CT: Praeger.