Double-blind, placebo-controlled clinical trials
Double-blind, placebo-controlled clinical trials are considered the gold standard in medical research for evaluating the efficacy of treatments. In these studies, participants are randomly assigned to receive either the actual treatment or a placebo, with neither the participants nor the researchers aware of who receives which. This "blinding" minimizes biases and confounding factors that might influence results, thereby enhancing the study's validity. Historically, the concept emerged in the 1950s and gained recognition in the 1960s, eventually becoming a requirement for new drug approvals by the FDA in the 1970s.
While pharmaceutical drugs are often the focus of these trials, herbs and supplements are also subjected to rigorous testing, especially as interest in natural treatments has surged. Some high-profile studies have indicated that many natural products, such as vitamin E and glucosamine, may not be effective for their claimed benefits. However, a 2007 analysis raised concerns about the integrity of double-blind trials, revealing that many studies do not adequately verify whether the blinding remained intact, which is crucial for interpreting results. This means that both negative and positive findings may be compromised, leading to skepticism about the reliability of many double-blind, placebo-controlled studies.
Double-blind, placebo-controlled clinical trials
Definition: Scientific trials in which a fake treatment (placebo) is used in conjunction with a real treatment and in which participants and researchers, through “blinding,” do not know which participants are receiving real or placebo treatments.
Overview
The gold standard for determining whether a medical treatment works is the double-blind, placebo-controlled study. With a few exceptions, new drugs must pass a number of double-blind studies to be approved by the US Food and Drug Administration (FDA). Conversely, when a drug already in use for a given purpose repeatedly fails to prove effective for that purpose in double-blind studies, that drug is eventually discredited.
Herbs and supplements do not need FDA approval, but they, too, are subjected to double-blind studies, and the results of those studies, whether positive or negative, can have a major impact on the public. Therefore, when an article published in the International Journal of Epidemiology in 2007 exposed a serious deficiency in the entire body of double-blind study literature, it struck at the very heart of evidence-based medicine. Based on the findings of this article, the results of virtually all modern double-blind trials are viewed with skepticism.
History of Double-Blind, Placebo-Controlled Studies
In a double-blind, placebo-controlled study, some participants are given the real treatment while others are given a fake treatment designed to appear as much as possible like the real treatment (the placebo control). The assignment to real or fake treatment groups is accomplished by flipping a coin or by a random number generator. Both participants and researchers are kept from knowing who is receiving real treatment and who is receiving fake treatment and are, therefore, “blinded.” The full technical name for these studies, randomized double-blind, placebo-controlled trials, is often abbreviated as RCTs, or randomized-controlled trials. However, this abbreviation leaves out both placebo and blinding and therefore, is not discussed here.
The double-blind, placebo-controlled study was first conceived by German researchers in the 1950s as a way to minimize the power of suggestion and other confounding factors. By the 1960s, the medical scientific community had come to recognize that double-blind trials are the essential means of establishing treatment efficacy. However, it was not until the 1970s that pharmaceuticals were routinely required to pass meaningful double-blind studies to obtain FDA approval. (Many drugs approved before this time were “grandfathered” in.)
At about the same time, double-blind studies of herbs and supplements began to appear sporadically in the literature, but such studies remained relatively uncommon until the late 1980s. The rate of publication of double-blind studies of natural products has since grown at an astonishing pace. In the early 1990s, months could go by before a new double-blind study of a natural product was published; in the first decade of the twenty-first century, fifteen to twenty such studies were published each week. Furthermore, while the typical study published in the early days of natural product testing involved ten or twenty participants, modern studies commonly enroll more than one hundred participants. Some studies are even larger: Studies of vitamin E, beta-carotene, and other antioxidants, for example, have enrolled tens of thousands of participants.
As natural treatments have begun to undergo systematic testing, many have failed to prove effective. For example, in the giant studies just mentioned, vitamin E and beta-carotene proved ineffective for preventing heart disease or cancer. Others also have proved ineffective: garlic for high cholesterol, glucosamine for arthritis, and calcium supplements for osteoporosis.
These negative trials have been discouraging for supporters of natural medicine. However, the foregoing journal article has cast doubt on these negative results, though it also casts doubt on all positive results. This landmark 2007 article by Danish researcher Asbjorn Hróbjartsson and colleagues in the International Journal of Epidemiology was, essentially, a study of studies. Through extensive analysis of published studies augmented by interviews with some of the researchers who published those studies, Hróbjartsson and colleagues documented that the vast majority of researchers who perform double-blind, placebo-controlled studies fail to carry out a central, essential task: that of determining whether the blinding held firm. In other words, researchers did not check whether participants and observers remained unable to distinguish the real treatment from the fake treatment.
It would not have been difficult to answer this question because one can simply poll participants and observers and ask them to guess. If the guesses come out correct no more than about one-half the time, then it would be fair to conclude that the blinding remained intact. However, researchers generally do not conduct such a poll, so they generally do not know whether the blinding remained intact. Knowing this, however, is essential to a study’s validity. If blinding does not hold–that is, if most people involved in the study figure out who is taking placebo and who is taking the real treatment because, for example, the real treatment is smelly–then the validity of the study is drastically and even fatally compromised.
Importance of Blinding
Suppose the researchers conducting a study want to prove that an herb or supplement does not work. Such bias does not matter much if the study is truly double-blind; because researchers cannot tell who is getting the real treatment and who is getting the placebo, the outcome of the study is insulated from their preferences. However, once the blinding is broken, this protection disappears. For example, researchers disinclined to believe that glucosamine can help arthritis may unconsciously underestimate or under-report benefits in patients they know are taking glucosamine. This, in turn, could lead to a false outcome in the study, an apparent failure of a treatment that actually works.
The reverse also is true. For example, if researchers are biased in favor of the natural product (or drug) being tested, their predilections will likely cause them to see benefits not actually present—this is only human nature. However, when researchers do not know who is getting the treatment and who is getting the placebo, their bias has no effect. (Bias also can appear through manipulation of statistics or dishonesty.)
The problem of observer bias is just one of the confounding factors that double-blinding forestalls. Still, Hróbjartsson and colleagues found that most researchers do not bother to poll their study participants to see if the blinding held. Among those who did conduct such a poll, less than one-half actually found that the blinding had been maintained. This means that more than one-half of all published double-blind studies are quite possibly invalid.
In one sense, this is good news for supporters of natural medicine. Negative studies regarding natural products may have been flawed by a broken blinding. In another sense, the very same research deficiency identified by Hróbjartsson and colleagues means that positive studies involving natural products also may be invalid.
Bibliography
Hróbjartsson, A., et al. “Blinded Trials Taken to the Test: An Analysis of Randomized Clinical Trials That Report Tests for the Success of Blinding.” International Journal of Epidemiology, vol. 36, no. 3, 2007, pp. 654-663. doi:10.1093/ije/dym020.
Kantor, M. “The Role of Rigorous Scientific Evaluation in the Use and Practice of Complementary and Alternative Medicine.” Journal of the American College of Radiology, vol. 6, no. 4, 2009, pp. 254-262.
Monaghan, Thomas F., et al. “Blinding in Clinical Trials: Seeing the Big Picture.” Medicina, vol. 57, no. 7, June 2021, p. 647. doi:10.3390/medicina57070647.
Neutens, James J., and Laurna Rubinson. Research Techniques for the Health Sciences. 5th ed., Pearson, 2014.