Reference: Int J Obes (Lond). 2024 Jan;48(1):83-93
Practice Point: To achieve and maintain a healthy weight, water remains the beverage of choice.
EBM Pearl: Fancy statistics to impute (i.e. make up) missing data can’t recoup the losses to validity incurred with massive attrition rates.
Patients frequently lament the challenges of weight loss and sugar cravings. In response, doctors often suggest replacing sugar-sweetened beverages with nonnutritive sweetened (NNS) drinks that contain artificial sweeteners like aspartame or sucralose, or natural alternatives like stevia or sugar alcohols. The overall aim is to reduce caloric intake, promote weight loss, and encourage a gradual shift to a healthier diet. Despite the widespread use of NNS, high-quality evidence supporting this approach is limited, and observational data suggest there may actually be harms including an increased risk of all-cause and cardiovascular mortality, insulin resistance, and disruption of the gut microbiome. A recent equivalence trial published in the International Journal of Obesity took a closer look at how NNS beverages like diet soda affect weight loss when compared to water.
An equivalence trial is kind of like a two-sided noninferiority trial, where the null hypothesis is that the two groups are different. In this one, investigators randomized 749 healthy adults in England (70% female, mean age 45) with a BMI of 27-35 to consume two 330 ml (11 oz) servings per day of a NNS beverage or water. Both groups participated in a group weight management program which met weekly for 12 weeks then monthly for 40 weeks. NNS beverages were not allowed for participants in the water group, but water was allowed for those assigned to NNS beverages. The NNS beverage group had a choice of 20 different brands provided by investigators and supplied by the American Beverage Association (problem #1). Sugar-sweetened beverages were allowed in both groups. Compliance was measured via daily online beverage logs, periodic food frequency questionnaires, and return of empty NNS beverage packaging. Participants were excluded from trial if they missed > 3 group sessions. The primary outcome was change in body weight from baseline to week 52.
At 52 weeks, 65% (487) of participants randomized had dropped out of the trial (problem #2). In an attempt to try to reclaim some of the power diminished by the very large attrition rate (almost three times the number of expected dropouts!), missing data were imputed based on two different statistical mechanisms (problem #3) and analyzed by intention-to-treat. Thankfully, a per protocol analysis was also done, narrowly avoiding the problem of only analyzing by intention-to-treat in an equivalence or noninferiority trial that we wrote about last week. According to the per protocol analysis, at 52 weeks the NNS group had statistically more weight loss than the water group (mean weight loss of 7.5 kg vs 6.1 kg, between-group difference 1.4 kg [90% CI {problem #4} 0.2-2.6 kg, p< 0.05]). The statistical significance did not persist when analyzed by intention-to-treat with either method of imputation. There were no significant differences in secondary anthropomorphic outcomes.
If you believe these results, it looks like NNS beverages may not be the devil after all. But do we believe these results? Let’s talk about the problems first. Problem #1: Conflict of interest. On the heels of bad press about NNS, this study was funded by the American Beverage Association, seemingly trying to show that NNS are not that bad. Problem #2: Massive attrition rates. Investigators make an educated guess about what the expected loss to follow will be based on existing data. They use this information to calculate power and determine how many people need to be enrolled to achieve that power. When the power is low, that means there is a greater chance of type II error - when one fails to reject a null hypothesis that is actually false. Problem #3: A fishing expedition. If you don’t have the number of data points you need to achieve adequate statistical power, the “right” thing to do is to repeat the trial with more participants. Casting a wide net and getting fancy with data imputation doesn’t make up the difference. Problem #4: Deviating from the norm without explanation. The investigators use a 90% CI for the primary outcome and a 95% CI for secondary outcomes. There is no explanation for this, leading to even less certainty about the less certain primary outcomes. Problems 1-4 usurp other issues such as generalizability and healthy-patient (compliance) bias also seen in this trial, so we’ll table those for now.
So what do we take away from this trial? That a supervised weight loss program might help healthy, compliant patients lose about 6-7 kg (14 lbs) over a year, which probably has little to do with whether they drink water or NNS beverages. Everything else is marketing.
For more information, see the topic Diets for Weight Loss in DynaMed.
DynaMed EBM Focus Editorial Team
This EBM Focus was written by Nicole Jensen, MD, Family Physician at WholeHealth Medical. Edited by Alan Ehrlich, MD, FAAFP, Executive Editor at DynaMed and Associate Professor in Family Medicine at the University of Massachusetts Medical School; Katharine DeGeorge, MD, MS, Senior Deputy Editor at DynaMed and Associate Professor of Family Medicine at the University of Virginia; Dan Randall, MD, MPH, FACP, Deputy Editor at DynaMed; Vincent Lemaitre, PhD, Medical Editor at DynaMed; Hannah Ekeh, MA, Senior Associate Editor at DynaMed; and Jennifer Wallace, BA, Senior Associate Editor at DynaMed.