For millions of TikTok users, protection from harmful content spirals depended on which side of an experiment they landed on.A confidential internal document obtained by Bloomberg Aug. 4 found that TikTok excluded 10 percent of its U.S. users from an algorithmic safety update in early 2022. At the time, that would have amounted to roughly 15 million people.Among them was Chase Nasca, a 16-year-old from Long Island who died by suicide in Feb. 2022.
You May Also Like
TikTok’s subsequent review found that Nasca’s account had become trapped in a filter bubble filled with videos about suicide, self-harm, depression, loneliness, and hopelessness. Because he was part of the control group, the report said, TikTok’s strategies for disrupting those repetitive recommendations "did not take effect on this user by design."Why did TikTok leave millions without the safeguard?TikTok had already acknowledged that its algorithm could show users too much of the same troubling material back in 2021, when the company announced that it was testing ways to break up repetitive recommendations involving subjects such as extreme dieting, sadness, and breakups. One video might not violate its rules, TikTok explained, but seeing similar videos repeatedly could create a harmful viewing experience.The company’s solution came with an experiment. By early 2022, TikTok had reportedly introduced the updated recommendation system for 90 percent of its U.S. users while keeping the remaining 10 percent on the older version.That split allowed TikTok to conduct a backtest, a form of A/B testing that compares what happens when one group receives a feature and another does not. Technology companies regularly use these experiments to test new products and design changes. In this case, however, the feature was intended to protect users from potentially dangerous content.According to Bloomberg, TikTok wanted to see whether breaking up filter bubbles would affect daily active users and other important engagement measurements. The company’s internal report described the choice as a "delicate balance" between user safety and its ability to measure the effect on those metrics.TikTok told Bloomberg that the experiment was intended to confirm whether its safety changes were working. The company said it has since refined its testing practices, although it did not explain what those changes were.What did TikTok find when it examined Nasca's feed?TikTok did not review Nasca’s viewing history immediately after his death. The company began looking into his account more than a year later, after Bloomberg contacted it back in March 2023 for an investigation into how the platform affected young users.
