In 2021, TikTok changed its algorithm to stop users being flooded with harmful content. But it did not give the safer version to everyone. It held 10% of US users on the old version, about 15 million people at the time. It wanted to test whether the safer feed was less addictive. That is according to a confidential document reported by Bloomberg.
The document says that control group included Chase Nasca, a 16-year-old from New York. His account was in the test. As a result, it says, “TikTok’s filter bubble prevention strategies did not take effect on this user by design”. His feed then served him a stream of suicide, self-harm and depressive content, the document says. He died by suicide in February 2022.
What the document says
The experiment was an A/B test, or “backtest”. It is a standard technique that runs two groups to measure the effect of a change. Here, Bloomberg reports, the change under test was the safety system itself. TikTok wanted to know whether the safer feed reduced “stickiness”, a metric Wall Street watches. The test ran for up to a year.
TikTok wrote the 14-page document in March 2023, and it is under a court seal, Bloomberg reports. It compiled the review only after Businessweek asked about teen safety. It reviewed the 7,563 videos Chase saw in his final two weeks. It found 73% carried themes such as sadness or mental-health struggles that did not break its rules. Almost 10% did break them.
