“We basically work from the assumption that if we want data, then we need to obtain it ourselves,” Sapiezynski said of their account cloning approach. “Even if we did, for example, want to use the official TikTok researcher API, none of the user agency is covered there. You can see what content is available, but you cannot see individual timelines that will tell you how the algorithm reacts to a particular user watching or not watching a particular video. Similarly, with the European Union’s researcher data access, all of this data can only be accessed aggregated and not from a perspective of a single user. So when you want to really study personalization, this research cannot be done on the aggregated data.”
Mind the gap
The team ran their experiments multiple times on the 90 cloned accounts and made side-by-side comparisons, using both implicit and explicit signals, to see how TikTok’s algorithm responded in terms of recommended content on the FYPs. They focused on three popular topics: cooking videos, fitness videos, and sports betting.
The “not interested” button proved most effective, reducing unwanted content by around 84 percent, compared to just a 48 percent reduction from merely skipping videos. “So if you don’t want to see something, you should be hitting that button,” said Kaplan. But the authors note that the “not interested” option seems to be deliberately hidden from users. And it was very easy for the algorithm to “relapse” into once again flooding an FYP with previously unwanted content; even a brief re-engagement by a user is sufficient.
