Reuters and Time are taking a new approach to bot blocking. Both publishers have recently started blocking all AI bots by default, and created whitelists of approved bots to access content on their sites.It’s a strategy already adopted by publishers like People Inc. at the beginning of this year and The Atlantic at the end of last year. Reuters and Time made the decision to block bots by default last month.
“We saw that there was an imbalance between the value that publishers like Reuters provide and the value… that Reuters receives in kind, and so instead we went from a default allow-all to a default disallow all,” said Josh London, head of Reuters Professional, who oversees the direct-to-consumer and direct-to-professional businesses. “Our content costs money to create. It has significant value, and the access to it, we feel, must be earned.”
Time now allows about 70 bots to access onsite content, according to its COO Mark Howard. Those bots range from crawlers run by big AI labs and social platforms, to the automated systems Time uses to keep its own website running. Time uses ScalePost to manage AI bots.
“Now we’re starting to think about: as the volume of bot traffic continues to increase significantly, and we see through a number of our vendor partners that we have very high domain authority with AI bot traffic, there’s value in that,” Howard said. That perceived value, he added, can help support the AI visibility product Time is developing to sell GEO insights to brand clients.
