Every scraping post I write lately is about working around something: bot walls, consent screens,...
Every scraping post I write lately is about working around something: bot walls, consent screens, keys that require a developer account. This one is different. arXiv runs one of the last genuinely open APIs on the research web, and it is the right way to keep up with the ~800 AI papers that land there every week.
The whole API is one endpoint
GET https://export.arxiv.org/api/query
?search_query=(all:"multi-agent") AND (cat:cs.AI)
&sortBy=submittedDate&sortOrder=descending
The arXiv (pronounced "archive") team recently announced a significant update to its official code of conduct. The popular…
Staying on Top of ML Research With ~10,000 new papers on Arxiv every month, staying...
ArXiv is doing more to crack down on the careless use of large language models in scientific papers.
arXiv's API doesn't send CORS headers, so a genuinely browser-only tool can't talk to it directly -...
The snowballing ability of artificial intelligence to trawl open data sets has some scientists worried about losing control of…
“You shouldn’t have put your content on the internet if you didn’t want it to be on the internet,” Common Crawl’s executive…
