Toolformer: Language Models Can Teach Themselves to Use Tools (2023) answers a question that sounds circular: how do you build a training set for tool use without anyone annotating when a tool should have been used? The answer is a filtering criterion, and it is the sort of idea worth stealing regardless of whether you care about the paper.
The question it asks
Language models are bad at things a calculator is good at, and cannot know anything after their training cutoff. Giving them access to external tools is obviously desirable. The hard part is not calling the API — it is deciding when a call is warranted, which tool, with what arguments, and how to use the result.
The obvious approach is supervised: have humans annotate a corpus with the calls that should appear. That is expensive, and it bakes in annotators’ opinions about when a tool helps, which may not match where the model is actually weak. The paper’s alternative is to let the model’s own uncertainty decide.
The method
