BoxAgnts' tool system, from the bottom-level WASM sandbox to the top-level Tool trait, has solved "how tools run safely." But tools ultimately need to be called by AI models — which introduces two engineering problems: the complete incompatibility of API formats across AI vendors, and the interleaved orchestration of conversation flow and tool execution. These two problems are solved by the Provider abstraction layer and the Agent query loop, respectively.
Provider Abstraction: Being an LLM Vendor Agnostic
Different types of AI model APIs differ significantly in request format, response format, and error handling.
Let's start with the request side. Anthropic splits roles into user and assistant, with the system prompt as an independent top-level system field; OpenAI treats the system prompt as a role: "system" message; Google Gemini places system_instruction at the top level of the request body but with yet another format. If the upper-layer Agent loop had to handle these differences directly, the code would become a giant match provider_id { ... } branch.
BoxAgnts' solution introduces three layers of abstraction:
