One of the most common failures with a free model endpoint in GitLab CI is not a wrong answer. It is a silent slowdown, an unexpected 429, or a timeout that leaves no trace in the job log. The job that summarizes failed tests has been running fine for three days, then it suddenly takes forty seconds and dies with a generic upstream error. The provider dashboard shows no useful history, and the only signal is a longer pipeline. You are flying blind.
The free model access and free server option referenced here are operator-supplied facts from MonkeyCode, and I have not verified their quotas, uptime, or stability. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
This post describes a small observability layer you can place in front of a free model endpoint. It records only request metadata — timestamps, status codes, latency, payload sizes, and error types — without storing prompts, responses, tokens, or any other potentially sensitive model input. The artifact is a GitLab CI job that runs a local reverse proxy, sends a controlled burst of smoke requests, then writes a decision table you can commit as an artifact.
Why storing prompts is a trap
When a model call fails, developers usually want to save the full request and response for debugging. That is tempting, but it creates three problems in CI:
