The bar we set
We approved a plan on 2026-07-10 with an acceptance test we weren't sure was reachable. Six drafted analyst-memo queries against Nigerian economic data, scored 0-10 across five dimensions — named-entity density, citation quality, sector-specific detail, honest-gap acknowledgement, decision-usefulness. The strict pass criterion: every query's mean score across three temperature=0.2 runs must be ≥8/10, with no query below 6 in any single run.
At approval time the aggregate was somewhere around 30/60 across the six queries — a system that produced grounded but generic answers, and refused competently but not always. The gap to the bar was real. We gave it 4-5 weeks.
What we shipped
Phase 1 — retrieval breadth. Kind-diversity enforcement across the top-K result set so a "start a fintech" query stopped collapsing into 12 CBN circulars and started pulling BOI, NEXIM, PayStack, Flutterwave, and the World Bank agribusiness chapters in the same context window. Named-entity boost when the query mentions "factory", "startup", "invest", "loan". Deduplication so a briefing about the same fact doesn't crowd out its own primary source. Phase 2 — a six-class rule-based intent classifier and memo templates. Sub-millisecond routing on regex patterns: venture\_feasibility\, strategic\_forecasting\, credit\_risk\, regulatory\_analysis\, market\_sizing\, general\_qa\. Each intent gets a memo template — a section-headed scaffold with a named-entity mandate, an honest-gaps section, and a 1000-1500 word target. The general\_qa\ template stays empty (no memo shape) so genuinely-general questions don't get forced into a memo they don't need. Phase 3 — composition quality. Two changes did most of the work here:
