Most "AI customer service" demos fall apart the same way. You ask a question the

model can't answer from data, and instead of stopping, it produces something

plausible. In a chat toy that's a curiosity. In after-sales support it's a

promise your company now has to honour — a refund that was never approved, a

delivery date that doesn't exist, a return window that isn't your policy.