There's a lot of public concern that an AI will someday go off the rails and annihilate humanity.

So it may be a little surprising that Boston Consulting Group is training its customer-facing AI agent, Jamie, on what not to do.

Japjit Ghai, a managing director and partner at BCG X, the firm's tech build-and-design division, said on a recent episode of the comany's podcast that Jamie is learning from both its strongest-performing sellers — and their worst decisions.

"We trained the agent by studying the best sellers — their call transcripts, how they engage with customers — and teaching Jamie to do the same," he said.

"We also trained Jamie not to replicate the worst seller experiences."