(Image credit: Shutterstock)
Most of us know ChatGPT isn't perfect. It can hallucinate facts, confidently invent citations and occasionally give completely incorrect answers. But a new study made me wonder whether the bigger problem isn't what AI gets wrong, but how AI might be changing about us as critical thinkers.Researchers found that when people relied on an AI assistant for difficult questions, they became far less likely to admit they didn't know the answer. Even more surprising, they grew much more confident in their responses, even when the AI led them straight into an error.That hit a little too close to home. I use ChatGPT all the time as a brainstorming partner, research assistant and sounding board. So, I decided to recreate the experiment myself to see whether I had quietly fallen into the same habit.The results weren't exactly what I expected.What the study found Researchers from Carnegie Mellon University and Microsoft Research designed an experiment around a simple question: What happens when people consult AI instead of relying on their own judgment?To find out, they intentionally used questions that the AI was likely to answer incorrectly. Participants could choose one of three options:Answer on their ownGet instant access to breaking news, the hottest reviews, great deals and helpful tips.Ask the AI for helpSay, "I don't know"Without AI, participants admitted they didn't know the answer nearly half the time. Once AI entered the picture, that changed dramatically.Instead of saying "I don't know," most participants accepted the AI's suggestion — even when it was wrong. Their confidence increased sharply, but their actual accuracy stagnated. Humans were simply becoming confidently incorrect.The researchers argue that this points to automation bias: our tendency to trust computer-generated recommendations simply because they're presented confidently.This study suggests that a polished, authoritative answer can make us feel more certain than the evidence warrants.I decided to try it myself
