Researchers Asked People a Question AI Almost Always Gets Wrong. Most Still Trusted the AI Anyway.

Here's a simple question: what color is the team's uniform in Bend It Like Beckham? If you're not sure, that's fine. Nobody expects you to know that off the top of your head.

A group of researchers asked people exactly that kind of question. Some answered alone. Others had an AI chatbot to consult first, one the researchers had specifically chosen because it usually got the answer wrong.

What happened next wasn't really about movie trivia at all.

The Setup

The study comes from Valerio Capraro of the University of Milano-Bicocca, working with Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome. Its title says exactly what it found: "AI advice suppresses people's willingness to say 'I don't know,' even when the advice is wrong and accuracy is incentivized."

• Participants answered questions about obscure visual details in films, deliberately picked because they'd be unlikely to appear in AI training data

• The main experiment used Step 3.5 Flash, a model chosen specifically because it was usually wrong on these questions

• That design choice mattered: it ruled out the explanation that people were simply making a sensible choice to trust an accurate tool

• Researchers also tested GPT-5.5, Claude Sonnet 4.6, and Gemini 3.5 Flash, which missed some of the same questions but got others right

• One group of participants had a financial incentive tied directly to giving the correct answer

The Numbers That Matter

Without AI, 44% of participants correctly said they didn't know the answer. With AI available, that number fell to 3%.

Accuracy moved the same direction. In the no-AI group, 27% of participants got the answer right. With AI in the mix, accuracy fell to 9%, roughly a third of the baseline.

Confidence did the opposite. Participants without AI reported 30% confidence in their answers. Participants with AI reported 76% confidence, more than double, while being far more likely to be wrong.

"People became much worse, the accuracy was only one third, but they were twice as confident," Capraro said.

Paying people to get it right helped, but only a little. Willingness to admit ignorance rose from 3% to 8%, and accuracy rose from 9% to 16%. Both numbers stayed well below what people managed entirely on their own.

Why This Isn't Just About Movie Trivia

The researchers were careful to rule out an obvious counterargument: maybe people were just correctly deferring to a smarter tool. That's why they picked a model that was usually wrong, and why some participants who would have answered correctly on their own changed their answer to a wrong one after asking the AI.

"For humans, the capacity to say, 'I don't know,' is very important because it represents the recognition of the limits of our own knowledge," Capraro said. "But now with AI, we can get an easy answer to virtually every question, so we wondered whether this would interfere with human capacity to say, 'I don't know,' to suspend judgment."

The finding echoes a term Wharton researchers coined earlier this year: "cognitive surrender," describing how people accept incorrect AI answers roughly 80% of the time while reporting higher confidence than those working without AI at all. Capraro's study adds a sharper point to that picture. It isn't only that people trust wrong answers once they see them. The mere availability of AI seems to suppress the habit of recognizing what you don't know in the first place.

The "I Don't Know" Study: FAQ

Jans Bock-Schroeder

Publisher & Founder of AI Angst

Coming from the world of art, photography, and the luxury market, Jans launched AI Angst in 2025 to explore the cultural, ethical, and psychological impacts of artificial intelligence. His work bridges creative vision with critical technology analysis, offering clarity in an era of rapid technological change.