LLMs Pass 99.8% of Survey Attention Checks, Posing 'Existential Threat' to Online Research
A new PNAS paper demonstrates that large language models can evade virtually all standard bot-detection methods used in online surveys — threatening the integrity of behavioral science data collected since at least 2022.
A paper published in PNAS presents evidence that large language models can pass attention checks, CAPTCHAs, and quality-control measures used in online surveys at a 99.8% rate, as @jayvanbavel reported. The researchers describe this as an "existential threat to online survey research" — strong language for an academic paper, but arguably warranted given how much of behavioral science, market research, and political polling relies on these methods.
Unlock the full briefing
Get every story in today's briefing, the full archive, and the daily AI intelligence brief.
All stories today
Full archive
Daily brief
Cancel anytime. Payments powered by Stripe.