"Poor and overconfident": developers are poor judges of LLM assertions

New arXiv paper: programmers are not only bad judges of LLM-generated assertions - they are overconfident in their judgment. The combo that derails AI code review.

The fact

The paper « Programmers Are Poor and Overconfident Judges of LLM-Generated Assertions » (arXiv:2607.08885, July 13, 2026) empirically documents a troubling pattern: when developers are asked to judge assertions generated by LLMs about code, they often make mistakes—and believe they are correct. Understanding and code review are yet central tasks of the profession, even more so since the arrival of generative tools.

Our reading

This result adds to a growing body of work: LLMs wrap their output in an eloquence that short-circuits doubt. A poorly formed assertion, but written in fluent English, passes human review. This is not a flaw of the developers—it is a well-known bias (fluency = truth) that the LLM medium amplifies. Corollary in the « hype-fatigue » thread: we have probably overestimated the value of the « human in the loop » when this human judges a polished output at face value.

To watch

Two product reactions to watch for: (a) IDEs that display visible uncertainty about AI assertions (beyond « accepted / rejected »); (b) teams that switch to automated review (executed tests, verified properties) rather than visual review.