neur.ro

Dunning-Kruger effect

type: effect · origin: study · evidence: supported · themes: judgements and probabilities

Claim assessed: Those who score lowest on a test overestimate their performance the most.

Published · how we work

In short

The Dunning-Kruger effect is the name given to the observation that people with the weakest scores on a test overestimate their performance the most, while those with the best scores tend to underestimate theirs slightly. The pattern itself shows up consistently. The debate is about why: from a lack of self-knowledge in weak performers, as the authors proposed, or, at least in part, from how the data are analysed.

What it says

The original study. Kruger and Dunning (1999) gave Cornell students tests of humour, grammar and logic, then asked them to estimate their performance. Across four studies, those in the bottom quarter overestimated themselves heavily: they scored at the 12th percentile but placed themselves, on average, at the 62nd. The authors proposed that the least skilled carry a “dual burden”: they make mistakes and, on top of that, they lack the very ability to recognise their mistakes. When their skills were improved, they judged their limits better. We haven’t read the study in full; we describe it from its abstract and from Magnus and Peresetsky (2022) and Mazor and Fleming (2021).

The statistical explanation. As early as 2002, other researchers noted that the pattern can appear without any psychological deficit. Two well-known facts are enough: most people think they are above average, and imperfect estimates tend to regress towards the mean. Magnus and Peresetsky (2022) add a simple argument: the scale has limits. A student who scores 95 out of 100 has very little room to overestimate and a lot of room to underestimate; one who scores 10 is in the opposite position. With data from 665 students in Moscow who predicted their grade on a statistics exam, a model that only takes these limits into account reproduces the pattern almost perfectly. Their conclusion: the effect exists, but it says nothing about human nature.

Top: in Kruger and Dunning's tests, people in the bottom quarter scored at the 12th percentile but placed themselves at the 62nd. Bottom, illustration: someone scoring 10 out of 100 has 10 points of room to underestimate and 90 to overestimate; someone scoring 95 has 95 points of room to underestimate and only 5 to overestimate. Overestimation or statistics? Kruger, Dunning (1999); Magnus, Peresetsky (2022) The bottom quarter, in percentiles tests of humour, grammar and logic actual score 12 where they placed themselves 62 How much room the error has, on a 0–100 scale an illustration, not data scoring 10: little room down, a lot up scoring 95: a lot of room down, little up The scale's limits alone produce part of the pattern.
Top: Kruger and Dunning (1999), from the article's abstract. Bottom: an illustration of Magnus and Peresetsky's (2022) argument; grey: room to underestimate, coloured: room to overestimate.

But not only statistics. Jansen, Rafferty and Griffiths (2021) repeated two of the original studies with about 4,000 participants each. They built a model in which all participants judge equally rationally and showed that it also produces the pattern, because people start from an expectation that they will do fairly well. They then compared this model with one in which weak performers judge their own answers worse. The data at the extremes, the very weak and the very strong, fitted the second model better. Mazor and Fleming (2021), commenting on the study, conclude that the effect isn’t just a statistical artefact, but that it isn’t clear whether it is a separate problem of self-knowledge or the same weakness showing up twice: once in the test and once in the estimate.

Example

Two classmates finish the same exam. One got half the questions wrong but thinks they got a good grade, because they can’t tell which answers are wrong. The other answered almost perfectly but doubts a few questions and estimates their grade lower than it is. The example is an illustration, not a study cited here.

How to apply it

The steps below are a practical way to apply it that we propose, based on the studies cited.

  1. Don’t use the effect as an insult. “Dunning-Kruger” has become a label for people we consider stupid. The studies talk about group averages and specific tests, not about individuals.
  2. Ask for concrete feedback, not just your own impression. If you can’t recognise a wrong answer by yourself, you need someone or something to show it to you.
  3. When you learn something new, assume your picture of what you know is incomplete. Kruger and Dunning show that, as skill grows, so does the ability to see your limits.
  4. When you read about the effect, look at the graph. A pattern in which weak performers seem to overestimate themselves can also come from the limits of the scale and from regression to the mean. See the entry on regression to the mean.

Limits and nuances

Sources

See also: Circle of competence, Curse of knowledge, Regression to the mean