Illusion of validity is a cognitive bias in which a person overestimates their ability to interpret and predict accurately the outcome when analyzing a set of data, in particular when the data analyzed show a very consistent pattern—that is, when the data "tells" a coherent story. This effect persists even when the person is aware of all the factors that limit the accuracy of their predictions, that is when the data and/or methods used to judge them lead to highly fallible predictions. Daniel Kahneman, Paul Slovic, and Amos Tversky explain the illusion as follows: "people often predict by selecting the output...that is most representative of the input....The confidence they have in their prediction depends primarily on the degree of representativeness...with little or no regard for the factors that limit predictive accuracy. Thus, people express great confidence in the prediction that a person is a librarian when given a description of his personality which matches the stereotype of librarians, even if the description is scanty, unreliable, or outdated. The unwarranted confidence which is produced by a good fit between the predicted outcome and the input information may be called the illusion of validity." Consistent patterns may be observed when input variables are highly redundant or correlated, which may increase subjective confidence. However, a number of highly correlated inputs should not increase confidence much more than only one of the inputs; instead higher confidence should be merited when a number of highly independent inputs show a consistent pattern.
Discovery and description This bias was first described by Amos Tversky and Daniel Kahneman in their 1973 paper "On the Psychology of Prediction".
In a 2011 article, Kahneman recounted the story of his discovery of the illusion of validity. After completing an undergraduate psychology degree and spending a year as an infantry officer in the Israeli Army, he was assigned to the army's Psychology Branch, where he helped evaluate candidates for officer training using a test called the Leaderless Group Challenge. Candidates were taken to an obstacle field and assigned a group task so that Kahneman and his fellow evaluators could discern their individual leadership qualities or lack thereof. But although Kahneman and his colleagues emerged from the exercise with very clear judgments as to who was and wasn't a potential leader, their forecasts proved "largely useless" in the long term. Comparing their original evaluations of candidates with the judgments of their officer-training school commanders months later, Kahneman and his colleagues found that their own "ability to predict performance at the school was negligible. Our forecasts were better than blind guesses, but not by much." Yet when asked to again to assess yet another group of candidates, their judgments were as clear as before. "The dismal truth about the quality of our predictions," recalled Kahneman, "had no effect whatsoever on how we evaluated new candidates and very little effect on the confidence we had in our judgments and predictions." Kahneman found this striking: "The statistical evidence of our failure should have shaken our confidence in our judgments of particular candidates, but it did not. It should also have caused us to moderate our predictions, but it did not." Kahneman named this cognitive fallacy "the illusion of validity". Decades later, Kahneman reflected that at least part of the reason for his and his colleagues' failure in assessing the officer candidates was that they had been confronted with a difficult question but had instead, without realizing it, answered an easier one instead. "We were required to predict a soldier's performance in officer training and in combat, but we did so by evaluating his behavior over one hour in an artificial situation. This was a perfect instance of a general rule that I call WYSIATI, 'What you see is all there is.' We had made up a story from the little we knew but had no way to allow for what we did not know about the individual’s future, which was almost everything that would actually matter."
… excerpt ends here. Continue reading the full article.
