A recent now-retracted paper in Psychological Science by Clark et al. (2020) caused some controversy lately in the psychological community. The authors found some correlations with other national-level data that could be theoretically important.

Among the reasons that people criticized the article was the authors’ use of estimated average national IQ scores (Becker, 2019). For some people the scores in some countries were unbelievably low:

This is one reasons why many people found the national IQ estimates invalid. Because there is no way that large percentages of people in these countries have an intellectual disability, it seems that there must be something wrong with the data.

However, the low estimated IQs for these countries do not imply that the majority (or even large percentages) of residents in these countries have an intellectual disability. That is because low IQ is not the sole diagnostic feature of an intellectual disability. According to the DSM-5:

The essential features of intellectual disability (intellectual developmental disorder) are deficits in general mental abilities (Criterion A) and impairment in everyday adaptive functioning, in comparison to an individual’s age-, gender-, and socioculturally matched peers (Criterion B). Onset is during the developmental period (Criterion C). The diagnosis of intellectual disability is based on both clinical assessment and standardized testing of intellectual and adaptive functions

American Psychiatric Association (2013, p. 37)

All three of these criteria must be met for a person to be diagnosed with an intellectual disability. Without any one of them, then a clinician should not diagnose a person with an intellectual disability.

Of these three criteria for diagnosis, an intelligence test can only provide evidence regarding Criterion A. The other two criteria require data about the presence and nature of functional impairments and a person’s developmental history. Intelligence tests do not provide this information (nor do test creators claim that they do).

People can have a low IQ score for a variety of reasons that have nothing to do with their actual intelligence (such as language barriers, low motivation, and malingering). For this reason, mental health professionals do not diagnose people with an intellectual disability solely on the basis of a low IQ. Indeed, because a person must perform lower than their “socioculturally matched peers” to have an intellectual disability, then by definition it is impossible for a majority (or even a substantially large minority) of people in a society to have an intellectual disability.

It is not possible to diagnose someone with an intellectual disability, let alone large numbers of people in a country, without examining how well they function in their day-to-day living. The inference that a low average national IQ indicates that many people have an intellectual disability is not supported.

IQ ≠ intelligence

It is also important to recognize that IQ and intelligence are not synonymous. Whether the average IQ for a group or nation is valid or not depends on the interpretation one wishes to draw (and, probably, the conditions that the individuals within the group experience). My viewpoint is that these average IQs are probably not a good measure of intelligence for many countries. But a more valid interpretation of these scores might be as a measure of human capital or developed reasoning ability. Therefore, the low estimated average IQs for some countries are not–by themselves–evidence that the IQ scores are invalid. Whether any score is “valid” hinges on how one interprets or uses the scores (American Educational Research Association et al., 2014).

That being said, it would be simplistic to take these scores at face value, and abnormally low averages are a good reason to investigate whether the test is functioning well in a particular population. IQ is a number that can often serve as a measure of intelligence, but–like any other psychometric score–it is also susceptible to influences from other sources that are not part of the construct being measured.

Map of estimated average national IQs worldwide, based on Becker (2019). Source.

Ironically, the logic of interpreting low average IQ scores as meaning that the majority of a country’s citizens have an intellectual disability is exactly the simplistic interpretation of IQ scores that no intelligence expert would ever make. Anyone making this inference from low average national IQ scores is only demonstrating their own ignorance about intellectual disability and the interpretation of IQ scores.

The critics are right to question the quality of data for some countries. The countries with the lowest average IQ scores tend to be impoverished and often have experienced civil unrest and war. They don’t have large-scale educational testing programs or an entrenched testing culture that regularly gives tests to representative samples. All these reasons (and others) decrease the data quality for countries with low average IQ scores.

Another problem is that the tests used for these countries are usually imported, and there is often little or no evidence that the tests are culturally appropriate for the examinees. The classic example of this is the Raven’s Progressive Matrices, which is a non-verbal test that is designed to be cross-culturally appropriate, but that still has problems when administered in sub-Sahara Africa:

Thus, I do not think that IQ is synonymous with intelligence, just as I do not think that low IQ is synonymous with an intellectual disability. IQ often serves as a measure of intelligence, but it is an imperfect one (just as all psychometric tests have their imperfections). Common sense, experience, and empirical research show that the imperfections are much more severe in countries that produce low average IQ scores, and we should be very skeptical that these scores measure intelligence in these countries.

However, I do not think that average national IQs are worthless. They do correlate with many important national-level variables (Jones, 2016; Rindermann, 2018). But it is not necessarily valid to say that “Country X is smarter than Country Y.” Rather, I think that these scores are an approximate rank order to how well the citizens of each nation have been trained to solve formal cognitive tasks. With the knowledge economy in its third decade, mastering this type of problem solving can be an important driver of economic growth and societal stability. The scores don’t have to correspond to intelligence to be important and worth studying.

Don’t like the data? Do something about it!

Many psychologists on social media seem content to just criticize Becker’s (2019) data. While discussion of the imperfections of national IQ estimates are valuable, it is a scientific dead end if the data quality do not improve. These criticisms should be a call to psychologists to collect better data about average national IQs with better tests and representative samples–and not an impetus to abandon estimating group-level IQ averages. As I stated in my upcoming book, In the Know: Debunking 35 Myths About Human Intelligence:

It is telling that scholars who demand extra care for controversial intelligence research do not design studies that meet their standards to investigate controversial topics. If their concern for methodological rigor were fully genuine, they would conduct the studies they demand. This would be the only way to both answer important scientific questions and ensure that those answers are based on trustworthy data. But the critics never seem interested in collecting data that they demand from others.

Warne (in press, p. 293).

If the average national IQ scores really are useless, then criticizing the data is only half the solution. It is important to collect the data that do not have the shortcomings of existing data. Even if a single article based on low-quality data is retracted, the research questions are still out there, needing to be answered.

Bad data lead to bad science. The claim that “here are the best data available” is not a blank check to publish bad research. Criticism in science is healthy, but is best when it changes research practices. So, if you don’t like the data on average national IQs, I encourage you to be part of the solution and collect the data you wish someone else had. That is the best way to help science progress.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association

American Psychiatric Association. (2013). Diagnostic and statistical manual for mental disorders (5th ed.). American Psychiatric Publishing.

Becker, D. (2019). The national IQ dataset (version 1.3.2). https://viewoniq.org/?page_id=9

Jones, G. (2016). Hive mind: How your nation’s IQ matters so much more than your own. Stanford University Press.

Rindermann, H. (2018). Cognitive capitalism: Human capital and the wellbeing of nations. Cambridge University Press.

Warne (in press). In the know: Debunking 35 myths about human intelligence. Cambridge University Press.

Note: I am not providing the full citation to the Clark et al. (2020) article because I do not want to raise a retracted article’s citation count.

css.php