For over a century, researchers have studied intelligence, developing measures like the Intelligence Quotient (IQ), exploring its contributing factors, and examining its implications. Despite extensive research, IQ remains a controversial topic—some argue it is a crucial measure that captures much of intelligence, while others dismiss it as meaningless or even pseudoscientific. But what's actually true?
We set out to answer this question (and many others about intelligence testing) and, to that end, spent over a year conducting a large-scale study of intelligence tests. We’ve investigated all sorts of questions, like:
Is IQ real or B.S.? Does it even make sense to talk about assigning a single number for performance on intelligence tests?
How predictive is IQ of a variety of life outcomes?
What is IQ correlated with?
As well as investigating over 30 other questions related to IQ. In conducting this research, we’ve come to understand both the strengths and short-comings of traditional intelligence testing, and made our own cognitive assessment tool in the process that we’re launching today!
Read on to:
Learn our answers to the questions above based on our research
Learn how the new Clearer Thinking Cognitive Assessment tool engages with our empirical findings and the (sometimes dark) history of intelligence testing
What do intelligence tests measure?
Traditional intelligence tests measure IQ. Some people think of IQ as a good proxy for intelligence and others don’t. But what is IQ?
It might surprise you to learn that nobody really knows.
One major claim related to intelligence testing is that if you test people on a whole bunch of different kinds of tasks commonly thought to involve intelligence (that can be done in a simple testing environment, such as within a web browser or within a psychology lab) you'll find that almost every one of these tasks positively correlates with almost every other one of these tasks. This finding of positive correlations across nearly all cognitive tasks is sometimes referred to as the "positive manifold."
But is this really what you get when you test a wide range of intelligence-related tasks? We set out to test this in our giant study on intelligence, which included 3691 study participants. But we also wanted to do so in a way that didn't involve just those intelligence tasks usually found in IQ tests. To that end, we included a very diverse set of potential intelligence tasks, 62 such tasks in total. Some of these were taken from the academic literature, others came about through crowdsourcing, and still others were new tasks that we developed ourselves (that we don't believe had ever been tested before). These intelligence-related tasks include everything from solving math problems, to vocabulary, to applying logic, to solving riddles, to game playing, to reaction time, to instruction following, to memorization to mentally rotating objects, and beyond. So our test included a great many things that one would normally think of as being intelligence-related, but not everything. For instance, it did not include tests for how well someone could navigate the natural world, complex motor skills (like dancing), or high-level creative skills (like making a piece of art). But it did include a great number of intelligence-related tasks.
So, what did we find? Well, exactly as the academic literature claims, performance on nearly every such task was positively correlated to performance on nearly every other task, as shown in the image below. Positive correlations are shown in blue and negative ones in red. The tasks that are most correlated with other tasks are at the top, and those least correlated to other tasks are at the bottom. Notice how there is a great deal of blue and very little red.
A second claim in the academic literature on IQ is that there is a hidden factor, represented by a weighted average of task performances, that accounts for a substantial percentage of the variation in scores. This factor can be computed using a method known as "principal component analysis" - essentially, trying to find a way to take a weighted average of the appropriately standardized test scores so as to produce a single number assigned to each study participant that accounts for as much variation (in all of their task scores) as possible. This single number is generally given the name ‘IQ.’ When we conduct this analysis, we find that IQ accounts for about 45% of the variance in scores across the tasks (as seen in the chart above). This is a fairly typical number when compared to other studies. So IQ, a single number that can be assigned to each study participant, enabled a reasonable degree of prediction at how that person would perform across a wide range of tasks (though such a prediction was far from perfect).
This seems to indicate that IQ is latching on to a real phenomenon. There really does seem to be some feature in the data that ‘IQ’ picks out. Diverse cognitive tasks tend to correlate, revealing an underlying factor - often called “g” - that IQ scores approximate.
But what is IQ truly measuring?
Nobody knows. But there are a variety of different theories. For example:
Theory 1: Single Underlying Resource
This theory posits that the positive correlations between performance at different tasks exist because there is a single, unified cognitive resource that intelligence-related tasks all make use of. The more of this resource a person has, the higher their IQ.
Theory 2: Working Memory / Executive Control
Some people think that IQ is essentially a measure of one's ability to manipulate, integrate and hold information in working memory, and then to act in a goal-directed manner via executive control. If most or all intelligence related-tasks fit this description, that could explain why performance on them is positively correlated.
Theory 3: Parieto-frontal integration theory
This theory claims that "intelligence [relates] to how well different brain regions integrate to form intelligent behaviors" and that "large scale brain networks connect brain regions, including regions within frontal, parietal, temporal, and cingulate cortices, underlie the biological basis of human intelligence." (source) It is based, in part, on findings that link IQ test performance to specific types of brain activity, suggesting that IQ could be measuring the extent to which (and how efficiently) brain regions integrate and how well they work together.
Theory 4: Process Overlap Theory (Kovacs & Conway)
This model suggests that "cognitive tests tap domain-general executive processes, identified primarily in research on working memory, as well as more domain-specific processes. Executive processes are tapped in an overlapping manner across cognitive tests such that they are required more often than domain-specific ones." This theory suggests that the correlation between intelligence-related tasks is due to overlap they happen to have in using distinct cognitive processes (source)
Theory 5: Cattell-Horn-Carroll (CHC) Theory
This is a hierarchical model that proposes that intelligence can be divided in a hierarchical structure, with g at the top, followed by broad abilities beneath (such as "long-term storage and retrieval" and "fluid reasoning") and narrow abilities beneath those, such as "spelling ability" and "visual memory." While this is the most widely accepted academic theory about the structure of cognitive abilities, it doesn't actually answer the question of what IQ (or g) truly is, though it does make claims about the nature of g, such as that it can be decomposed, and makes specific claims about what those subdivisions look like.
Okay, so it seems IQ captures something real, but there is disagreement about what it is, exactly, that it's capturing.
Here’s another open question: It seems like whatever IQ captures must be related to what people generally have in mind when they use the word ‘intelligence' since it predicts performance at many intelligence related tasks. But how much of intelligence is captured by IQ? There are lots of possibilities.
For example, it could be that IQ is a perfect proxy for what people mean by "intelligence", or it could be that there are a great many forms of intelligence (such as emotional intelligence, creativity, social intelligence, spiritual intelligence, academic intelligence, street smarts, and more) and that these are only partially (or in some cases, perhaps not at all) captured by IQ. Here are just some of the many possibilities represented pictorially:
The area inside the dotted line represents the full extension of the ordinary meaning of the word ‘intelligence’. Different colored circles represent different things that may fall inside or outside of that meaning (the circles shown are not intended to be exhaustive).
Which case do you think is most accurate?
It seems reasonable to argue against case 1 on the grounds that there are things it is reasonable to call ‘intelligence’ that are not accurately captured with the single number of a person's IQ score. An additional argument against case 1 is that people with the same IQ score show different levels of ability at distinct intelligence related tasks - for instance, one person may be better at math related tasks and another person with the same IQ score may be better at verbal tasks.
It also seems reasonable to argue against case 4 on the grounds that many of the things that are tested by IQ tests (e.g., reasoning skills, verbal ability, and math ability) and which correlate with IQ scores (e.g., educational attainment) do seem relevant to intelligence.
But, as we've mentioned, the exact nature of the relationship between IQ and intelligence is not settled.
The strengths and limitations of the predictive power of intelligence testing and IQ
It is well known that IQ can predict some things about people. But what can it predict, and how well? In our large study, we found that IQ was correlated (either positively or negatively) with all sorts of things, such as:
Actively open-minded thinking: a fairly large positive correlation (r = +0.43), using a sample size of n = 670. (This phrase refers to a disposition to seek out information that challenges one’s existing views and a willingness to update one’s views in light of new evidence.)
Self-reported household income: a small positive correlation (r = 0.15), using a sample size of n = 3688
Self-reported job performance: a small positive correlation (r = 0.18), using a sample size of n = 686. However, when the sample is divided into three groups (one group with lower IQ scores (92 and below); one group with middling IQ scores (93 to 108); and one group with higher IQ scores (109 and above)) we find a stronger correlation (r = 0.48, n = 175) between job performance and IQ scores in the lower-scoring group. However, the correlation between job performance and the other two groups basically disappears.
Pathological celebrity worship: a fairly strong negative correlation r = -0.42, from a sample size of n = 681. (This was measured by asking whether people agree with a variety of statements along the lines of, “If I met my favorite celebrity, and they asked me to do a favor for them, I would do it, even if it was illegal and dangerous.”)
And loads more. When we release the full report of our study (within the next few weeks), you’ll be able to see many other correlations that we found.
These correlations suggest that intelligence testing can help us to understand ourselves.
However, when it comes to using our study data to determine how much IQ can tell us about life outcomes, the results are somewhat limited. For instance, we compared how much variation in life outcomes could be explained by IQ versus by personality (as measured by the Big Five personality traits: openness, conscientiousness, extraversion, agreeableness, and neuroticism).
We found that for almost every outcome we tested, personality (i.e., taking all five of the Big Five personality traits together) was a better predictor of that outcome than IQ.
The only outcomes we tested where we found that IQ fared similarly accurate as a predictor were high-school and college GPA scores -for all others personality was a better predictor. This gap was most pronounced when it came to happiness and life satisfaction.
We’ve illustrated this graphically with circles:
The biggest, white circle represents the whole outcome, and smaller colored circles represent the percentage of that outcome that is explained either by IQ alone (green circle), personality alone (red circle) or IQ and personality together (blue circle - labeled "total"). In all cases, the personality circle is bigger than the IQ circle meaning that it explains a higher percentage of variance in an outcome. In some cases personality circle is almost as big as the blue circle, meaning that IQ adds only a negligible predictive power over personality (i.e., we can predict that outcome almost equally well using only personality compared to using personality and IQ together.
The upshot here is that IQ has some ability to predict a number of things - but it's not the only predictor of important life outcomes, and it's very far from destiny! While IQ correlates with some outcomes, your performance on cognitive tests cannot be used to accurately infer what you will achieve, or how happy you’ll be. IQ tests have been used in this way in the past, and such uses are not grounded in empirical fact.
So, is testing useless?
While cognitive assessments are only useful in some situations, we think cognitive assessments can be helpful for individuals. That's why we're now making our cognitive assessment available starting today, which gives you feedback on a number of aspects of your abilities. Here are some ways we believe it can be useful:
1️⃣ Gaining more self-knowledge
Our assessment helps you understand how you think and learn best. This awareness can be applied to a number of areas of your life, from personal relationships to problem-solving at work.
Even though performance on different intelligence-related tasks tends to all be correlated, those correlations are often only moderate in size, and that means that even if you compare people who get the exact same IQ score, you’ll see variation in their strengths and weaknesses. That's why our cognitive assessment can help you understand your relative strengths and weaknesses much more deeply than can a single IQ score.
2️⃣ Making more informed decisions
When you’re making decisions about where you want to work, and in what field, what are the factors you consider? We think that at least the following three make sense:
Your level of interest in or passion for the work
How good at the work you think you are
The effects of the work on yourself and others
Taking our assessment will help you to understand your cognitive strengths and weaknesses. This, in turn, can help you make more informed judgments about b (how good at the work you think you are). For instance, if you find that you excel at numerical reasoning, you’ll have reason to think you would be especially good at work involving numerical reasoning - work like data analysis, financial modeling, engineering, accounting, market research, or scientific research.
This means we think this test can be used for cases like the following:
If you're a student considering which subjects to study, your results might reveal a natural aptitude for logical thinking and numerical reasoning, suggesting that STEM fields could be a good fit. Conversely, your results might show weaknesses in those areas but strong linguistic and conceptual knowledge scores - this might give you a reason to lean away from STEM fields, towards humanities or social sciences. Or, if you decide to pursue STEM fields despite a weakness, the cognitive assessment can help you understand what skills you may need to work to improve in order to excel.
If you're considering a career change, the assessment can highlight your strongest cognitive abilities, helping you identify careers where you're more likely to thrive. Perhaps your spatial visualization skills are exceptional, suggesting a potential career in architecture or design.
Our test can’t help you with every aspect of this kind of decision (you’ll also want to consider your level of interest / passion, and the effects of the work) but it can help you with reflecting on how good you might be at certain kinds of work.
The Ethics of Testing
Intelligence testing has a dark and disturbing moral history. Some early tests were developed by eugenicists (such as the psychologist Lewis Terman) and used to support eugenics, including enacting forced sterilization programs in the US. They’ve also been associated with racial discrimination, draconian immigration policies, segregation in schools, the promotion of so-called ‘scientific’ racism, attempts to justify large economic inequalities, and much more.
On the other hand, intelligence testing has also been used for good - for instance to help identify young people from underprivileged backgrounds that could benefit from being fast tracked academically, and to help people better understand themselves.
In conducting our research and creating our tool, we’ve tried to be careful about and sensitive to the dubious aspects of its history to help make sure we're doing good and not harm through this work. We don’t want this (or indeed any) tool used in harmful ways. That's one of the reasons we've decided to keep this tool underwraps in a way that we haven't done so for previous tools: whereas all our prior tools are free, we'll be charging for use of this tool (with proceeds supporting our mission), with a licensing agreement that stipulates that the tool is for your own personal individual use only, with other uses only subject to our approval, so that we can monitor how it gets used and make sure it's used in ethical ways.
We’ve also made sure to make clear, within the tool, that these results are not destiny. They can help you understand yourself, but they can’t predict your future with great accuracy.
The Clearer Thinking Assessment
Our cognitive assessment tool goes far beyond simple intelligence tests because it reveals your cognitive abilities across seven key dimensions of intelligence:
Right after completing the cognitive assessment, you'll receive a comprehensive report detailing your strengths and areas for growth, along with tailored insights and recommendations connecting your results to areas such as education, career path, and personal growth.
Your Investment in Knowledge
We are currently offering the Cognitive Assessment for $50.
Currently, the Cognitive Assessment is also available for free to Clearer Thinking Supporters for no extra charge. If you're interested in receiving it along with a host of other perks starting at $29 per month, consider becoming a Supporter.
If you're already a subscriber or a Supporter, make sure to enter the email address that you use in Clearer Thinking at checkout.
Clearer Thinking is a not-for-profit project (we're focused on helping individuals and improving society), so all proceeds will be reinvested into research and the development of new initiatives that align with our mission.
Whether you choose to use the Cognitive Assessment as a monthly Supporter or through a one-time payment, here is what you get:
Three test attempts
Results across seven dimensions with tailored insights and recommendations
Optionally (if you activate it): professional certificate with an estimate of your IQ score
Career and learning recommendations based on your results
Are you ready? Click the button below to get a comprehensive report of your cognitive abilities today!