Saturday, August 18, 2012

Flynn Effect Cohort Comparisons

I'm going to follow up a little further to my previous two posts and talk about making cohort comparisons under a Flynn effect.

It's quite common for researchers to talk about the Flynn effect advantage (or disadvantage) of one population cohort compared to another. For instance, in Dickinson & Hiscock (2010) and Agbayani (2011), this type of statement is frequent. Unfortunately for these researchers, a Flynn effect cohort comparison is not technically correct. In the best of circumstances it can be ambiguous, and as it is typically done, it's wrong.

The best way to see this is to understand first that the Flynn effect works across time, not across cohorts. Nearly every empirical study involving the Flynn effect shows that raw intelligence scores tend to increase universally and uniformly over time, no matter the age group, no matter the population. Whoever you are, wherever you are, no matter how old you are, what the Flynn effect cares about is how much time has passed. If you make a comparison across ten years of time, then you'll get ten years worth of Flynn effect. If you make a comparison across fifty years of time, you'll get fifty years worth of Flynn effect. If you make a comparison across zero years of time, then you'll get zero years worth of Flynn effect (which is to say, no Flynn effect at all).

So what does this mean for a cohort comparison? Let me return to the idealized chart of data I used in Intelligence as Field, and let's examine the raw intelligence history of two cohorts within it, the population born at time 100 and the population born at time 140.

AgeRaw Intelligence Scores by Age and Year
95 40.8 41.6 42.5 43.3 44.2 45.1 46.0 46.9 47.9 48.8 49.8
85 45.6 46.5 47.5 48.4 49.4 50.4 51.4 52.4 53.5 54.6 55.7
75 50.4 51.4 52.5 53.5 54.6 55.7 56.8 58.0 59.1 60.3 61.5
65 54.0 55.1 56.2 57.3 58.5 59.7 60.9 62.1 63.4 64.6 65.9
55 55.8 56.9 58.1 59.2 60.4 61.7 62.9 64.2 65.5 66.8 68.1
45 57.6 58.8 59.9 61.2 62.4 63.7 64.9 66.2 67.6 68.9 70.3
35 58.8 60.0 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8
25 60.0 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8 73.3
15 51.0 52.0 53.1 54.2 55.2 56.4 57.5 58.7 59.8 61.0 62.3
5 15.0 15.3 15.6 15.9 16.2 16.6 16.9 17.3 17.6 18.0 18.3
105 115 125 135 145 155 165 175 185 195 205
Year

The assumption that gets blindly made by researchers is that the birth year 140 cohort (BY140) is more intelligent than the birth year 100 cohort (BY100), due to the Flynn effect. But that's technically incorrect, or at the very least, it's technically ambiguous. The result of the comparison is going to very much depend on how the comparison gets made.

If the comparison is across time, then yes, BY140 will always emerge as more intelligent than BY100. For instance, if we are comparing the 25 year-olds of BY140 to the 25 year-olds of BY100, then there is forty years of time span between those comparison points, and thus forty years worth of Flynn effect. The same result happens if we compare the 65 year-olds of BY140 to the 65 year-olds of BY100. Cohort comparisons that span across time will always reveal the specific impact of a Flynn effect.

But that's not how researchers typically do it.

Researchers typically make their comparison at a particular point in time. For instance, researchers will say something like, while comparing BY100 to BY140 at time 165, a Flynn effect adjustment must be made to account for the Flynn effect disadvantage BY100 has relative to BY140. Or it might be put this way: the relative scores of BY100 and BY140 at time 165 are distorted due to the Flynn effect.

Such statements are pure nonsense.

In any comparison made at a particular point in time, the Flynn effect disappears. To repeat: the Flynn effect works across time, not across cohorts. If the cohort comparison being made involves no span of time, then there can be no Flynn effect distinction. Period.

As I mentioned in Intelligence as Field, there seems to be an odd sense floating about within the research community that one's Flynn effect is established once and for all at one's birth. It is this odd sense that must be leading people to assume that the Flynn effect works across cohorts and not across time. But the empirical data does not support that assumption. The empirical data shows that the Flynn effect continues to work equally at all places and at all times—it works throughout a person's entire lifetime, it is not established once and for all at one's birth. Therefore, the Flynn effect is not a function of birth or of cohorts, it is purely a function of time.

The mistakes I'm pointing out here are extremely common within the intelligence research community, and truth be told it's a shame. These mistakes mask the true nature of the Flynn effect, a phenomenon which has very much to tell us about the nature of human intelligence. But the Flynn effect is only going to speak to us if we don't first go out of our way to misunderstand it.


References

Agbayani, K. A. (2011). Patterns of age-related IQ changes from the WAIS to WAIS-III after adjusting for the Flynn effect. Retrieved online from http://repositories.tdl.org/uh-ir/handle/10657/236.

Dickinson, M. D. & Hiscock, M. (2010). Age-related IQ decline is reduced markedly after adjustment for the Flynn effect. Journal of Clinical and Experimental Neuropsychology, 32(8), 865-870.

Friday, August 17, 2012

Dickinson, Hiscock and Agbayani

[Note: as with the previous post, if the data charts below are difficult to read, try reading the analysis at this alternative location. ]

As a follow up to my previous post, Intelligence as Field, I would like to talk about two papers that essentially cover much of the same ground: Dickinson & Hiscock (2010) and Agbayani (2011). Unfortunately, I can't find a non-paywalled version of Dickinson & Hiscock (2010), and therefore I can't link to its full text, nor have I been able to read more than its title and abstract. [This is where I would normally begin a long rant about the ridiculously closed nature of science these days, but that's a worn-out subject, so let's move on.] To the rescue, Agbayani (2011) is available online and it provides all the essential information regarding the results and methods of Dickinson & Hiscock (2010). Agbayani is apparently a student of Hiscock, and Agbayani (2011) is a thesis paper that both outlines the approach of Dickinson & Hiscock (2010) and extends the range of its data and analysis.

The approach these authors take can be outlined as follows:
  • They use data from WAIS, WAIS-R, and WAIS-III to compare intelligence scores across age groups and across time.
  • They use a reverse norming methodology to place all scores on an equal footing, so that they can be directly compared as though they were raw scores from the same exam.
  • They adjust these raw scores for the Flynn effect, adding an empirically reasonable amount to the older age group scores to account for the generational difference between the older age groups and the younger age groups.
  • These adjusted raw scores are labeled as true aging effect scores and are shown to be similar to the pattern of scores that show up for individuals under longitudinal studies.
This approach of Dickinson, Hiscock and Agbayani is essentially equivalent to the approach I took with my idealized chart of data in Intelligence as Field. To see this, you can consider one of the charts of data with which that essay began and one of the comparative data sets that arises from it:

AgeRaw Intelligence Scores by Age and Year
95 40.8 41.6 42.0 42.5 43.3 44.2 45.1 46.0 46.9 47.9 48.8 49.8
85 45.6 46.5 47.0 47.5 48.4 49.4 50.4 51.4 52.4 53.5 54.6 55.7
75 50.4 51.4 51.9 52.5 53.5 54.6 55.7 56.8 58.0 59.1 60.3 61.5
65 54.0 55.1 55.6 56.2 57.3 58.5 59.7 60.9 62.1 63.4 64.6 65.9
55 55.8 56.9 57.5 58.1 59.2 60.4 61.7 62.9 64.2 65.5 66.8 68.1
45 57.6 58.8 59.3 59.9 61.2 62.4 63.7 64.9 66.2 67.6 68.9 70.3
35 58.8 60.0 60.6 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8
25 60.0 61.2 61.8 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8 73.3
15 51.0 52.0 52.5 53.1 54.2 55.2 56.4 57.5 58.7 59.8 61.0 62.3
5 15.0 15.3 15.4 15.6 15.9 16.2 16.6 16.9 17.3 17.6 18.0 18.3
105 115 120 125 135 145 155 165 175 185 195 205
Year


    Age   
   Without 
   Flynn Effect   
   With 
   Flynn Effect   

   Difference   
85 47.0 55.7 8.7
75 51.9 60.3 8.4
65 55.6 63.4 7.8
55 57.5 64.2 6.7
45 59.3 64.9 5.6
35 60.6 65.0 4.4
25 61.8 65.0 3.2
15 52.5 54.2 1.7
5 15.4 15.6 0.2

If you were to apply the approach of Dickinson, Hiscock and Agbayani to the original chart of data, you would arrive at exactly the same comparative data set, only with a different set of labels:


    Age   
   Scores that  
   Reflect AGD   
   Scores that  
   Reflect TAE   

   FED   
85 47.0 55.7 8.7
75 51.9 60.3 8.4
65 55.6 63.4 7.8
55 57.5 64.2 6.7
45 59.3 64.9 5.6
35 60.6 65.0 4.4
25 61.8 65.0 3.2
15 52.5 54.2 1.7
5 15.4 15.6 0.2

In the terminology of Dickinson, Hiscock and Agbayani, AGD stands for age group difference, reflecting the type of pattern that emerges from cross-sectional studies (that is, from reading up the chart at any given time). TAE stands for true aging effect and reflects the type of pattern that emerges from longitudinal studies (that is, from reading diagonally across the chart for any population cohort). FED is the Flynn effect difference.

In a certain sense, I'm quite pleased that Dickinson & Hiscock (2010) and Agbayani (2011) exist. They are the nearest thing I can find to a real-world analysis similar to what I outlined in Intelligence as Field, and of course it is gratifying to know that the real-world outcome turns out to be essentially the same as my idealized approach.

On the other hand, there is a major problem. Although it appears to me that Dickinson, Hiscock and Agbayani have done a creditable job in the gathering of their data, they have also managed to utterly mangle its interpretation.

In reading the conclusions these authors draw from their analysis (indeed, in reading through their entire approach to the problem), one quickly realizes that they are (mistakenly) saying the following:
  • The age-based differences that show up in cross-sectional studies (reading up the chart at any given time) are distorted because of the Flynn effect.
  • By adjusting for the influence of the Flynn effect, one arrives at a longitudinal set of values (reading diagonally across the chart) that represents the true age-based difference in individuals—that is, the age-based difference that would show up in the absence of a Flynn effect.
That logic is completely backwards.

The only way to make logical sense of the data is to state it the other way. The cross-sectional studies (reading up the chart at any given time) are the true age-based differences—that is, the age-based differences that would show up in the absence of a Flynn effect. The longitudinal values (reading diagonally across the chart) represent the combination of age-based differences and the Flynn effect. The reason that the longitudinal raw scores remain fairly constant across adulthood is that the two competing influences (age-based decline and Flynn effect increase) are in rough equilibrium.

As far as I can tell, there is no reasonable way to make sense out of the Dickinson/Hiscock/Agbayani interpretation. To see this, consider what must happen to the data under and not under the influence of a Flynn effect. Let me use a version of my idealized chart of data, and let's assume there is no Flynn effect for the first fifty years. Under these conditions and under my interpretation, the chart of data would look something like this:

AgeRaw Intelligence Scores by Age and Year
95 40.8 40.8 40.8 40.8 40.8 40.8
85 45.6 45.6 45.6 45.6 45.6 45.6
75 50.4 50.4 50.4 50.4 50.4 50.4
65 54.0 54.0 54.0 54.0 54.0 54.0
55 55.8 55.8 55.8 55.8 55.8 55.8
45 57.6 57.6 57.6 57.6 57.6 57.6
35 58.8 58.8 58.8 58.8 58.8 58.8
25 60.0 60.0 60.0 60.0 60.0 60.0
15 51.0 51.0 51.0 51.0 51.0 51.0
5 15.0 15.0 15.0 15.0 15.0 15.0
105 115 125 135 145 155 165 175 185 195 205
Year

But by the account of Dickinson, Hiscock, and Agbayani, the chart would need to look much different. Since they are saying that the true aging effect scores reflect age-based differences sans a Flynn effect, then their chart of data (under no Flynn effect) would need to look more like this:

AgeRaw Intelligence Scores by Age and Year
95 48.0 48.0 48.0 48.0 48.0 48.0
85 53.6 53.6 53.6 53.6 53.6 53.6
75 58.0 58.0 58.0 58.0 58.0 58.0
65 61.0 61.0 61.0 61.0 61.0 61.0
55 61.7 61.7 61.7 61.7 61.7 61.7
45 62.4 62.4 62.4 62.4 62.4 62.4
35 62.5 62.5 62.5 62.5 62.5 62.5
25 62.5 62.5 62.5 62.5 62.5 62.5
15 52.1 52.1 52.1 52.1 52.1 52.1
5 15.0 15.0 15.0 15.0 15.0 15.0
105 115 125 135 145 155 165 175 185 195 205
Year

Now consider what would happen if a Flynn effect kicked in beginning at time 155.

In my chart and under my interpretation, the progression is quite natural. Raw scores begin to go up by say 2% every ten years for all age groups, and what results is a chart of data for years 165 to 205 that has all the same patterns we currently see in the empirical data for humans. Note that the pattern of age-based differences remains invariant under the changing Flynn effect assumptions:

AgeRaw Intelligence Scores by Age and Year
95 40.8 40.8 40.8 40.8 40.8 40.8 41.6 42.5 43.3 44.2 45.1
85 45.6 45.6 45.6 45.6 45.6 45.6 46.5 47.5 48.4 49.4 50.4
75 50.4 50.4 50.4 50.4 50.4 50.4 51.4 52.5 53.5 54.6 55.7
65 54.0 54.0 54.0 54.0 54.0 54.0 55.1 56.2 57.3 58.5 59.7
55 55.8 55.8 55.8 55.8 55.8 55.8 56.9 58.1 59.2 60.4 61.7
45 57.6 57.6 57.6 57.6 57.6 57.6 58.8 59.9 61.2 62.4 63.7
35 58.8 58.8 58.8 58.8 58.8 58.8 60.0 61.2 62.4 63.7 65.0
25 60.0 60.0 60.0 60.0 60.0 60.0 61.2 62.4 63.7 65.0 66.3
15 51.0 51.0 51.0 51.0 51.0 51.0 52.0 53.1 54.2 55.2 56.4
5 15.0 15.0 15.0 15.0 15.0 15.0 15.3 15.6 15.9 16.2 16.6
105 115 125 135 145 155 165 175 185 195 205
Year

But what are Dickinson, Hiscock and Agbayani going to do? How can they reasonably introduce a Flynn effect at time 155 and still remain true to the empirical data? For instance, they can't just begin to boost scores across all age groups, because then their chart would end up looking like this:

AgeRaw Intelligence Scores by Age and Year
95 48.0 48.0 48.0 48.0 48.0 48.0 49.0 50.0 51.0 52.0 53.0
85 53.6 53.6 53.6 53.6 53.6 53.6 54.6 55.7 56.9 58.0 59.2
75 58.0 58.0 58.0 58.0 58.0 58.0 59.2 60.3 61.6 62.8 64.1
65 61.0 61.0 61.0 61.0 61.0 61.0 62.2 63.4 64.7 66.0 67.4
55 61.7 61.7 61.7 61.7 61.7 61.7 63.0 64.2 65.5 66.9 68.2
45 62.4 62.4 62.4 62.4 62.4 62.4 63.7 64.9 66.3 67.6 69.0
35 62.5 62.5 62.5 62.5 62.5 62.5 63.8 65.0 66.4 67.7 69.1
25 62.5 62.5 62.5 62.5 62.5 62.5 63.8 65.0 66.4 67.7 69.1
15 52.1 52.1 52.1 52.1 52.1 52.1 53.2 54.2 55.3 56.5 57.6
5 15.0 15.0 15.0 15.0 15.0 15.0 15.3 15.6 15.9 16.2 16.6
105 115 125 135 145 155 165 175 185 195 205
Year

For years 165 and beyond, that chart does not match the current empirical data for humans, it is the chart of a completely different kind of population. So instead, let's let Dickinson, Hiscock and Agbayani try another approach, forcing a match to the empirical data. Then their chart might end up looking something like this:

AgeRaw Intelligence Scores by Age and Year
95 48.0 48.0 48.0 48.0 48.0 48.0 41.6 42.5 43.3 44.2 45.1
85 53.6 53.6 53.6 53.6 53.6 53.6 46.5 47.5 48.4 49.4 50.4
75 58.0 58.0 58.0 58.0 58.0 58.0 51.4 52.5 53.5 54.6 55.7
65 61.0 61.0 61.0 61.0 61.0 61.0 55.1 56.2 57.3 58.5 59.7
55 61.7 61.7 61.7 61.7 61.7 61.7 56.9 58.1 59.2 60.4 61.7
45 62.4 62.4 62.4 62.4 62.4 62.4 58.8 59.9 61.2 62.4 63.7
35 62.5 62.5 62.5 62.5 62.5 62.5 60.0 61.2 62.4 63.7 65.0
25 62.5 62.5 62.5 62.5 62.5 62.5 61.2 62.4 63.7 65.0 66.3
15 52.1 52.1 52.1 52.1 52.1 52.1 52.0 53.1 54.2 55.2 56.4
5 15.0 15.0 15.0 15.0 15.0 15.0 15.3 15.6 15.9 16.2 16.6
105 115 125 135 145 155 165 175 185 195 205
Year

That would be better if it weren't for the jarring discontinuity between the years 155 and 165. Why would the introduction of a Flynn effect cause such an immediate and ragged discontinuity across age groups and time? The answer of course is that it wouldn't.

There is only one logically correct interpretation:
  • Cross-sectional studies (reading up the chart) do show a true age-related difference, most likely due to the biological impacts of aging.
  • Cross-time studies (reading straight across the chart) show a Flynn effect, applicable with roughly equal magnitude to every age group.
  • Longitudinal studies (reading diagonally across the chart) show the combination of age-based differences and the Flynn effect.
As a consequence, the title of Dickinson & Hickson (2010) suggests a gross misinterpretation.

References

Agbayani, K. A. (2011). Patterns of age-related IQ changes from the WAIS to WAIS-III after adjusting for the Flynn effect. Retrieved online from http://repositories.tdl.org/uh-ir/handle/10657/236.

Dickinson, M. D. & Hiscock, M. (2010). Age-related IQ decline is reduced markedly after adjustment for the Flynn effect. Journal of Clinical and Experimental Neuropsychology, 32(8), 865-870.

Sunday, August 12, 2012

Intelligence as Field

[Note: if the data charts below are difficult to read, try reading the essay at this alternative location.]


Let me begin with a chart of data:

AgeRaw Intelligence Scores by Age and Year
95 40.8 41.6 42.5 43.3 44.2 45.1 46.0 46.9 47.9 48.8 49.8
85 45.6 46.5 47.5 48.4 49.4 50.4 51.4 52.4 53.5 54.6 55.7
75 50.4 51.4 52.5 53.5 54.6 55.7 56.8 58.0 59.1 60.3 61.5
65 54.0 55.1 56.2 57.3 58.5 59.7 60.9 62.1 63.4 64.6 65.9
55 55.8 56.9 58.1 59.2 60.4 61.7 62.9 64.2 65.5 66.8 68.1
45 57.6 58.8 59.9 61.2 62.4 63.7 64.9 66.2 67.6 68.9 70.3
35 58.8 60.0 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8
25 60.0 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8 73.3
15 51.0 52.0 53.1 54.2 55.2 56.4 57.5 58.7 59.8 61.0 62.3
5 15.0 15.3 15.6 15.9 16.2 16.6 16.9 17.3 17.6 18.0 18.3
105 115 125 135 145 155 165 175 185 195 205
Year

This chart represents the results from an idealized experiment. Every ten years, beginning in year 105 and running to year 205, an intelligence test is administered to sample members from a population. They are recruited to match the *5 age points—that is, the test is administered each time to 5 year-olds, 15 year-olds, 25 year-olds, and so on up to 95 years of age. This allows us to follow each generation born at the decade boundaries: for instance, those born in the year 100 would take the exam as 5 year olds in the year 105, as 15 year olds in the year 115, and so on. Successive generations can be followed accordingly.

The test given is always the same—it does not change from decade to decade or from age group to age group. Furthermore, the test is administered in such a way as to make each trial independent of all previous outcomes. In practical terms, this would mean that each member of the population could be given the exam at most once during their lifetime, and there would be no such thing as cheat sheets, practice exams, or other extraneous factors that might influence scores across time. Admittedly, this requirement would be difficult to enact in reality, which is one of the reasons why we must resort to an idealized experiment.

The exam is considered to be a good measure of general intelligence abilities—in modern parlance, it has a high g loading. The values shown in the chart reflect raw scores on the exam and thus are directly comparable. We might think of the exam as consisting of one hundred questions, ranging from simple to difficult, with one point given for each correct answer and with an individual score above 95 considered to be nearly impossible, thus reducing or eliminating any ceiling effects.

The scores shown are the mean for the population, but it can also be assumed that the temporal patterns evident in these scores remain invariant across sub-populations. For instance, a given ethnic sub-population might have mean scores that are higher or lower than those shown in the above chart, but the pattern of scores from decade to decade and from age group to age group will remain quite similar. The same can be said for a sub-population of the highly intelligent or a sub-population of the less intelligent (or a sub-population of the middling intelligent, for that matter). The temporal patterns evident in the mean scores are indicative of similar temporal patterns that appear in nearly every significant sub-population that comes under study.

Although this experiment and its results have been idealized, they are intended to reflect current reality. In particular they reflect the two most experimentally agreed-upon temporal patterns witnessed in human raw intelligence scores. First there is the age-based pattern of intelligence performance, which can be seen by reading up the chart for any given year. At each time period, raw intelligence ability across age groups is seen as low but quickly increasing during the formative childhood years, peaking and plateauing during early adulthood, and finally diminishing gradually with advancing age. In the real world, the degree of these age-related changes in intelligence ability might differ from experiment to experiment or from exam to exam, but the pattern of intelligence growth, peak and gradual decline tends to show up in nearly every experiment, across nearly every intelligence exam, and for nearly every sub-population that comes under study.

The second experimentally agreed-upon temporal pattern in human raw intelligence scores can be seen by reading across the chart. At each age level, raw intelligence scores gradually and consistently increase with each passing decade. This is a demonstration of the well known Flynn effect, and once again, in the real world, the degree of these time-related changes might differ from experiment to experiment or from exam to exam, but the pattern of widespread intelligence growth tends to show up in nearly every experiment, for nearly every intelligence exam, and across nearly every sub-population that comes under study.

It would be helpful if there were a real-world gathering of intelligence data that might be compared to our idealized chart—and maybe one day there will be—but as of today there are difficulties in the path of this realization. For one, intelligence exams are changed from time to time and different levels of exam are often given to different age groups, making direct comparisons problematic. Furthermore, widespread awareness of intelligence tests has become established within the culture, so that independence of test-taking results is not as assured as it once was. Finally, at this point in human history, collecting a hundred years worth of data on any particular intelligence exam, especially for all age groups, would be unrealistic. Perhaps the Raven's test might be the most amenable to the type of data gathering being imagined here, but even in that instance, the data set would have to be described as incomplete at best.

Nonetheless, there remains compelling reason to accept the idealized chart as being essentially accurate. The chart was after all constructed specifically to reflect two of the most widely held and experimentally backed assumptions regarding human intelligence scores, namely the age-based pattern of growth, peak and gradual decline, and also the Flynn effect. If the real-world counterpart to our idealized chart were to somehow be essentially different in its temporal patterns, then the phenomena themselves would have to be called into question. If you accept the age-based pattern of growth, peak and decline, and if you accept the reality of the Flynn effect, then you would also have to accept the essential accuracy of the idealized chart of data—it reflects (ideally) the impact those phenomena must have on raw intelligence scores.


One of the reasons I have taken the time both to explain and to justify this chart of data is that I want to make use of it to dispel a common myth surrounding the Flynn effect, a myth regarding generational comparisons of human intelligence ability. This myth is usually presented along the lines of saying that a younger generation, by virtue of the Flynn effect, is more intelligent than an older generation (or for those of a more pessimistic bent, by saying that the older generation is generally less intelligent than the newer generation). An echo of this sentiment can be heard in the following passage from James Flynn's book What is Intelligence?
“If huge IQ gains are intelligence gains, why are we not struck by the extraordinary subtlety of our children's conversation? Why do we not have to make allowances for the limitations of our parents? A difference of some 18 points in Full Scale IQ over two generations ought to be highly visible.”
Such statements, when taken at face value, suggest that the Flynn effect should be producing a discernible distinction in intelligence ability across near generations, with younger generations emerging in real time as the more intelligent. But a quick perusal of the above chart (not to mention a careful observation of everyday experience) reveals the idea to be complete nonsense. If, for instance, we compare raw intelligence abilities of 15 year-olds against 55 year-olds, the 55 year-olds always emerge as the more intelligent, and furthermore, the relative distinction never changes. At each decade of measurement, the intelligence advantage of 55 year-olds over 15 year-olds remains exactly the same, driven by each group's respective position along the age-based pattern of human intelligence, and this continues despite the fact that the Flynn effect remains in full evidence across the entire time period of measurement:

AgeRaw Intelligence Scores by Age and Year
95 40.8 41.6 42.5 43.3 44.2 45.1 46.0 46.9 47.9 48.8 49.8
85 45.6 46.5 47.5 48.4 49.4 50.4 51.4 52.4 53.5 54.6 55.7
75 50.4 51.4 52.5 53.5 54.6 55.7 56.8 58.0 59.1 60.3 61.5
65 54.0 55.1 56.2 57.3 58.5 59.7 60.9 62.1 63.4 64.6 65.9
55 55.8 56.9 58.1 59.2 60.4 61.7 62.9 64.2 65.5 66.8 68.1
45 57.6 58.8 59.9 61.2 62.4 63.7 64.9 66.2 67.6 68.9 70.3
35 58.8 60.0 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8
25 60.0 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8 73.3
15 51.0 52.0 53.1 54.2 55.2 56.4 57.5 58.7 59.8 61.0 62.3
5 15.0 15.3 15.6 15.9 16.2 16.6 16.9 17.3 17.6 18.0 18.3
105 115 125 135 145 155 165 175 185 195 205
Year

So where did the Flynn effect go? Why is it not more apparent in these generational comparisons? Well, of course the Flynn effect did not go anywhere, the problem is actually in the comparison itself. The myth is essentially caused by the mistaken notion that a Flynn effect distinction can be discerned by comparing up and down the chart, that is by making a generational comparison at a particular point in time—for instance, by comparing 15 year-olds to 55 year-olds at time 175 and expecting a Flynn effect distinction will somehow emerge. But in fact, comparisons up and down the chart will never reveal any such thing.

The only proper Flynn effect comparison that can be made is across the chart. For instance, all we can accurately say about 15 year-olds at time 175 is that they reveal more raw intelligence ability than did the 15 year-olds at say time 135. But it is just as important to note that the 55 year-olds at time 175 (who of course were the 15 year-olds at time 135) also reveal more raw intelligence ability than did the 55 year-olds at time 135, and this difference, along with the normal age-related distinctions, has a predictable and consistent impact. The 55 year-olds at time 175 (along with the 55 year-olds at any time period) are not generationally lacking in raw intelligence. Quite the opposite.

Let's take one of the 55 year-olds at time 175, the ones being inaccurately characterized as somehow less intelligent than the younger generations from that same time period, and let's call him person A. One of the drivers behind the myth that Person A must be less intelligent than those of the younger generations is the mistaken sense that Person A's intelligence is irrevocably tied to the year of his birth. It is as though the intelligence characteristics of Person A are being determined by reading up the chart from the year 120 (which would be formed roughly from the average of the years 115 and 125):

AgeRaw Intelligence Scores by Age and Year
95 40.8 41.6 42.0 42.5 43.3 44.2 45.1 46.0 46.9 47.9 48.8 49.8
85 45.6 46.5 47.0 47.5 48.4 49.4 50.4 51.4 52.4 53.5 54.6 55.7
75 50.4 51.4 51.9 52.5 53.5 54.6 55.7 56.8 58.0 59.1 60.3 61.5
65 54.0 55.1 55.6 56.2 57.3 58.5 59.7 60.9 62.1 63.4 64.6 65.9
55 55.8 56.9 57.5 58.1 59.2 60.4 61.7 62.9 64.2 65.5 66.8 68.1
45 57.6 58.8 59.3 59.9 61.2 62.4 63.7 64.9 66.2 67.6 68.9 70.3
35 58.8 60.0 60.6 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8
25 60.0 61.2 61.8 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8 73.3
15 51.0 52.0 52.5 53.1 54.2 55.2 56.4 57.5 58.7 59.8 61.0 62.3
5 15.0 15.3 15.4 15.6 15.9 16.2 16.6 16.9 17.3 17.6 18.0 18.3
105 115 120 125 135 145 155 165 175 185 195 205
Year

There! You can practically see it for yourself! The 55 year-olds from around the year 120 were not very intelligent, and Person A, who after all was born in the year 120 and is a 55 year-old now—well, he must not be very intelligent either!

But of course that line of reasoning is completely wrongheaded. An individual's intelligence characteristics are not determined by reading up the chart from the year of his birth, they are determined instead by reading diagonally across the chart. Person A's intelligence characteristics (and by extension, the characteristics of his entire generation) are more accurately portrayed as follows:

AgeRaw Intelligence Scores by Age and Year
95 40.8 41.6 42.0 42.5 43.3 44.2 45.1 46.0 46.9 47.9 48.8 49.8
85 45.6 46.5 47.0 47.5 48.4 49.4 50.4 51.4 52.4 53.5 54.6 55.7
75 50.4 51.4 51.9 52.5 53.5 54.6 55.7 56.8 58.0 59.1 60.3 61.5
65 54.0 55.1 55.6 56.2 57.3 58.5 59.7 60.9 62.1 63.4 64.6 65.9
55 55.8 56.9 57.5 58.1 59.2 60.4 61.7 62.9 64.2 65.5 66.8 68.1
45 57.6 58.8 59.3 59.9 61.2 62.4 63.7 64.9 66.2 67.6 68.9 70.3
35 58.8 60.0 60.6 61.2 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8
25 60.0 61.2 61.8 62.4 63.7 65.0 66.3 67.6 69.0 70.4 71.8 73.3
15 51.0 52.0 52.5 53.1 54.2 55.2 56.4 57.5 58.7 59.8 61.0 62.3
5 15.0 15.3 15.4 15.6 15.9 16.2 16.6 16.9 17.3 17.6 18.0 18.3
105 115 120 125 135 145 155 165 175 185 195 205
Year

Reading Person A's actual intelligence history makes it more clear why Person A is not generationally disadvantaged at time 175. The Flynn effect, which has not gone anywhere, that very same Flynn effect that has been helping produce an increased level of intelligence in the 15 year-olds at time 175, it has also been advancing the intelligence characteristics of Person A across the entire period of his life. By the time Person A reaches the year 175, he has had 55 years worth of Flynn effect actively pushing his intelligence ability ever higher, and thus he has no difficulty holding his relative intelligence position against the younger generations. Across the entire time period of the chart, the Flynn effect works with equal magnitude upon every person in the population, so that by any particular point in time, the only difference that can emerge is the age-based difference.


This concept is so important that it deserves to be examined in greater detail. We can begin by comparing side-by-side Person A's intelligence characteristics under and not under the influence of a Flynn effect. Without a Flynn effect, Person A's intelligence history would turn out to be exactly the same as what we saw by reading up the chart from the year 120. With a Flynn effect, as we have seen, Person A's intelligence characteristics are determined by reading diagonally across the chart. The contrast is most revealing:


Age
Without
Flynn Effect
With
Flynn Effect

Difference
85 47.0 55.7 8.7
75 51.9 60.3 8.4
65 55.6 63.4 7.8
55 57.5 64.2 6.7
45 59.3 64.9 5.6
35 60.6 65.0 4.4
25 61.8 65.0 3.2
15 52.5 54.2 1.7
5 15.4 15.6 0.2

What should immediately leap out from this side-by-side comparison is the primary characteristic of the Flynn effect itself, a characteristic which unfortunately almost never receives the attention of cognitive scientists. The primary characteristic of the Flynn effect, so apparent here, is that it is continuous—and relentlessly so. This side-by-side comparison of Person A's intelligence history, under and not under the influence of a Flynn effect, makes it abundantly clear that the Flynn effect does not have just sudden impact at Person A's birth, nor does it create an overly strong influence during Person A's childhood education, nor does it produce momentous occasion in Person A's adulthood or at any other time during Person A's life. Instead, the Flynn effect works smoothly, consistently and relentlessly throughout Person A's entire lifetime, all the way from its beginning to its end, with no apparent discontinuities. If one were to describe the impact of the Flynn effect on Person A's life, one would have to refer not to the discrete influences on Person A's existence, but one could instead more profitably rely upon differential equations.

Person A is of course just one individual, but when we examine the entire chart of idealized data, we realize that what must be said about Person A's intelligence history must also be said about everyone else, for there is nothing in the chart to suggest otherwise. Taking the diagonal intellectual course of any person's life who falls under the range of the chart of data, comparing it side-by-side to that person's intelligence characteristics minus the Flynn effect, the same continuous, persistent phenomenon predictably emerges. No matter what person is under study, no matter what place that person may find himself, no matter what time period is being given consideration, the Flynn effect unveils itself as ubiquitous, continuous and relentless.

This is why I remain highly skeptical of nearly every proposed cause for the Flynn effect. Heterosis, advanced education, better nutrition, social multipliers, video games, so many others—all these proposed causations for the Flynn effect have been put forth as discrete influences on the members of the human population, intended to produce a distinct impact upon the human brain. But if such a discrete influence were actually true it would have to produce a more noticeable disturbance in the temporal pattern of raw intelligence scores, and yet as far as I can tell, in the real world, there has been not the slightest hint of evidence for any such disturbance.

Thus it is that I think better sense might be made of the Flynn effect by giving the problem not to a cognitive scientist, but instead by giving it to a physicist. For instance, let's try the following scenario: let's give our idealized chart of data to a physicist but hide from him the fact that the numbers represent intelligence data and that the subject of study is human behavior. Instead, we simply put the matter to him as follows:

“Here's a chart of data from an experiment we've been conducting. We are measuring a characteristic for some objects and these are the results. The vertical axis represents the age of the objects since the start of their existence, and the horizontal axis represents the time periods during which measurements were taken. The data captured is the raw magnitude of the characteristic under study. We've run the experiment several times and the results are consistent, but we've had a hard time figuring out what's going on. Can you make any sense out of this data?”

It would not be all that unthinkable to conceive of our physicist friend as inspecting the data for a few moments, then nodding his head slowly and finally saying yes, he believes he can shed some light on what is going on.

“I don't recognize what objects these are or what characteristic you're studying,” he might begin, “but it appears to me that your objects are under the influence of a dynamic field, and the characteristic you're studying is essentially defined by that field. The objects themselves do have variable responsiveness to the field—less so in their early career, more in middle life, tapering off near the top of the chart—that behavior is evident in each object and would be an inherent part of the object itself. But the characteristic you're studying is something quite different, it seems to be determined primarily by the traits of the field. Look at how the field becomes more uniformly intense over time, increasing the raw measurement on each object. I can't say what this characteristic might be, but I'm fairly sure it would be best described as a field—we see this sort of thing all the time in physics.”

Field theory was first introduced into the world of physics by James Maxwell in the 1860s, when he published a series of differential equations that described the essential characteristics of an electromagnetic field. Up until that time, physics had been dominated by purely mechanical descriptions of all known physical phenomena, a lingering outcome from Isaac Newton's successful ideas. Some phenomena, however, such as electricity, were beginning to feel the strain of the mechanistic point of view. As more and more experimental data began to accumulate, a multitude of theories attempting to describe charged objects as being constituted out of electrical fluids or out of electric pieces arranged in a variety of shapes and sizes began to suffer the dubious distinction of being inadequate for explaining the observable data or of being so convoluted as to defy common sense.

Maxwell dispelled the confusion by essentially changing the paradigm of the problem. Instead of attributing electricity to the charged objects themselves, Maxwell moved the essential traits of electricity (magnetism and light too) into the spatial-temporal field in which charged objects existed. Charged objects no longer contained electricity so much as they responded to it—some more so, and some less, each according to its physical traits. But the structural characteristics of electricity itself belonged instead to the field, a field whose dynamic features and explanatory power had been elegantly captured inside Maxwell's differential equations.

As with most ideas that run against the grain of the prevailing view, Maxwell's innovations were a bit slow at first to catch on, but by the end of the nineteenth century they had entirely revolutionized physics, ushering in an unprecedented era of growth and discovery. Einstein, for example, would make extensive use of field theory in the development of his general theory of relativity. Use of field theory has become commonplace in physics today, it is the lingua franca for a large variety of physical phenomena, and a modern physicist would have little difficulty recognizing a field's distinctive signature.

But that of course is physics. What about intelligence? Can intelligence also be profitably described through the means of field theory?

Well, our physicist friend certainly seems to think so. Note that although he remains entirely unaware that the objects under study are humans and that the characteristic being measured is intelligence, such knowledge is inessential to his judgment. The physicist recognizes a dynamic field at work through the patterns in the numbers and through a familiarity with the theme, and the fact that the objects actually are humans and that the characteristic being measured actually is intelligence does not alter the reality of the patterns, nor does it disqualify the application of the theme.

The world of intelligence research is currently dominated by a mechanistic/biological point of view. In particular, nearly every scientific theory regarding human intelligence either explicitly or implicitly assumes that human intelligence is directly attributable to the material properties of neurons, is directly derivable from the substance of the human brain. This view has become so dominant that it almost seems superfluous to state it. But unfortunately, dominance is not the same thing as completion. The work that is still to be done by brain-based proponents of human intelligence remains notably conspicuous by its glaring absence. What would really compel us to accept a brain-based notion of human intelligence is a convincing description of the form the brain's material substance must take in order to accurately and informatively explain the known intelligence data, including the data contained in the chart at the beginning of this essay. Yet no such description has even been placed on the table. In its absence, what has unfolded is a crowded cacophony of competing ideas, replete with a host of unstated presumptions and a mass of garbled-up data (see all of neuroscience, for instance). And nowhere is this chaotic scene on more obvious display than in the many-fangled attempts to explain the Flynn effect. The competing biological and socio-environmental hypotheses (all with an underlying assumption of ultimate impact on the human brain) have grown so broad-ranging and to such great multitude that by logical necessity alone, the grand majority must be suffering from the dubious distinction of being unable to account for the observable data or of being so convoluted as to defy common sense (see for instance the Dickens-Flynn model and Woodley's theory of fast and slow life). The intelligence research community has become utterly convinced that intelligence must be a brain-based phenomenon, and yet it seems utterly uncertain about everything else.

It is my contention that the confusion can be dispelled by changing the paradigm of the problem. Instead of ascribing intelligence to the material substances found inside the human head, it would be far more enlightening—and far more fitting to the data—to ascribe intelligence to the structural traits of a surrounding field, a field which would encompass the entire spatial-temporal range of human existence. Humans would be seen not so much as carrying intelligence as they would be seen as responding to it—some more so and some less, each according to his biological capacity (for instance, we see one such biologically driven variation in responsiveness in the table above, in the age-related patterns of human intelligence, and another might be the racial-ethnic differences sometimes evidenced, and of course at a deep enough level, each individual produces a unique responsiveness to the intelligence field). But responsiveness to a characteristic is not the same thing as the characteristic itself. Intelligence, properly speaking, is not to be defined as a material property of human biology, it is instead to be defined by the structural traits of an intelligence field.

From our idealized chart of data, it is an easy matter to describe the dynamic temporal traits of the chart's intelligence field: that field has the characteristic defined roughly by the differential equation

dy / dt = .002 y,

where y is the raw score and t is in years. This is the simple mathematical form of the Flynn effect. In the real world of course, we might anticipate a little more messiness, as certain types of fluctuations and perturbations are likely to occur, and yet based on what we do know about the Flynn effect, we can still state with some confidence that its apparently linear influence will almost certainly dominate any real-world equation.

This way of describing intelligence, by reference to the structural traits of a dynamic field, must no doubt seem foreign to the reader, but it has an obvious advantage. First of all, the technique directly addresses the data, something that a neuron-based description will almost certainly never do, and furthermore the technique addresses that data with conciseness, simplicity and elegance—the expected consequence of any successful application of field theory.


I know that by placing the structural characteristics of intelligence firmly outside the human brain I am running full tilt against the prevailing view. But my challenge to the many who now hold to a brain-based, mechanistic, biological model of human intelligence is that they must explain the chart of data with which this essay began. In particular, how does any brain-centered, discrete influence not leave a massive ripple in that enormous pond? Just look at the pattern of that data! So continuous, so persistent, so ubiquitous—like a slowly building wave, like a relentlessly rising tide. It should give one pause. But for those who must defend the brain-based models of human intelligence, my prediction is that confusion and convolution will continue to dog their steps. Me, I much prefer elegance. I prefer intelligence as field.


References

Dickens, W. T. & Flynn, J. R. (2001). Heritability estimates versus large environmental effects: The IQ paradox resolved. Psychological Review, 108: 346–369.
Flynn, J. R. (2007). What is intelligence? Beyond the Flynn effect. New York: Cambridge University Press.
Woodley, M. A. (2012). A life history model of the Lynn-Flynn effect. Personality and Individual Differences, 53: 2. 152–156.

Saturday, August 11, 2012

Repetition

As the Pharisees and scribes were to Jesus, as the Catholic Church was to Galileo, so is modern science to the genius of today.

Friday, August 10, 2012

Definition of a Modern Scientist

One who buries his logic errors under a mound of data.

Thursday, August 9, 2012

Courage in Action

A modern scientist thinks he's being revolutionary when he adds an additional decimal point to the data's mean.

Wednesday, August 8, 2012

The Serpent Swallowing His Tail

It seems that the most productive job in neuroscience right now is to statistically demonstrate how neuroscience has gone statistically wrong.