As a follow up to my previous post, Intelligence as Field, I would like to talk about two papers that essentially cover much of the same ground: Dickinson & Hiscock (2010) and Agbayani (2011). Unfortunately, I can't find a non-paywalled version of Dickinson & Hiscock (2010), and therefore I can't link to its full text, nor have I been able to read more than its title and abstract. [This is where I would normally begin a long rant about the ridiculously closed nature of science these days, but that's a worn-out subject, so let's move on.] To the rescue, Agbayani (2011) is available online and it provides all the essential information regarding the results and methods of Dickinson & Hiscock (2010). Agbayani is apparently a student of Hiscock, and Agbayani (2011) is a thesis paper that both outlines the approach of Dickinson & Hiscock (2010) and extends the range of its data and analysis.
The approach these authors take can be outlined as follows:
- They use data from WAIS, WAIS-R, and WAIS-III to compare intelligence scores across age groups and across time.
- They use a reverse norming methodology to place all scores on an equal footing, so that they can be directly compared as though they were raw scores from the same exam.
- They adjust these raw scores for the Flynn effect, adding an empirically reasonable amount to the older age group scores to account for the generational difference between the older age groups and the younger age groups.
- These adjusted raw scores are labeled as true aging effect scores and are shown to be similar to the pattern of scores that show up for individuals under longitudinal studies.
| Age | Raw Intelligence Scores by Age and Year | ||||||||||||
| 95 | 40.8 | 41.6 | 42.0 | 42.5 | 43.3 | 44.2 | 45.1 | 46.0 | 46.9 | 47.9 | 48.8 | 49.8 | |
| 85 | 45.6 | 46.5 | 47.0 | 47.5 | 48.4 | 49.4 | 50.4 | 51.4 | 52.4 | 53.5 | 54.6 | 55.7 | |
| 75 | 50.4 | 51.4 | 51.9 | 52.5 | 53.5 | 54.6 | 55.7 | 56.8 | 58.0 | 59.1 | 60.3 | 61.5 | |
| 65 | 54.0 | 55.1 | 55.6 | 56.2 | 57.3 | 58.5 | 59.7 | 60.9 | 62.1 | 63.4 | 64.6 | 65.9 | |
| 55 | 55.8 | 56.9 | 57.5 | 58.1 | 59.2 | 60.4 | 61.7 | 62.9 | 64.2 | 65.5 | 66.8 | 68.1 | |
| 45 | 57.6 | 58.8 | 59.3 | 59.9 | 61.2 | 62.4 | 63.7 | 64.9 | 66.2 | 67.6 | 68.9 | 70.3 | |
| 35 | 58.8 | 60.0 | 60.6 | 61.2 | 62.4 | 63.7 | 65.0 | 66.3 | 67.6 | 69.0 | 70.4 | 71.8 | |
| 25 | 60.0 | 61.2 | 61.8 | 62.4 | 63.7 | 65.0 | 66.3 | 67.6 | 69.0 | 70.4 | 71.8 | 73.3 | |
| 15 | 51.0 | 52.0 | 52.5 | 53.1 | 54.2 | 55.2 | 56.4 | 57.5 | 58.7 | 59.8 | 61.0 | 62.3 | |
| 5 | 15.0 | 15.3 | 15.4 | 15.6 | 15.9 | 16.2 | 16.6 | 16.9 | 17.3 | 17.6 | 18.0 | 18.3 | |
| 105 | 115 | 120 | 125 | 135 | 145 | 155 | 165 | 175 | 185 | 195 | 205 | ||
| Year | |||||||||||||
Age |
Without Flynn Effect |
With Flynn Effect |
Difference |
|---|---|---|---|
| 85 | 47.0 | 55.7 | 8.7 |
| 75 | 51.9 | 60.3 | 8.4 |
| 65 | 55.6 | 63.4 | 7.8 |
| 55 | 57.5 | 64.2 | 6.7 |
| 45 | 59.3 | 64.9 | 5.6 |
| 35 | 60.6 | 65.0 | 4.4 |
| 25 | 61.8 | 65.0 | 3.2 |
| 15 | 52.5 | 54.2 | 1.7 |
| 5 | 15.4 | 15.6 | 0.2 |
If you were to apply the approach of Dickinson, Hiscock and Agbayani to the original chart of data, you would arrive at exactly the same comparative data set, only with a different set of labels:
Age |
Scores that Reflect AGD |
Scores that Reflect TAE |
FED |
|---|---|---|---|
| 85 | 47.0 | 55.7 | 8.7 |
| 75 | 51.9 | 60.3 | 8.4 |
| 65 | 55.6 | 63.4 | 7.8 |
| 55 | 57.5 | 64.2 | 6.7 |
| 45 | 59.3 | 64.9 | 5.6 |
| 35 | 60.6 | 65.0 | 4.4 |
| 25 | 61.8 | 65.0 | 3.2 |
| 15 | 52.5 | 54.2 | 1.7 |
| 5 | 15.4 | 15.6 | 0.2 |
In the terminology of Dickinson, Hiscock and Agbayani, AGD stands for age group difference, reflecting the type of pattern that emerges from cross-sectional studies (that is, from reading up the chart at any given time). TAE stands for true aging effect and reflects the type of pattern that emerges from longitudinal studies (that is, from reading diagonally across the chart for any population cohort). FED is the Flynn effect difference.
In a certain sense, I'm quite pleased that Dickinson & Hiscock (2010) and Agbayani (2011) exist. They are the nearest thing I can find to a real-world analysis similar to what I outlined in Intelligence as Field, and of course it is gratifying to know that the real-world outcome turns out to be essentially the same as my idealized approach.
On the other hand, there is a major problem. Although it appears to me that Dickinson, Hiscock and Agbayani have done a creditable job in the gathering of their data, they have also managed to utterly mangle its interpretation.
In reading the conclusions these authors draw from their analysis (indeed, in reading through their entire approach to the problem), one quickly realizes that they are (mistakenly) saying the following:
- The age-based differences that show up in cross-sectional studies (reading up the chart at any given time) are distorted because of the Flynn effect.
- By adjusting for the influence of the Flynn effect, one arrives at a longitudinal set of values (reading diagonally across the chart) that represents the true age-based difference in individuals—that is, the age-based difference that would show up in the absence of a Flynn effect.
The only way to make logical sense of the data is to state it the other way. The cross-sectional studies (reading up the chart at any given time) are the true age-based differences—that is, the age-based differences that would show up in the absence of a Flynn effect. The longitudinal values (reading diagonally across the chart) represent the combination of age-based differences and the Flynn effect. The reason that the longitudinal raw scores remain fairly constant across adulthood is that the two competing influences (age-based decline and Flynn effect increase) are in rough equilibrium.
As far as I can tell, there is no reasonable way to make sense out of the Dickinson/Hiscock/Agbayani interpretation. To see this, consider what must happen to the data under and not under the influence of a Flynn effect. Let me use a version of my idealized chart of data, and let's assume there is no Flynn effect for the first fifty years. Under these conditions and under my interpretation, the chart of data would look something like this:
| Age | Raw Intelligence Scores by Age and Year | |||||||||||
| 95 | 40.8 | 40.8 | 40.8 | 40.8 | 40.8 | 40.8 | ||||||
| 85 | 45.6 | 45.6 | 45.6 | 45.6 | 45.6 | 45.6 | ||||||
| 75 | 50.4 | 50.4 | 50.4 | 50.4 | 50.4 | 50.4 | ||||||
| 65 | 54.0 | 54.0 | 54.0 | 54.0 | 54.0 | 54.0 | ||||||
| 55 | 55.8 | 55.8 | 55.8 | 55.8 | 55.8 | 55.8 | ||||||
| 45 | 57.6 | 57.6 | 57.6 | 57.6 | 57.6 | 57.6 | ||||||
| 35 | 58.8 | 58.8 | 58.8 | 58.8 | 58.8 | 58.8 | ||||||
| 25 | 60.0 | 60.0 | 60.0 | 60.0 | 60.0 | 60.0 | ||||||
| 15 | 51.0 | 51.0 | 51.0 | 51.0 | 51.0 | 51.0 | ||||||
| 5 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | ||||||
| 105 | 115 | 125 | 135 | 145 | 155 | 165 | 175 | 185 | 195 | 205 | ||
| Year | ||||||||||||
But by the account of Dickinson, Hiscock, and Agbayani, the chart would need to look much different. Since they are saying that the true aging effect scores reflect age-based differences sans a Flynn effect, then their chart of data (under no Flynn effect) would need to look more like this:
| Age | Raw Intelligence Scores by Age and Year | |||||||||||
| 95 | 48.0 | 48.0 | 48.0 | 48.0 | 48.0 | 48.0 | ||||||
| 85 | 53.6 | 53.6 | 53.6 | 53.6 | 53.6 | 53.6 | ||||||
| 75 | 58.0 | 58.0 | 58.0 | 58.0 | 58.0 | 58.0 | ||||||
| 65 | 61.0 | 61.0 | 61.0 | 61.0 | 61.0 | 61.0 | ||||||
| 55 | 61.7 | 61.7 | 61.7 | 61.7 | 61.7 | 61.7 | ||||||
| 45 | 62.4 | 62.4 | 62.4 | 62.4 | 62.4 | 62.4 | ||||||
| 35 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | ||||||
| 25 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | ||||||
| 15 | 52.1 | 52.1 | 52.1 | 52.1 | 52.1 | 52.1 | ||||||
| 5 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | ||||||
| 105 | 115 | 125 | 135 | 145 | 155 | 165 | 175 | 185 | 195 | 205 | ||
| Year | ||||||||||||
Now consider what would happen if a Flynn effect kicked in beginning at time 155.
In my chart and under my interpretation, the progression is quite natural. Raw scores begin to go up by say 2% every ten years for all age groups, and what results is a chart of data for years 165 to 205 that has all the same patterns we currently see in the empirical data for humans. Note that the pattern of age-based differences remains invariant under the changing Flynn effect assumptions:
| Age | Raw Intelligence Scores by Age and Year | |||||||||||
| 95 | 40.8 | 40.8 | 40.8 | 40.8 | 40.8 | 40.8 | 41.6 | 42.5 | 43.3 | 44.2 | 45.1 | |
| 85 | 45.6 | 45.6 | 45.6 | 45.6 | 45.6 | 45.6 | 46.5 | 47.5 | 48.4 | 49.4 | 50.4 | |
| 75 | 50.4 | 50.4 | 50.4 | 50.4 | 50.4 | 50.4 | 51.4 | 52.5 | 53.5 | 54.6 | 55.7 | |
| 65 | 54.0 | 54.0 | 54.0 | 54.0 | 54.0 | 54.0 | 55.1 | 56.2 | 57.3 | 58.5 | 59.7 | |
| 55 | 55.8 | 55.8 | 55.8 | 55.8 | 55.8 | 55.8 | 56.9 | 58.1 | 59.2 | 60.4 | 61.7 | |
| 45 | 57.6 | 57.6 | 57.6 | 57.6 | 57.6 | 57.6 | 58.8 | 59.9 | 61.2 | 62.4 | 63.7 | |
| 35 | 58.8 | 58.8 | 58.8 | 58.8 | 58.8 | 58.8 | 60.0 | 61.2 | 62.4 | 63.7 | 65.0 | |
| 25 | 60.0 | 60.0 | 60.0 | 60.0 | 60.0 | 60.0 | 61.2 | 62.4 | 63.7 | 65.0 | 66.3 | |
| 15 | 51.0 | 51.0 | 51.0 | 51.0 | 51.0 | 51.0 | 52.0 | 53.1 | 54.2 | 55.2 | 56.4 | |
| 5 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.3 | 15.6 | 15.9 | 16.2 | 16.6 | |
| 105 | 115 | 125 | 135 | 145 | 155 | 165 | 175 | 185 | 195 | 205 | ||
| Year | ||||||||||||
But what are Dickinson, Hiscock and Agbayani going to do? How can they reasonably introduce a Flynn effect at time 155 and still remain true to the empirical data? For instance, they can't just begin to boost scores across all age groups, because then their chart would end up looking like this:
| Age | Raw Intelligence Scores by Age and Year | |||||||||||
| 95 | 48.0 | 48.0 | 48.0 | 48.0 | 48.0 | 48.0 | 49.0 | 50.0 | 51.0 | 52.0 | 53.0 | |
| 85 | 53.6 | 53.6 | 53.6 | 53.6 | 53.6 | 53.6 | 54.6 | 55.7 | 56.9 | 58.0 | 59.2 | |
| 75 | 58.0 | 58.0 | 58.0 | 58.0 | 58.0 | 58.0 | 59.2 | 60.3 | 61.6 | 62.8 | 64.1 | |
| 65 | 61.0 | 61.0 | 61.0 | 61.0 | 61.0 | 61.0 | 62.2 | 63.4 | 64.7 | 66.0 | 67.4 | |
| 55 | 61.7 | 61.7 | 61.7 | 61.7 | 61.7 | 61.7 | 63.0 | 64.2 | 65.5 | 66.9 | 68.2 | |
| 45 | 62.4 | 62.4 | 62.4 | 62.4 | 62.4 | 62.4 | 63.7 | 64.9 | 66.3 | 67.6 | 69.0 | |
| 35 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 63.8 | 65.0 | 66.4 | 67.7 | 69.1 | |
| 25 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 63.8 | 65.0 | 66.4 | 67.7 | 69.1 | |
| 15 | 52.1 | 52.1 | 52.1 | 52.1 | 52.1 | 52.1 | 53.2 | 54.2 | 55.3 | 56.5 | 57.6 | |
| 5 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.3 | 15.6 | 15.9 | 16.2 | 16.6 | |
| 105 | 115 | 125 | 135 | 145 | 155 | 165 | 175 | 185 | 195 | 205 | ||
| Year | ||||||||||||
For years 165 and beyond, that chart does not match the current empirical data for humans, it is the chart of a completely different kind of population. So instead, let's let Dickinson, Hiscock and Agbayani try another approach, forcing a match to the empirical data. Then their chart might end up looking something like this:
| Age | Raw Intelligence Scores by Age and Year | |||||||||||
| 95 | 48.0 | 48.0 | 48.0 | 48.0 | 48.0 | 48.0 | 41.6 | 42.5 | 43.3 | 44.2 | 45.1 | |
| 85 | 53.6 | 53.6 | 53.6 | 53.6 | 53.6 | 53.6 | 46.5 | 47.5 | 48.4 | 49.4 | 50.4 | |
| 75 | 58.0 | 58.0 | 58.0 | 58.0 | 58.0 | 58.0 | 51.4 | 52.5 | 53.5 | 54.6 | 55.7 | |
| 65 | 61.0 | 61.0 | 61.0 | 61.0 | 61.0 | 61.0 | 55.1 | 56.2 | 57.3 | 58.5 | 59.7 | |
| 55 | 61.7 | 61.7 | 61.7 | 61.7 | 61.7 | 61.7 | 56.9 | 58.1 | 59.2 | 60.4 | 61.7 | |
| 45 | 62.4 | 62.4 | 62.4 | 62.4 | 62.4 | 62.4 | 58.8 | 59.9 | 61.2 | 62.4 | 63.7 | |
| 35 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 60.0 | 61.2 | 62.4 | 63.7 | 65.0 | |
| 25 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 62.5 | 61.2 | 62.4 | 63.7 | 65.0 | 66.3 | |
| 15 | 52.1 | 52.1 | 52.1 | 52.1 | 52.1 | 52.1 | 52.0 | 53.1 | 54.2 | 55.2 | 56.4 | |
| 5 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.3 | 15.6 | 15.9 | 16.2 | 16.6 | |
| 105 | 115 | 125 | 135 | 145 | 155 | 165 | 175 | 185 | 195 | 205 | ||
| Year | ||||||||||||
That would be better if it weren't for the jarring discontinuity between the years 155 and 165. Why would the introduction of a Flynn effect cause such an immediate and ragged discontinuity across age groups and time? The answer of course is that it wouldn't.
There is only one logically correct interpretation:
- Cross-sectional studies (reading up the chart) do show a true age-related difference, most likely due to the biological impacts of aging.
- Cross-time studies (reading straight across the chart) show a Flynn effect, applicable with roughly equal magnitude to every age group.
- Longitudinal studies (reading diagonally across the chart) show the combination of age-based differences and the Flynn effect.
References
Agbayani, K. A. (2011). Patterns of age-related IQ changes from the WAIS to WAIS-III after adjusting for the Flynn effect. Retrieved online from http://repositories.tdl.org/uh-ir/handle/10657/236.Dickinson, M. D. & Hiscock, M. (2010). Age-related IQ decline is reduced markedly after adjustment for the Flynn effect. Journal of Clinical and Experimental Neuropsychology, 32(8), 865-870.
No comments:
Post a Comment