Hello There, Guest!  

Canadian IQ on WAIS IV

#21
The author has sent me some more information regarding the study. The problem is that he received some material from Pearson on the condition that he does not share it publicly. This sort of conflicts with the mandatory data sharing policy. In this case, however, the only relevant data are in the table found in the paper as well as a few numbers also mentioned in the paper, so there is no conflict in this case. But future cases may cause problems.

In reading the legally unshareable source material (other reviewers may also request it, presumably), I became more aware of a problem.

From the paper:
Quote:The American and Canadian standardisation samples were then matched in terms of education, ethnicity and sex to give two matched samples of 488 (aged 16 to 69) and 101 (aged 70 - 90) which were 92% white and 8% 'Asian.' This reduced the Canadian advantage to 2.1 points. This implies that of the 4.5 IQ points advantage of Canada, 2.4 IQ points are attributable to racial differences.

This is not right. Matching groups on education attenuates differences in g (because g causes educational differences). The decrease in the difference cannot be wholly attributed to racial composition based on these data. Unfortunately, the manual does not actually provide the data, so one cannot make a proper control.

Furthermore, there are two pairs of matched samples. A younger one with N's = 488 and an older one with N's = 101. The text is unclear as to which sample the 2.13 FSIQ difference is from. One cannot even check their calculation of statistical significance because the summary data isn't presented. No means, no SD's. Assuming an SD of 15 in both samples, means of 100 and 102.13, and doing unpaired t-test gives P's of .3 and .02 for the two samples

The journal name is misspelled in the acknowledgements section.

Another problem is that we are using FSIQ not g scores. FSIQ is a proxy for g scores and of no inherent theoretical interest. Without the data, we cannot easily calculate whether the difference is g-loaded (Spearman's law). The subtest means are given however, for the unmatched sample, and so if one can find the g-loadings of the subtests somewhere else, one can do a MCV analysis.

To test the racial composition idea, we want matched samples on age+sex+race but not education. To actually calculate whether the differences are statistically reliable, we need the FSIQ (g scores better!) SD's and means.

Can the author contact Pearson and request more data about the two matched subsamples?
 Reply
#22
(2014-Aug-16, 05:00:12)Emil Wrote: The author has sent me some more information regarding the study. The problem is that he received some material from Pearson on the condition that he does share it publicly. This sort of conflicts with the mandatory data sharing policy. In this case, however, the only relevant data are in the table found in the paper as well as a few numbers also mentioned in the paper, so there is no conflict in this case. But future cases may cause problems.

In reading the legally unshareable source material (other reviewers may also request it, presumably), I became more aware of a problem.

From the paper:
Quote:The American and Canadian standardisation samples were then matched in terms of education, ethnicity and sex to give two matched samples of 488 (aged 16 to 69) and 101 (aged 70 - 90) which were 92% white and 8% 'Asian.' This reduced the Canadian advantage to 2.1 points. This implies that of the 4.5 IQ points advantage of Canada, 2.4 IQ points are attributable to racial differences.

This is not right. Matching groups on education attenuates differences in g (because g causes educational differences). The decrease in the difference cannot be wholly attributed to racial composition based on these data. Unfortunately, the manual does not actually provide the data, so one cannot make a proper control.

Furthermore, there are two pairs of matched samples. A younger one with N's = 488 and an older one with N's = 101. The text is unclear as to which sample the 2.13 FSIQ difference is from. One cannot even check their calculation of statistical significance because the summary data isn't presented. No means, no SD's. Assuming an SD of 15 in both samples, means of 100 and 102.13, and doing unpaired t-test gives P's of .3 and .02 for the two samples

The journal name is misspelled in the acknowledgements section.

Another problem is that we are using FSIQ not g scores. FSIQ is a proxy for g scores and of no inherent theoretical interest. Without the data, we cannot easily calculate whether the difference is g-loaded (Spearman's law). The subtest means are given however, for the unmatched sample, and so if one can find the g-loadings of the subtests somewhere else, one can do a MCV analysis.

To test the racial composition idea, we want matched samples on age+sex+race but not education. To actually calculate whether the differences are statistically reliable, we need the FSIQ (g scores better!) SD's and means.

Can the author contact Pearson and request more data about the two matched subsamples?


I think it's very likely that the 2.4 points difference is due to the different racial compositions. The percentages of Blacks in Canada is only 2.9% vs about 13% in the US. Latin Americans in Canada are only 1.2% vs about 15% in the US. On the other hand, Asians in Canada constitute a far higher percentage of the population than in the US.
Finally, the educational systems in the US and in Canada are very similar so I do not think that matching samples on education is gonna skew the results.
 Reply
#23
Quote: I think it's very likely that the 2.4 points difference is due to the different racial compositions. The percentages of Blacks in Canada is only 2.9% vs about 13% in the US. Latin Americans in Canada are only 1.2% vs about 15% in the US. On the other hand, Asians in Canada constitute a far higher percentage of the population than in the US.

Sure.

Quote: Finally, the educational systems in the US and in Canada are very similar so I do not think that matching samples on education is gonna skew the results.

Matching can skew results even if systems are the same. Matching US blacks and whites on SES reduces CA d to ~.67. But this doesn't mean they are equally smart, that is the sociologist's fallacy. One may make the same fallacy with these data, altho of course much smaller in effect size.

I would be satisfied about this issue if the author simply wrote that this reduction is expected based on racial composition data and their known IQ's, but that matching on educational attainment can bias results.

Better would of course be if Pearson would give us numbers matched on age, sex, race but not educational attainment.
 Reply
#24
(2014-Aug-16, 13:26:55)Emil Wrote:
Quote: I think it's very likely that the 2.4 points difference is due to the different racial compositions. The percentages of Blacks in Canada is only 2.9% vs about 13% in the US. Latin Americans in Canada are only 1.2% vs about 15% in the US. On the other hand, Asians in Canada constitute a far higher percentage of the population than in the US.

Sure.

Quote: Finally, the educational systems in the US and in Canada are very similar so I do not think that matching samples on education is gonna skew the results.

Matching can skew results even if systems are the same. Matching US blacks and whites on SES brings reduces d to ~.67. But this doesn't mean they are equally smart, that is the sociologist's fallacy. One may make the same fallacy with these data, altho of course much smaller in effect size.

I would be satisfied about this issue if the author simply wrote that this reduction is expected based on racial composition data and their known IQ's, but that matching on educational attainment can bias results.

Better would of course be if Pearson would give us numbers matched on age, sex, race but not educational attainment.


All US or Canadian patriots please forgive me but Canada and the US are pretty much identical with regards to culture or educational systems. Blacks and whites live in very different communities with different access to educational opportunities so of course matching them on education is gonna skew the results. But US Whites and White Canadians are pretty much the same culture, with a few differences here and there (a Canadian friend of mine once told me that the US is Coca Cola and Canada is Pepsi) so matching them on education is gonna have very small effect on the results.
 Reply
#25
As I wrote before, that may be true but it is irrelevant.
 Reply
#26
Can you articulate your argument and explain why it is irrelevant? And explain logically the sociologist's fallacy?
 Reply
#27
For the sociologist's fallacy, see:
Jensen, Arthur. 1973. Educability and Group Differences. Chapter 11 "Equating for socioeconomic variables"
Also discussed in Jensen, A. 1998. The g Factor: The Science of Mental Ability. p. 449ff.

The goal is to find the relative g difference between two populations. The usual sociological practice is to equate them for socioeconomic variables as was done by Pearson with the data presented in this paper. This usually reduces the difference between groups (also in this case). Then it is interpreted as showing that these environmental factors are causally responsible for the difference. This is not right. It may be that g causes the environmental difference in SES, and so controlled for/equating SES means controlling/equating for g indirectly and partly.

In this study, Pearson has matched two subsamples for age, sex, race (white) and educational attainment. This means that they are possibly indirectly controlling for g.

The matter is made worse because the persons are from two different countries which certainly have some differences in what the standards are for awarding people degrees (cf. academic inflation).
 Reply
#28
(2014-Aug-16, 16:39:52)Emil Wrote: For the sociologist's fallacy, see:
Jensen, Arthur. 1973. Educability and Group Differences. Chapter 11 "Equating for socioeconomic variables"
Also discussed in Jensen, A. 1998. The g Factor: The Science of Mental Ability. p. 449ff.

The goal is to find the relative g difference between two populations. The usual sociological practice is to equate them for socioeconomic variables as was done by Pearson with the data presented in this paper. This usually reduces the difference between groups (also in this case). Then it is interpreted as showing that these environmental factors are causally responsible for the difference. This is not right. It may be that g causes the environmental difference in SES, and so controlled for/equating SES means controlling/equating for g indirectly and partly.

In this study, Pearson has matched two subsamples for age, sex, race (white) and educational attainment. This means that they are possibly indirectly controlling for g.

The matter is made worse because the persons are from two different countries which certainly have some differences in what the standards are for awarding people degrees (cf. academic inflation).


Ok I see your point. To geneticists, this is known as gene-environment correlation.
Edit 1: I had interpreted differently what they meant by matching for educational level. I had assumed they had compared each educational level in turn (e.g. Americans with a hs degree vs Canadians with a hs degree, etc.). Instead what they are doing is different: e.g. if a country has more % people with university degrees than the other, they'll match the numbers so that both samples have the same % people with a university degree.
 Reply
#29
It was made quite clear to me that being sent that information was a kind of favour. They're not prepared to give out information that they haven't published. Accordingly, I have made the small changes which Emil said he would be satisfied with. Indeed, they were there in the first place but removed by my co-author in the edit.


Attached Files
.doc   Canada IQ WAIS 1V.doc (Size: 89.5 KB / Downloads: 969)
 Reply
#30
(2014-Aug-18, 12:33:12)Barleymow Wrote: It was made quite clear to me that being sent that information was a kind of favour. They're not prepared to give out information that they haven't published. Accordingly, I have made the small changes which Emil said he would be satisfied with. Indeed, they were there in the first place but removed by my co-author in the edit.


Can you also state that the higher Canadian vs US IQ is probably due to more blacks living in the US than in Canada?
 Reply
 
Forum Jump:

Users browsing this thread: 1 Guest(s)