The author has sent me some more information regarding the study. The problem is that he received some material from Pearson on the condition that he does not share it publicly. This sort of conflicts with the mandatory data sharing policy. In this case, however, the only relevant data are in the table found in the paper as well as a few numbers also mentioned in the paper, so there is no conflict in this case. But future cases may cause problems.

In reading the legally unshareable source material (other reviewers may also request it, presumably), I became more aware of a problem.

From the paper:

This is not right. Matching groups on education attenuates differences in g (because g causes educational differences). The decrease in the difference cannot be wholly attributed to racial composition based on these data. Unfortunately, the manual does not actually provide the data, so one cannot make a proper control.

Furthermore, there are two pairs of matched samples. A younger one with N's = 488 and an older one with N's = 101. The text is unclear as to which sample the 2.13 FSIQ difference is from. One cannot even check their calculation of statistical significance because the summary data isn't presented. No means, no SD's. Assuming an SD of 15 in both samples, means of 100 and 102.13, and doing unpaired t-test gives P's of .3 and .02 for the two samples

The journal name is misspelled in the acknowledgements section.

Another problem is that we are using FSIQ not g scores. FSIQ is a proxy for g scores and of no inherent theoretical interest. Without the data, we cannot easily calculate whether the difference is g-loaded (Spearman's law). The subtest means are given however, for the unmatched sample, and so if one can find the g-loadings of the subtests somewhere else, one can do a MCV analysis.

To test the racial composition idea, we want matched samples on age+sex+race but not education. To actually calculate whether the differences are statistically reliable, we need the FSIQ (g scores better!) SD's and means.

Can the author contact Pearson and request more data about the two matched subsamples?

In reading the legally unshareable source material (other reviewers may also request it, presumably), I became more aware of a problem.

From the paper:

Quote:The American and Canadian standardisation samples were then matched in terms of education, ethnicity and sex to give two matched samples of 488 (aged 16 to 69) and 101 (aged 70 - 90) which were 92% white and 8% 'Asian.' This reduced the Canadian advantage to 2.1 points. This implies that of the 4.5 IQ points advantage of Canada, 2.4 IQ points are attributable to racial differences.

This is not right. Matching groups on education attenuates differences in g (because g causes educational differences). The decrease in the difference cannot be wholly attributed to racial composition based on these data. Unfortunately, the manual does not actually provide the data, so one cannot make a proper control.

Furthermore, there are two pairs of matched samples. A younger one with N's = 488 and an older one with N's = 101. The text is unclear as to which sample the 2.13 FSIQ difference is from. One cannot even check their calculation of statistical significance because the summary data isn't presented. No means, no SD's. Assuming an SD of 15 in both samples, means of 100 and 102.13, and doing unpaired t-test gives P's of .3 and .02 for the two samples

The journal name is misspelled in the acknowledgements section.

Another problem is that we are using FSIQ not g scores. FSIQ is a proxy for g scores and of no inherent theoretical interest. Without the data, we cannot easily calculate whether the difference is g-loaded (Spearman's law). The subtest means are given however, for the unmatched sample, and so if one can find the g-loadings of the subtests somewhere else, one can do a MCV analysis.

To test the racial composition idea, we want matched samples on age+sex+race but not education. To actually calculate whether the differences are statistically reliable, we need the FSIQ (g scores better!) SD's and means.

Can the author contact Pearson and request more data about the two matched subsamples?