2014-Oct-09, 01:56:33

There are a few ways it can go wrong. E.g. if one uses only a specific type of subtest (e.g. 10 types of arithmetic tests), or there is not much variance between subtest factor-loadings (restriction of variance, common in normal IQ batteries because they selected only useful subtests on purpose), or just ordinary sampling error due to low number of subtests (also often the case, since N_subtest is often only 5-10). For these reasons, MCV is not a very robust metric (susceptible to artifacts). The term you propose seems to be an inclusive concerning errors in estimating the true correlation between factor loadings and the vector of interest.

Kranzler, J. H., and Jensen, A. R. (1991). Unitary g: Unquestioned postulate or empirical fact? Intelligence, 15, 437—448.

Jensen, A. R. (1993). Psychometric g and achievement. In B. R. Gifford (Ed.), Policy perspectives on educational testing. Norwell, MA: Kluwer Academic Publishers. Pp. 117-227.

Dragt, J. (2010). Causes of group differences studied with the method of correlated vectors: A psychometric meta-analysis of Spearman’s hypothesis.

Just the first three hits I got from googling "psychometric sampling error". Note, however, that the S factor is not a psychological construct, so calling it "psychometric" does not make sense. What subject-neutral term do you propose? It is sampling error involving the indicator variables of a latent variable/factor.

Quote:Certainly, like any other statistic, a g factor based on a limited number of mental tests and a limited number of subjects will contain error. There are three main sources of such error: (1) subject sampling error, because a sample does not perfectly represent the population; (2) psychometric sampling error, because a limited number of diverse mental tests does not perfectly represent the total population of mental tests, actual or conceivable; and (3) all factor scores in a common factor analysis, including g factor scores, by any method of derivation, are only estimates of the true factor scores, which remain unknown, in the same sense that obtained scores are estimates of true scores, with some determinable margin of probable error, in classical measurement theory. It has been deter- mined mathematically that the average minimum correlation between estimated factor scores and their corresponding hypothetical true factor scores rapidly increases as a function of the ratio of the number of tests to the number of first- order factors (Gorsuch, 1983, p. 259). With 11 tests and two first-order factors, as in this psychometric battery, the minimum correlation between estimated and true factor scores would be + .84, and the actual correlation could be well above this value.

Kranzler, J. H., and Jensen, A. R. (1991). Unitary g: Unquestioned postulate or empirical fact? Intelligence, 15, 437—448.

Quote:Just as we can think statistically in terms of the sampling error of a statistic, when we randomly select a limited group of subjects from a population, or of measurement error, when we obtain a limited number of measurements of a particular variable, so too we can think in terms of a psychometric sampling error. In making up any collection of cognitive tests, we do not have a perfectly representative sample of the entire population of cognitive tests or of all possible cognitive tests, and so any one limited sample of tests will not yield exactly the same g as another limited sample. The sample values of g are affected by subject sampling error, measurement error, and psychometric sampling error. But the fact that g is very substantially correlated across different test batteries means that the variable values of g can all be interpreted as estimates of some true (but unknown) g, in the same sense that, in classical test theory, an obtained score is viewed as an estimate of a true score.

Jensen, A. R. (1993). Psychometric g and achievement. In B. R. Gifford (Ed.), Policy perspectives on educational testing. Norwell, MA: Kluwer Academic Publishers. Pp. 117-227.

Quote:The deviation from perfect construct validity in g attenuates the values of r(g× d). In making up any collection of cognitive tests, we do not have a perfectly representative sample of the entire universe of all possible cognitive tests. Therefore any one limited sample of tests will not yield exactly the same gas another such sample. The sample values of g are affected by psychometric sampling error, but the fact thatgis very substantially correlated across different test batteries implies that the differing obtained values of g can all be interpreted as estimates of a “true” g. The values of r (g× d) are attenuated by psychometric sampling error in each of the batteries from which a gfactor has been extracted. We carried out a separate study to empirically estimate the values for this correction.

Dragt, J. (2010). Causes of group differences studied with the method of correlated vectors: A psychometric meta-analysis of Spearman’s hypothesis.

Just the first three hits I got from googling "psychometric sampling error". Note, however, that the S factor is not a psychological construct, so calling it "psychometric" does not make sense. What subject-neutral term do you propose? It is sampling error involving the indicator variables of a latent variable/factor.