Hello There, Guest!  

[ODP] The personal Jensen coefficient does not predict grades beyond its association

#11
(2014-Oct-03, 14:16:31)Emil Wrote: New revision (#6) with the above results added in a new section. Nothing else changed.

https://osf.io/gb3cy/


"The personal Jensen coefficient correlates moderately with both g factor scores (.35) and the unit-mean (.23)."

Given SLDR, shouldn't the personal Jensen coefficient negatively correlate with g-scores? Or doesn't SLDR work this way? If it does, then your sample is, on this account, problematic - and you should note this.
 Reply
#12
I don't see why SLDR should predict that. The correlation between personal Jensen coef. and g scores and unit-mean simply shows that the smarter people tend to get their higher scores on the more g-loaded tests. The measure Piffer came up with (g score minus unit-mean) shows the same behavior.
 Reply
#13
g score is higher than full scale IQ score if the Jensen effect (cor with g-loadings) is positive, but g score is lower than full scale IQ if Jensen effect is negative. Now, what SLODR says is that the strength of the (sub)tests intercorrelation is lower at higher IQ levels. I don't remember it says that the correlation with g-loadings becomes negative. But perhaps it's possible that g-loadings' correlation is lower when IQ levels go up.
 Reply
#14
(2014-Oct-15, 19:52:54)Emil Wrote: I don't see why SLDR should predict that. The correlation between personal Jensen coef. and g scores and unit-mean simply shows that the smarter people tend to get their higher scores on the more g-loaded tests. The measure Piffer came up with (g score minus unit-mean) shows the same behavior.


It would depend on how one conceptualized the mechanism behind SLDR. I was imaging that the population level correlations were lower at higher IQs, because on the individual level g was less potent. If this was the case, I would expect an individual level anti-Jensen Effect at higher IQs. I thought Armstrong suggested this. But I agree that this need not be the mechanism.

I like this paper. It was well written, simple and to the point. An interesting idea was explored.

I approve.
 Reply
#15
1) "correlation of subtests' g-loading"

g-loadingS

2) "Dutch students"

Specify, "university students"

3) "Perhaps this is because it is a student dataset with an above average level of g. According to the ability differentiation hypothesis, the higher the level of g, the weaker the g factor."

If the university students are selected based on g, there's also range restriction which reduces g variance.

4) "A conceptually similar measure is the g minus unit-mean metric (g advantage). This value is positive when the person has his highest scores on the more g-loaded subtests, and lower than the opposite is the case."

Rephrase the "lower than the opposite is the case." Also, with an increasing number of tests, the correlation between equally weighted and g-weighted scores approaches 1 because only the g variance tends to cumulate into composite scores regardless of weights used. See p. 103 in Jensen's g factor book. Accordingly, the correlation between g scores and unweighted scores in your data is 0.99, and the g advantage has no validity independently of g scores.

5) "They do not seem to have any unique predictive power for GPA beyond their association with g. Multiple regression gave a similar result (results not shown)."

What's the point of using MR here? It's superfluous with the partial correlation.

6) "Verbal analogies has a p value of .04 (N=289, two-tailed)"

The other p-values were >0.05, right? You should mention that.

7) "I ran the partial correlations with GPA and g partialled out"

Rephrase. You ran correlations between GPA and subtests, with g scores partialled out.

8) As a general point, the g factor is a between-individuals variable whereas your personal Jensen coefficient is a within-individual variable. You cannot easily generalize from individual differences processes to within-person processes, so your entire analysis is a bit suspect. Peter Molenaar has written about this a lot.
 Reply
#16
Thank you for the criticism Dalliard. I will update the revision with the fixes. As for the last point, the personal Jensen coef. is dependent both on the between-individual data and the within-individual data. The first is needed to find the g-loadings, the second to calculate the effect. I think it was an interesting idea, but it didn't pan out. In the spirit of preventing publication bias in the scientific literature, I decided to write up the results for a paper instead of just thinking "hm, I guess that didn't work" or blogging about it.
 Reply
#17
Quote:1) "correlation of subtests' g-loading"

g-loadingS

Fixed.

Quote:2) "Dutch students"

Specify, "university students"

Fixed.

Quote:3) "Perhaps this is because it is a student dataset with an above average level of g. According to the ability differentiation hypothesis, the higher the level of g, the weaker the g factor."

If the university students are selected based on g, there's also range restriction which reduces g variance.

Added: "Alternatively, one may think of it as range restriction of g, so that it is relatively smaller compared to the other sources of variance in the cognitive data."

Quote: 4) "A conceptually similar measure is the g minus unit-mean metric (g advantage). This value is positive when the person has his highest scores on the more g-loaded subtests, and lower than the opposite is the case."

Rephrase the "lower than the opposite is the case." Also, with an increasing number of tests, the correlation between equally weighted and g-weighted scores approaches 1 because only the g variance tends to cumulate into composite scores regardless of weights used. See p. 103 in Jensen's g factor book. Accordingly, the correlation between g scores and unweighted scores in your data is 0.99, and the g advantage has no validity independently of g scores.

Changed to "This value is positive when the person has his highest scores on the more g-loaded subtests, and lower when the opposite is the case."

I agree regarding the comment. The g factor scores had no incremental validity above unit-weighted scores in accordance with statistical theory.

Quote:5) "They do not seem to have any unique predictive power for GPA beyond their association with g. Multiple regression gave a similar result (results not shown)."

What's the point of using MR here? It's superfluous with the partial correlation.

I have found that sometimes MR and partial correlations give markedly different results. For this reason I often test both to make sure it isn't some strange statistical fuck-up.

Quote: 6) "Verbal analogies has a p value of .04 (N=289, two-tailed)"

The other p-values were >0.05, right? You should mention that.

Yes. Changed to "All the partial correlations were weak. Three were in the wrong direction. Only verbal analogies has a p-value below .05 (.04) (N=289, two-tailed) but since I tested 7 subtests and there is no adjustment made for multiple comparisons, it may be a fluke."

Quote: 7) "I ran the partial correlations with GPA and g partialled out"

Rephrase. You ran correlations between GPA and subtests, with g scores partialled out.

Changed to: "Do the Jensen coefficient and g adv. explain unique parts of the variance of GPA? To test this, I ran the partial correlations between both measures and GPA controlling for g scores."

---

New revision can be found at OSF. https://osf.io/gb3cy/ Revision #7, dated 24th Oct.

Additionally, I reran all the analyses from my laptop. All analyses produced the same results as before (analytic reproducibility).
 Reply
#18
"A Jensen coefficient is the correlation between a subtests’ g-loadings and a vector of interest."

"I hypothesized that the personal Jensen coefficient from a subjects’ subtest scores will predict grade point average beyond g."

Rewrite: "Alternatively, one may think of it as range restriction of g, so that it is relatively smaller compared to the other sources of variance in the cognitive data" What is "it" referencing?

Was there no way to correct for subtest reliability? Could you mention that you were unable to do this despite this being standard methodology.
 Reply
#19
Thank you for reviewing, Chuck.

Quote: "A Jensen coefficient is the correlation between a subtests’ g-loadings and a vector of interest."

Presumably, you copied this from the abstract. However, the text is better as it is. It is nonsensical as you propose it to be.

Quote: "I hypothesized that the personal Jensen coefficient from a subjects’ subtest scores will predict grade point average beyond g."

Both additions are unnecessary. The first one makes it worse.

Quote:Rewrite: "Alternatively, one may think of it as range restriction of g, so that it is relatively smaller compared to the other sources of variance in the cognitive data" What is "it" referencing?

Changed it to "Alternatively, one may think of it as range restriction of g, so that the g variance is relatively smaller compared to the other sources of variance in the cognitive data."

Quote:Was there no way to correct for subtest reliability? Could you mention that you were unable to do this despite this being standard methodology.

Subtest reliability is not reported by Wicherts et al. As far as I can tell, these are uncommon tests, not commonly used tests so it will be harder to find reliability data. For th RAPM, one can find data, but correcting one without the others is bad for MCV.

-

I didn't upload the new version yet as there may be more mickey mouse edits necessary.
 Reply
#20
1) "finding the correlation between subtests' g-loading and each person's scores on the subtests" --> g-loadingS

2) Rephrase: "I ran the partial correlations with GPA and g partialled out."

3) I don't think corrections for measurement error make sense here because even if you had reliability data you could not correct the individual scores because the amount of error differs between individuals. Correcting the g-loadings might add as much error as it removes.

4) I think there should be more reflection on the intra- versus inter-individual differences problem, but I approve publication in any case.
 Reply
 
Forum Jump:

Users browsing this thread: 1 Guest(s)