Hello There, Guest!

# [ODP] The international general socioeconomic factor: Factor analyzing international

Davenport, E. C. (1990). Significance testing of congruence coefficients: A good idea?. Educational and psychological measurement, 50(2), 289-296.

... I am left with the impression it's very bad method. You should be careful with that. (The version of the paper I have can't allow copy paste, but check the pages 293-295.) The congruence coeff seems to constantly give you very high value even in situations where they should be (theoretically) small, or not high at all.

Can you upload that paper or send it to me? There are other sources that are more sanguine about the CC, e.g., https://media.psy.utexas.edu/sandbox/gro...ruence.pdf In any case, the problems with using Pearson's r in the analysis of factor loadings are even greater.

Quote:Regarding the question of negative loadings, i don't understand your discussion here, both of you. My opinion is that when you have small loadings (such as 0.20 or less) in the 1rst unrotated factor, regardless of the direction, you should remove it because it's a poor measure of this factor.

I don't think the size of the loadings is that important here, only the sign. In scale development, it makes sense to remove indicator variables with small loadings (e.g., <0.3) because the purpose is to come up with a reliable measurement instrument. However, the purpose of Emil's paper is not to develop a scale but to investigate the correlation structure of international socioeconomic differences.

(2014-Jul-31, 20:25:10)Emil Wrote: However, one point. Yes, it is possible that the 2nd factor is almost the same size as the first. I had actually checked this because I initially did some analyses in SPSS before moving to R (it's my first time using R for a project). Here's what one can do in R:

Code:
```y_ml.2 = fa(y,nfactors=2,rotate="none",scores="regression",fm="ml") #FA with 2 factors y_ml.2 #display results plot(y_ml.2\$loadings[1:54],y_ml\$loadings) #plots first factors cor(y_ml.2\$loadings[1:54],y_ml\$loadings) #correlation ^ y_ml.3 = fa(y,nfactors=3,rotate="none",scores="regression",fm="ml") #same as above just for 3 factors y_ml.3 plot(y_ml.3\$loadings[1:54],y_ml\$loadings) cor(y_ml.3\$loadings[1:54],y_ml\$loadings)```

One will get the 2 factor and 3 factor solutions using max. likelihood. Apparently the first factor is not completely identical across the nfactors to extract, but almost so. ML1 (with nfactors=1) with ML1 from nfactors=2 and 3 was .999.

With nfactors=2, the 2nd factor was much smaller. Var% for ML1 is about 41%, for ML2 it is about 11%,

With 3, ML1=41%, ML2=10%, ML3=5%.

Discuss that in the paper, and explain why you disregard the other factors. As to the number of factors, use Kaiser's rule (eigenvalue>1) or, if you can, parallel analysis. Are the other factors interpretable based on which variables load strongly on them?
I will update the paper with many changes based on your last substantive review. No worries. I will include a section on the number of factors to use, and their interpretability.

I also ran a oblimin rotation. With nfactors=5, all factors extracted correlate with each other in the right direction (note ML2 is reversed). This indicates a general, higher-order factor, yes?

Code:
```Loadings:                                                        ML1   ML3   ML4   ML2   ML5   Undernourishmentofpop                                                     1.01       Depthoffooddeficitcaloriesundernourishedperson                            1.01       Maternalmortalityratedeaths100000livebirths            -0.47              0.24 -0.40 Stillbirthratedeaths1000livebirths                     -0.57       -0.14  0.15 -0.20 Childmortalityratedeaths1000livebirths                 -0.53              0.15 -0.43 Deathsfrominfectiousdiseasesdeaths100000               -0.38  0.16        0.22 -0.56 Accesstopipedwaterofpop                                 0.43  0.11       -0.26  0.36 Ruralvs.urbanaccesstoimprovedwatersourceabsolutediffer -0.33       -0.20  0.21 -0.20 Accesstoimprovedsanitationfacilitiesofpop               0.62 -0.11  0.10 -0.18  0.25 Availabilityofaffordablehousingsatisfied                      0.17  0.14        0.18 Accesstoelectricityofpop                                0.52 -0.13       -0.33  0.35 Qualityofelectricitysupply1low7high                     0.36        0.45        0.18 Indoorairpollutionattributabledeathsdeaths100000       -0.17       -0.10  0.37 -0.30 Homiciderate12100000520100000                          -0.11  0.26 -0.59  0.21 -0.21 Levelofviolentcrime1low5high                           -0.18       -0.72             Perceivedcriminality1low5high                          -0.16 -0.14 -0.56             Politicalterror1low5high                               -0.17 -0.57 -0.32  0.13  0.11 Trafficdeathsdeaths100000                              -0.23 -0.19 -0.41       -0.19 Adultliteracyrateofpop.aged15                           0.97                         Primaryschoolenrollmentofchildren                       0.52        0.11  0.21  0.35 Lowersecondaryschoolenrollmentofchildren                0.57             -0.17  0.21 Uppersecondaryschoolenrollmentofchildren                0.59        0.20 -0.23       Genderparityinsecondaryenrollmentgirlsboys              0.60  0.17 -0.24             Mobiletelephonesubscriptionssubscriptions100people      0.45  0.11       -0.31       Internetusersofpop                                      0.38  0.17  0.38 -0.24  0.15 PressFreedomIndex0mostfree100leastfree                       -0.81 -0.11  0.12       Lifeexpectancyyears                                     0.28        0.14 -0.19  0.60 Noncommunicablediseasedeathsbetweentheagesof30and70p    0.12 -0.30 -0.19  0.11 -0.69 Obesityrateofpop                                        0.46       -0.17 -0.36       Outdoorairpollutionattributabledeathsdeaths100000       0.41 -0.34       -0.24 -0.23 Suicideratedeaths100000                                 0.50        0.11  0.21 -0.20 GreenhousegasemissionsCO2equivalentsperGDP                                0.21 -0.23 Waterwithdrawalsasapercentofresources                   0.25 -0.38       -0.13  0.14 Biodiversityandhabitat0noprotection100highprotection          0.31              0.14 Politicalrights1fullrights7norights                          -0.70        0.14 -0.18 Freedomofspeech0low2high                                      0.58              0.11 Freedomofassemblyassociation0low2high                   0.13  0.74                   Freedomofmovement0low4high                              0.13  0.69 -0.14             Privatepropertyrights0none100full                             0.40  0.57        0.14 Freedomoverlifechoicessatisfied                        -0.13  0.41  0.27        0.19 Freedomofreligion1low4high                                    0.89 -0.19             Modernslaveryhumantraffickingandchildmarriage1low100   -0.41                   -0.27 Satisfieddemandforcontraceptionofwomen                  0.64                    0.30 Corruption0high100low                                         0.41  0.63        0.11 Womentreatedwithrespect0low100high                     -0.14 -0.27  0.78       -0.16 Toleranceforimmigrants0low100high                      -0.33  0.53  0.23             Toleranceforhomosexuals0low100high                            0.42  0.23        0.39 Discriminationandviolenceagainstminorities0low10high   -0.19 -0.61 -0.27        0.11 Religioustolerance1low4high                             0.15  0.55 -0.16  0.12 -0.10 Communitysafetynet0low100high                           0.44  0.30       -0.13       Yearsoftertiaryschooling                                0.27        0.31 -0.16  0.12 Womensaverageyearsinschool                              0.87  0.10       -0.13       Inequalityintheattainmentofeducation0low1high          -0.87 -0.13 -0.19        0.14 Numberofgloballyrankeduniversities0none550                    0.11  0.50        0.25                 ML1  ML3  ML4  ML2  ML5 SS loadings    8.23 6.25 4.21 3.47 3.14 Proportion Var 0.15 0.12 0.08 0.06 0.06 Cumulative Var 0.15 0.27 0.35 0.41 0.47 Factor intercorrelations:       [,1]  [,2]  [,3]  [,4]  [,5] [1,]  1.00  0.19  0.33 -0.55  0.53 [2,]  0.19  1.00  0.27 -0.14  0.19 [3,]  0.33  0.27  1.00 -0.35  0.27 [4,] -0.55 -0.14 -0.35  1.00 -0.49 [5,]  0.53  0.19  0.27 -0.49  1.00```

To reproduce use:
Code:
```y.oblimin.ml = fa(y,nfactors=5,rotate="oblimin",scores="regression",fm="ml") print(y.oblimin.ml\$loadings,digits=2) print(y.oblimin.ml\$Phi,digits=2)```

(2014-Aug-01, 17:20:07)Dalliard Wrote:

Davenport, E. C. (1990). Significance testing of congruence coefficients: A good idea?. Educational and psychological measurement, 50(2), 289-296.

... I am left with the impression it's very bad method. You should be careful with that. (The version of the paper I have can't allow copy paste, but check the pages 293-295.) The congruence coeff seems to constantly give you very high value even in situations where they should be (theoretically) small, or not high at all.

Can you upload that paper or send it to me? There are other sources that are more sanguine about the CC, e.g., https://media.psy.utexas.edu/sandbox/gro...ruence.pdf In any case, the problems with using Pearson's r in the analysis of factor loadings are even greater.

Quote:gateway incorrect
error 502

Anway, I attach the documents you asked. The Davenport study, I have heard of it in this paper :

Quote:Table 13 provides the fit indices of the various factor models. The baseline model (Model 1: configural invariance) fits sufficiently, as judged by the CFI, although RMSEA is somewhat on the high side. Moreover, it is apparent that the metric invariance model (Model 2) fits worse than the configural invariance model does. All fit measures, except the CAIC, show deteriorating fit. Therefore, factor loadings cannot be considered cohort invariant (i.e., Λ1≠Λ2). Note that this is in stark contrast with the high congruence coefficient of the first principal component found by Must et al. (2003). This is due to the different natures of principal component analysis (PCA) and confirmatory factor analysis. PCA is an exploratory analysis that does not involve explicit hypothesis testing, as is the case with MGCFA. In addition, the congruence coefficient has been criticized for sometimes giving unjustifiably high values (Davenport, 1990).

Thus, if you wish, you can recommend the use of MGCFA testing of MI at the factor loading level. It's the best alternative of CC I am aware of.

Attached Files
Significance Testing of Congruence Coefficients - A Good Idea (Davenport 1990).pdf (Size: 696.64 KB / Downloads: 588)
A Monte Carlo Study of the Sampling Distribution of the Congruence Coefficient (Broadbooks, Elmore, 1987).pdf (Size: 951.04 KB / Downloads: 581)
I am in Leipzig for the time being visiting my girlfriend who lives there. I have only brought my laptop with me. I have access to my files and have R installed, however I don't have LATEX installed, so I cannot update the PDF file while I am here. I will get home on the 8th.

In the meanwhile, we can discuss problems with the paper. All quotes are from Dalliard.

Quote:So the criterion used was to extract just one factor. You should state that explicitly in the paper and justify the decision. At the limit, given that your variance explained is <50%, it is possible (though extremely unlikely) that there's a second factor that explains almost as much. In that case, denoting one of them as a general factor would be arbitrary. At the very least, you should tell how many factors with eigenvalues>1 there are.

I answered this partly before. I was only looking for a general socioeconomic factor and did not have any specific model in mind. No hypotheses were made about any non-general factors, nor was any specific model of the data advanced beforehand. This is why I did not use CFA either.

Quote:Still badly worded. Who are the 'they' referenced? Clark does not write about a "general socioeconomic factor." He writes about "social competence." Perhaps the Social Competence Factor would be a better label for your factor, given that some of its indicators are not usually thought of as indicating socioeconomic status.

I will remove the ”they”.

Quote:'Well-doing' is archaic-sounding and does not mean what you think it does:

well-doing (uncountable)
1.The practice of doing good; virtuousness, good conduct.

'Well-being' implies material prosperity, too, and is quite sufficient for your purposes:

well-being (uncountable)
1.a state of health, happiness and/or prosperity

Fine. I will use that then and drop the other.

Quote:If you are going to include an itemized description of SPI, you should tell more about DP, too.

If you think it is necessary. Note that whichever structure of the index the author decided on, I am not using it. I am only using their indicators. I showed the SPI just to show that there is some elaborate structure decided upon by the authors, which may not be supported by the actual intercorrelations in the data.

Quote:Most factor analyses will produce first factors that are substantially larger than the subsequent ones. What is the standard for "a very large" factor? The g factor is a general factor because all cognitive abilities do load positively on it.

More than a third of the loadings on the SPI factor are negative, so it's not a general factor. Similarly, if you had a cognitive test battery where some subtests loaded positively on the first unrotated factor and other subtests loaded negatively on it, there'd be no general factor, no matter how much variance the first factor explained.

Of course, in the case of the SPI factor the negative loadings are mostly an artefact of your failing to reverse code the negatively valenced variables. It's possible that this decision has some effect on all loadings.

Say, >30% I'd consider a large factor. I would not use the word ”fail” as that implies some attempt was made (which didn't succeed). No such attempt was made nor is it necessary IMO. The S factor is general in both datasets because: 1) it is very large (40-47% of var), 2) it is general in that almost all socially valued indicators load so that the desirability pole is towards the same end. I discuss the exceptions to this in a section in the paper too. Think about how the results could have looked like. Imagine a two factor result instead, with about half of the indicators loading on the one factor but not the other, each factor perhaps accounting for 20% of the variance. Clearly, this would be a most interesting result and clear disproof of any general country well-being. However, this is not what was found.

Quote:I cannot access that paper, but whatever bias the CC has is miniscule compared to the bias that Pearson's r can produce when it is used to compare factor loadings. Look at the example on p. 100 in The g Factor by Jensen. The CC is the standard method for comparing factor loadings in EFA, and you should use it.

I will report both in the paper. And in any case, the results were virtually identical.

Quote:State explicitly in the paper what you are doing. Computing correlations between factors can be done in many different ways (e.g., factor scores, congruence coefficient, CFA latent factor correlations).

I will update it to be more clear.

Quote:If you interpret a factor in a realist manner, as g is usually interpreted, then such studies are relevant, but your S factor seems completely artefactual.

'Completely artefactual'?

Quote:Ignore the replacement issue, with 1000 iterations you of course have to reuse the variables.

If you look in the code, you can see that it does exactly this for the subset x subset analyses:
1. Pick 10 random numbers between 1 and the number of variables without replacement (no overlap).
2. Divide that (unordered) list into two.
3. Get the first factor from each set of variables.
4. Correlate the scores from each first factor.
5. Save this result.
6. Repeat steps 1-5 1000 times.
7. Average the results.
8. Output the results.

In retrospect, I wrote the code in a dumb way that made it both slower (because using loops) and didn't save all the information generated only the final results.

Quote:OK, it's very linear. I'd include at least one of the scatter plots in the paper.

The other question refers to the fact that the variables with negative loadings may have an outsized influence on the MCV correlations because they somewhat artificially increase the range of values analyzed.

One could view the negative codings (not my doing) as inflating the variance which inflates the correlation. But one might as well argue that reverse coding them on purpose to produce only positive loadings is artificially decreasing the variance and thus lowering the correlations.

I ran the correlations again but with absolute values. The r drops a bit to around .95 which may be somewhat inflated. This is because any variable with a positive cor with IQ and negative loading (or reversely) would get 'fixed' to both positive so it would create a spuriously high correlation. I have attached the plot of the MCV on SPI with national IQ and absolute values. r=.95.

Quote:The MISTRA IQ correlation matrices have been published: http://www.newtreedesign2.com/isironline...RAData.pdf

I am aware. But it is not enough for my analyses. I need the scores too. I can repeat the loadings analyses but not the score x score ones.

Quote:I agree that the MCV results are perhaps of some interest.

It is well known that nations that are richer also have better health care, less malnutrition, higher life expectancy, better educational systems, better infrastructure, etc., so the S factor is unsurprising. The g factor is surprising because many assume that various domains of intelligence are strongly differentiated. Moreover, the realist interpretation of g does not rely just on the positive manifold, but on many completely independent lines of evidence (e.g., from multivariate behavioral genetics). So my criticisms are quite unlike Gould's.

What is this general socioeconomic factor? Why is it worth studying? Is it a formative or reflective factor?

Unsurprising, maybe, but no one has showed it to be there before. Remember that just because GDP (”richer”) correlates with variables X1, ..., Xn, does not show that these also intercorrelate strongly to create a large general factor.

The S factor provides a framework to think about national g proxies x other variables of interest. Right now whenever such a correlation is found, people usually attempt some specific theory of why this specific variable correlates with national g proxies (e.g. institutions, wealth, freedom of the press, atheism).

I don't know what you mean regarding formative vs. reflective factor. Perhaps you can link to some material that covers these concepts or explain them briefly.

If it concerns causality, I think S is primarily caused by G, but that there is some backwards causation in poorer countries due to nutrition (vitamin deficiency, protein deficiency), health care (certain illnesses may lower g) and perhaps pollution (e.g. heavy metal poisoning). Political structure also has an influence on S, especially totalitarian regimes of the communist variety seem to make things worse (China, North Korea, Cuba, Venezuela). However, as I said earlier, I don't want to push this or that interpretation in the paper as this would 1) make it much longer, 2) make it take forever to get thru peer review. I think such discussion is better left for another paper (or a book).

Quote:There are all kinds of stuff in your supplemental materials, but if someone wants to replicate or extend your analysis, correlation matrices are what they need. Although I think it's unfortunate that you have not reversed the scoring of the negative variables.

Well, since I published all the data files, they can easily generate the correlation matrices if they want those. They are not optimal because one cannot do score x score analyses with them. I don't see any reason to specifically attach correlation matrices when all the data files are there already.

Quote:Methodologists are adamant about the fact that PCA and FA are quite different animals. But I'm not going to insist on this.

I can insert a note saying that they are treated as the same even tho some people think they should not be so. Sounds good?

Attached Files Thumbnail(s)

I manged to install LATEX and get it working.

Changes:
- Got rid of “well-doing”, replaced either with “well-being” or nothing when appropriate.
Since I was only concerned with the question of a general factor, all analyses used only the first factor.
- Removed “they” from the sentence in Introduction.
- Added a plot with MCV results.

Attached Files
I found a mistake in my paper. When I calculated the mean correlation, I used mean() on the correlation matrix. However, the diagonals are always 1, so this inflates the results upwards. I think the differences are slight, but I will fix this in my next revision.
1) "I answered this partly before. I was only looking for a general socioeconomic factor and did not have any specific model in mind. No hypotheses were made about any non-general factors, nor was any specific model of the data advanced beforehand. This is why I did not use CFA either."

You should mention the size of the subsequent factors, i.e., variance explained.

2) 'General factor' is still not defined in the paper. The fact that lots of the variables have negative loadings on the first factor and the implications of this fact should be discussed straight away in section 3. How many loadings have the correct sign? It would make sense to merge sections 3 and 8, because you should discuss what the loadings look like before rather than after running all those analyses using the loadings.

3) "I also ran a oblimin rotation. With nfactors=5, all factors extracted correlate with each other in the right direction (note ML2 is reversed). This indicates a general, higher-order factor, yes?"

Yes, this is evidence in favor of a general factor unlike the analyses in the paper. However, the number of factors extracted should be decided based on accepted methods (Kaiser's rule, scree plot, parallel analysis...) rather than forced on the data.

4) "Similarly, the congruence factor was 1.0."

It's the congruence coefficient.

5) In sections 4-6 the method of calculating the factor/component scores should be mentioned, and it should be clarified that when different extraction methods are compared across different numbers of variables, PCA is always compared to PCA, ML to ML, etc.

6) "These very high correlations resulting from the use of method of correlated vectors in these two datasets give indirect support for researchers who have been arguing that the heterogeneous and lower than unity results are due to statistical artifacts, especially sampling error and restriction of range of subtests"

This sentence is unclear because it is not specified what the researchers have argued. Mention g loadings, IQ, or something.

7) "There is a question concerning whether it is proper to analyze do the MCV analyses without reversing the variables that have negative loadings on the S factor first. Using the non-reversed variables means the variance is higher which increases the correlation. Reversing them would decrease the correlations. I decided to use the data as they were given by the authors i.e. with no reversing of variables. As one can see from the plots, reversing them would not substantially change the results."

Say what the MCV results are with reversed variables.

8) "The analyses carried out in this paper suggest that the S factor is not quite like g. Correlations between the first factor from different subsets did not reach unity, even when extracted from 10 non-overlapping randomly picked tests (mean r’s = .874 and .902)."

You compared factor scores, Johnson et al. compared latent factors in CFA. Different methods, so no reason to expect similar results. It is unsurprising that factor scores based on 10 variables (many with measurement scale issues to boot) are not perfectly correlated with the underlying factor because such scores, especially PCA ones, have plenty of specific and error variance in them. In contrast, CFA factors based on, say, 10 or more variables are highly stable and contain no specific/error variance. When you increase the number of variables to 40 or 50 in EFA, factor scores will be much more highly correlated with the putative underlying factor than with 10 variables. The almost-unity correlation between the SPI and DP factors is an indicator of the S factor's stability while the lower correlations using 10 variables are what you would expect regardless of the nature of the underlying factor--for example, the correlation between g scores from two IQ batteries with 10 subtests each will not be 1.00.

9) "I don't know what you mean regarding formative vs. reflective factor. Perhaps you can link to some material that covers these concepts or explain them briefly."

In reflective factor models the factor causes differences in its indicators, while in formative models the factor is non-causal and the indicators cause variance in it. The g factor is typically seen as a reflective factor (general intellectual capacity causes performance differences in cognitive tests), while SES is typically seen as a formative factor (differences in income, occupation, etc. cause differences in SES).

Is your S factor just a stalking horse for the G factor? Then you could argue that different indicators of international well-being are just "items" in an international IQ test, cf. Gordon's discussion of life events as IQ items in his "Everyday life as an intelligence test." However, I think that this analogy is awkward when the cases are countries rather than individuals, and the psychometric quality of the international IQ data is as bad as it is.

---

(2014-Aug-01, 23:32:03)menghu1001 Wrote:
(2014-Aug-01, 17:20:07)Dalliard Wrote: There are other sources that are more sanguine about the CC, e.g., https://media.psy.utexas.edu/sandbox/gro...ruence.pdf In any case, the problems with using Pearson's r in the analysis of factor loadings are even greater.

Quote:gateway incorrect
error 502

The CC paper is attached.

Attached Files
Good criticism. I will work on a new draft tomorrow

ETA. I didn't have time to do the reverse coding today. So I will reply later.
I am working on the reversing, but it is unclear how this should be done. For instance, in the DR dataset, there is a variable about electricity use per capita. Is that considered good or bad? On the one hand, it is a sign of a rich country that we are able to use some many resources per person. In the environmentalist view, it is bad since we are... using so many resources per person. There is no obvious way to deal with this.

Reversing the obvious cases in the SPI dataset (there are no obvious cases in DR), results in a MCV r=.98. Very similar.

I have reversed the following vars: 1:6,8,13:18,26,28:33,35,42,48,53

Attached Files Thumbnail(s)

This one has a new section of oblimin rotated factor analysis. Results are much the same as using the first unrotated factor: r's = .97.

I have added more discussion of reversing variables, and results of MCV using those. Almost the same results too.

Rewrote some of the discussion concerning comparisons with the Johnson et al studies.

Attached Files