Hello There, Guest!  

[ODP] - Parents’ Income is a Poor Predictor of SAT Score

#11
MengHu Wrote:Emil and duxide, if you can read this message...

i would like to request something important. I don't have much free time, probably you don't either. Each time I review some articles, when the author makes changes, he has never precised WHICH part of the article is modified or what has been added. I must constantly search for the addition and modification, which causes me to re-read the article entirely once again. I prefer not to waste my time searching for this, for that...

So, I suggest either 2 options :

1. the author copy-past in the forum which portion is added or modified.
2. the author marks in color (blue or red) the portion that is added or modified.

What do you think ?


There are various ways to do this. One can certainly color things. If one writes in a standard word processor, then one can have it record changes automatically. This is the standard practice I think.

Since I usually write in LATEX this option isn't easily available.

Get Matlab here:
https://torrentz.eu/search?q=matlab

csdunkel Wrote:I am not reviewing the manuscript, but is it OK to make a simple suggestion? Please spell out the full name before using the SAT and ACT. This is of minor importance, but I have run into confusion with the SAT because it is the same acronym used for the Stanford Achievement Test.


As also noted above, they are not real acronyms anymore. One can refer to them as widely used achievement tests in the US or some such if one wants to, or by their previous name, or by the publisher. I don't care that much. :)
 Reply
#12
(2014-Jun-11, 23:33:21)nooffensebut Wrote: So, if my model can achieve an R^2 of 0.942, then adding in those indirect effects could add up to 0.058 to the coefficient of determination, and you want me to think that such a miniscule change would so profoundly transform my model that, in its current form, “there is no possibility to say which [variable] has the strongest effect.”


This is because you believe R^2 tells you the predictive power of the direct effects of the variables in the model while in reality it tells you the predictive power of the variables included in the model. In other words, it still regards the total effect.

It's interesting that each time I try to constrain the indirect paths in SEM model to zero, the r² systematically decreases. And I'm thinking that if r² regards only the direct paths, as you imply, this can never happen.

I believe I had a good paper on this subject about the irrelevance of r² in multiple regression, but I'm busy and I cannot find it among the messy millions of documents I have in my computer. (but I'll edit the post later if I find it)

Quote:that MR underestimates the true explanatory power of a variable because it only includes direct effects

My earlier comment was clear : the total effect can be either lower, higher, similar than the direct path.

I will probably come here later, because right now, I find it impossible to understand the 2nd part of your comment.
 Reply
#13
<blockquote>This is because you believe R^2 tells you the predictive power of the direct effects of the variables in the model while in reality it tells you the predictive power of the variables included in the model. In other words, it still regards the total effect.</blockquote>

Do you have a source for any of this logic, or are you coming up with it, yourself? When you say total effect, you mean the true total effect that accounts for longitudinal SEM variable effects??? No, the coefficient of determination is specific to the model under consideration. It is the explained sum of squares divided by the total sum of squares, using the variables specified without additional interaction variables, unless the model includes them.

The whole point of standardized regression coefficients is to be able to compare them. Otherwise, why standardize? Research that uses multiple regression and compares beta coefficients is common, but, as <a href="http://pareonline.net/pdf/v17n9.pdf">Nathans et al</a> point out, “it is often not best to rely only on beta weights when interpreting MR results. In MR applications, independent variables are often intercorrelated, resulting in a statistical phenomenon that is referred to as <b>multicollinearity</b>…” (emphasis added). However, they still recommend calculating beta weights: “It is recommended that all researchers begin MR analyses with beta weights, as they are easily computed with most statistical software packages and can provide an initial rank ordering of variable contributions in one computation. If there are no associations between independent variables <b>or the model is perfectly specified, no other techniques need to be employed”</b> (emphasis added). Berry and Feldman’s Multiple Regression in Practice points out that “because multicollinearity increases the standard errors of coefficient estimators, the major effect of multicollinearity is on significance tests and confidence intervals for regression coefficients. When high multicollinearity is present, confidence intervals for coefficients tend to be very wide, and t-statistics for significance tests tend to be very small” (p. 41). My model achieved high coefficients of determination, which reflects positively upon its specifications. I examined my results to see if the standard errors were high or the confidence intervals were wide. They usually weren’t because the sample sizes were large, (defining sample size by the number of state averages, rather than the number of students). They were worst for the education variable when it meant having at least a high school education. I think it maxed out at 97 with a confidence interval of -200 to 180 (not standardized coefficients), when race was Asian, and income was greater than $20K. I could use this information to support my conclusion that the education variable is not useful when it is high school diploma, but it is difficult to briefly quantify hundreds of regression results. So, the issue of multicollinearity actually strengthens my conclusions.

<blockquote>I find it impossible to understand the 2nd part of your comment.</blockquote>

Reductio ad absurdum takes one to one’s logical conclusion. The whole point is to show that your logical conclusion should not make sense.
 Reply
#14
nooffensebut,

Did you update the PDF file in the original post? That's not usually the way we do it here. 1) It removes the previous edits so that one cannot follow the edit changes backwards e.g. to see the original submission. 2) When one updates the file, the thread is not marked as "new post", which means that reviewers don't know that anything has happened.

For these reasons, it is best to post a new reply every time one has a new edit, except in very trivial cases (like changing a typo or something).

Sorry if this practice was not clear. I will add a note about it.
 Reply
#15
@Emil

Sorry. I made only slight changes on two occasions, which I briefly described in the original post message.
 Reply
#16
So far there are two reviewers who agree to publication (here and here).

I don't have any objections. Meng Hu seems to want to discuss some stuff about how to interpret MR. Are they necessary to resolve (if resolvable) before publication?

I did not see any obvious mistakes in the paper, so I concur with publication. Can Meng Hu say whether he agrees, and if not, what needs to be changed specifically?
 Reply
#17
nooffensebut Wrote:Do you have a source for any of this logic, or are you coming up with it, yourself? When you say total effect, you mean the true total effect that accounts for longitudinal SEM variable effects??? No, the coefficient of determination is specific to the model under consideration.


I have, if I remember, already given one proof of it, in the 2nd paragraph in my comment. Also, I thought everyone who is doing MR should know that r² measures the predictive power of a regression model. And by your comment, you show you already know that. The only instance where r² is equal to the regress coeff is when you have only 1 indep var., and in this case your MR is nothing more than a bivariate correlation.

In any case, if my entire point on MR is wrong. That necessarily means the SEM stuff makes no sense at all. That is, there is no such thing as indirect effect or total effect. Before going this far, you must remember that SEM is nothing more than a MR which, however, has more possibilities, e.g., multiple dependent var., longitudinal and repeated measures, model fit, etc. Given that my blog post showed that the direct path in SEM was indeed the regress coeff as shown in MR, I don't have any doubt about my main point. If you disagree again, you have to explain why the direct path in SEM is the exact equivalent of regress coeff in MR.

Quote:The whole point of standardized regression coefficients is to be able to compare them. Otherwise, why standardize?

I don't understand. I have already explained this point in my post. It does not matter, B or Beta. You must know the total effect. Or at least make some assumptions. For instance, if you have an indep. var. that is unlikely to be caused by others. Say, if you include parents income and parents education, I'm sure you can assume and write that parents income will not cause parents education, and if there is a direction, it must be education->income.

And I don't also understand the stuff about CI of the education variable which includes zero in its band. Because we are still dealing with the direct path.

Another thing I don't understand is why you have put in bold the term multicollinearity. If you think you don't have multicollinearity, then, does the absence of multicollinearity proves there is no correlation between the indep var. ? i.e., that there is no indirect effect ?

See here, by the way.

Quote:If there are no associations between independent variables or the model is perfectly specified, no other techniques need to be employed

The MR is not supposed to give you the r's between the indep. var., or perhaps it's possible to request them, but I don't remember seeing them in SPSS or in Stata anyway. Thus I don't understand how it is possible to make such conclusions. SEM instead can give you the r's between the indep var.

.... .... ....

Emil, normally, I give a(n) (dis)approval+comment if I think I know what to say and what it's about. In the present case, if I accept publication, it means I must endorse a use of MR that I believe is just wrong. Of course, my opinion on that is unlikely to be accepted by others, unless a superman of statistics endorse this view that this is followed by many others. Thus, I don't know what to do. Actually, my feeling is more "no" than "yes" because I don't accept argument from authority, that is, just because most people disagree with me does not mean i'm wrong. This issue must be discussed and we must arrive at an agreement (if we can). So, I'm still continuing the conversation. i want to reach an agreement.

By the way, I don't think it's a problem if I had to refuse to approve. If someone else decides it must be published, I will not say anything more. 3 agreement=publication, it's simple as that. Like I said before, the best strategy is always to side with the majority, even if you're wrong. In other journals, it's clear no one else will take my comments on MR seriously, and everyone will ignore me. Then if you do the same, i will not object.
 Reply
#18
<blockquote>I don't think Jensen necessarily understands the problem of MR, given what I remember he has written about MR in The g Factor</blockquote>

Well, not only is it true that The g Factor should not have been published, but really all research on IQ and all research that considers race should be stopped because, as you pointed out:

<blockquote>The only way to get your full effects properly is by way of SEM+longitudinal data. There is no other way. Causality must deal with changes (over time). A “static mediation” like what happens when they use cross-sectional data is nonsense.</blockquote>

Thus, the only way to prove the causality of IQ’s influence is to greatly change a person’s IQ, which can be hard to do. In fact, it is impossible to change a person’s race. (<a href="http://www.dailymail.co.uk/news/article-2645950/I-fun-bein-Korean-Blonde-Brazilian-man-undergoes-extraordinary-surgery-achieve-convincing-Oriental-look.html">Or is it???</a>)

<blockquote>I have, if I remember, already given one proof of it, in the 2nd paragraph in my comment.</blockquote>
<blockquote>It's interesting that each time I try to constrain the indirect paths in SEM model to zero, the r² systematically decreases. And I'm thinking that if r² regards only the direct paths, as you imply, this can never happen.</blockquote>

Okay, so beta coefficients can’t see the total effect, but coefficients of determination can because they can in an SEM model. I assume you are referring to some other research project you have, but if it is not MR, how does that prove that the coefficient of determination in an MR model can see the same thing?


<blockquote>I have already explained this point in my post. It does not matter, B or Beta. You must know the total effect.</blockquote>
<blockquote>multiple regression should be used to test whether any variable can be weakened in its direct effect by other independent variables but not to test between them as if they were in competition with each other.</blockquote>

Therefore, any MR study that mentions a beta value is wrong because beta values assume that there is some purpose in standardizing a variable for the purpose of a comparison, which is wrong according to your research.
 Reply
#19
Heavy metal poisoning seems to be useful in changing one's IQ. Only in the wrong direction, however.

By the usual standards, publication can proceed. I am waiting merely to see if some form of agreement can be found regarding the MR issue. I'm afraid I'm not quite certain what the exact problem is with the paper. MR finds only the strength of the direct path, right. A small or zero b does not prove that a variable has a small total effect, sure. This does not seem to be news.

My understanding is that the point of the paper is to respond to the claim that SAT tests just measure parental income. The author shows that this does not pan out in regressions with other important variables.
 Reply
#20
I said before I had a paper on r². It took me an eternity to find it. Hopefully I got it. the reason why I failed to catch that right in time, perhaps not uniquely due to the fact I have lot of documents, but because of this, I create multiple folders, one for CFA/SEM, DIF, multiple regression, other basic stats things, and all, because too lot of things. But the paper was a presentation of a tool for R package, so I put it there, in my folder "R packages", whereas I was searching again and again in my folder "regressions" and "CFA/SEM"... in vein.

You can have a free link here :

http://www.tarleton.edu/institutionalres...ethods.pdf

The proof is given directly in table 6, where the R² is the total of the unique and common effects. They also say :

Quote:Also called element analysis, commonality analysis was developed in the 1960s as a method of partitioning variance (R2) into unique and nonunique parts (Mayeske et al., 1969; Mood, 1969, 1971; Newton & Spurrell, 1967).

And because I'm a little curious, I have searched through Google scholar, and found this one :

Seibold DR, Mcphee RD. Commonality analysis: A method for decomposing explained variance in multiple regression analyses. Human Communication Research. 1979;5:355–365.

Quote:Whether in the “elements analysis” of Newton and Spurrell(1967), the “components analysis” of Wisler (1969), or the “commonality analysis” of Mood (1969, 1971), it was also noted that the unique effects of all predictors, when added, rarely summed to the total explained variance.

So in the case of even five predictors R2 may be decomposed into 31 elements. Since 26 of these are commonalities, difficulties in interpreting higher-order common effects increase in proportion to increases in the number of predictors.

And by the same token, I also found this :
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2930209/

It says the same thing, and I haven't read it carefully (just quickly) but the first one is more than sufficient.

(2014-Jun-16, 01:51:23)nooffensebut Wrote: Okay, so beta coefficients can’t see the total effect, but coefficients of determination can because they can in an SEM model. I assume you are referring to some other research project you have, but if it is not MR, how does that prove that the coefficient of determination in an MR model can see the same thing?


You can see above. But regardless of that, it's common sense. I mean, MR and SEM are both multiple regression. It's the same stuff. It's just that SEM offers more possibilities and thus is even more complex. Look here. Same numbers everywhere.

http://humanvarietiesdotorg.files.wordpr...ession.png
http://humanvarietiesdotorg.files.wordpr...os-sem.png

Quote:Therefore, any MR study that mentions a beta value is wrong because beta values assume that there is some purpose in standardizing a variable for the purpose of a comparison, which is wrong according to your research.

No. In my last comment I have said you can make some assumptions. But this should be either theoretically based on supported/suggested by previous research. This relies on strong assumption of course, and it makes your interpretation very dependent on your assumption. This is doable but you cannot say there is no ambiguity in MR.

Emil Wrote:A small or zero b does not prove that a variable has a small total effect, sure. This does not seem to be news.


Considering how many if not all of the researchers interpret MR coefficient the wrong way, I have the impression this is new to them. Because if not, then I don't understand why they continue to say stuff like that.

In any case, I'm not arguing the author must agree with me here. Like I said, my opinion is not "recognized". However, if you want me to approve, then the author has 2 options :

1. He beats me in the ongoing dual. And I will withdraw my argument, and approve.
2. He chooses to make explicitly in the text that he acknowledges the "possibility" that MR coefficient evaluates the direct effect (not total) of the independent variables, and thus in this case, the total effects of all these variables need not be the same as what is displayed by the MR regress coefficient. For example, he can say (linking to my blogpost) that MR coefficient is the equivalent of the direct path in SEM models. I need only this modification, a sort of "caveat" for the readers, and then I approve. Once again, I don't say he needs to endorse my views, but only shows he is open to this possibility.

Does it sound reasonable ?

EDIT.

Emil Wrote:My understanding is that the point of the paper is to respond to the claim that SAT tests just measure parental income. The author shows that this does not pan out in regressions with other important variables.


I'm Ok with that. I don't doubt the effects of these variables. I'm just thinking about their relative strength, e.g., which one is the strongest. Or what is the total effect of x1, x2 etc...
Some variables have strong effect, e.g., participation, some others have an effect that is close to zero, e.g., income. And like I said before, I doubt income can cause education (unless education is time2 and income time1, in a repeated-measure analysis). But now, like I also said, the more indep var you have, the more you need to disentangle these possible indirect effects, and write either x1 can cause x2, etc.

I never said he needs to redo the analysis. Sure not. But he needs to makes crystal clear how he interprets the output, and specifically, the possible indirect effects.
 Reply
 
Forum Jump:

Users browsing this thread: 2 Guest(s)