Hello There, Guest!  
 Previous 1 2 3 4 5 7 Next   

[ODP] An update on the narrowing of the black-white gap in the Wordsum

#21
(2014-Oct-10, 02:11:07)Chuck Wrote: What were the gaps for each age group in the first and last survey year?


Cohort or survey year ? I thought you would need cohort as well ?
I don't know how you want me to calculate the age gap. By way of correlation ?
The numbers are displayed below, for survey year categories. I also display the SD of wordsum and age for black and white groups separately, you see that the SD of the two variables declines over time. This means that the increase in age gap by survey year is under-estimated. Still, I think the increase in the age gap is clearly big. (the correlations are computed for the entire group, I did not separate blacks and whites)

correlate wordsum age [aweight = weight] if year4==1

r=0.0684

sd of wordsum
2.01 (for blacks) 2.12 (for whites)
sd of age
14.62 (for blacks) 14.61 (for whites)

correlate wordsum age [aweight = weight] if year4==2

r=0.0934

sd of wordsum
1.95 (for blacks) 2.09 (for whites)
sd of age
14.51 (for blacks) 14.12 (for whites)

correlate wordsum age [aweight = weight] if year4==3

r=0.1174

sd of wordsum
1.86 (for blacks) 2.02 (for whites)
sd of age
12.81 (for blacks) 13.29 (for whites)

correlate wordsum age [aweight = weight] if year4==4

r=0.1541

sd of wordsum
1.87 (for blacks) 1.84 (for whites)
sd of age
13.52 (for blacks) 13.80 (for whites)

The numbers below are the correlations for blacks, and whites separately.

correlate wordsum age [aweight = weight] if year4==1 & bw1==0
correlate wordsum age [aweight = weight] if year4==2 & bw1==0
correlate wordsum age [aweight = weight] if year4==3 & bw1==0
correlate wordsum age [aweight = weight] if year4==4 & bw1==0

-0.0480
0.0298
-0.0592
0.0059

correlate wordsum age [aweight = weight] if year4==1 & bw1==1
correlate wordsum age [aweight = weight] if year4==2 & bw1==1
correlate wordsum age [aweight = weight] if year4==3 & bw1==1
correlate wordsum age [aweight = weight] if year4==4 & bw1==1

0.0706
0.0897
0.1318
0.1258

No increase in age gap among blacks but a large one for whites.

Now, if you want the results by way of regression, the unstandardized coefficients are shown below, again for each category of survey year. For blacks first, and for whites next.

tobit wordsum age if year4==1 & bw1==0 [pweight = weight], ll ul
tobit wordsum age if year4==2 & bw1==0 [pweight = weight], ll ul
tobit wordsum age if year4==3 & bw1==0 [pweight = weight], ll ul
tobit wordsum age if year4==4 & bw1==0 [pweight = weight], ll ul

-.0070517
.0041364
-.0091082
.0007717

tobit wordsum age if year4==1 & bw1==1 [pweight = weight], ll ul
tobit wordsum age if year4==2 & bw1==1 [pweight = weight], ll ul
tobit wordsum age if year4==3 & bw1==1 [pweight = weight], ll ul
tobit wordsum age if year4==4 & bw1==1 [pweight = weight], ll ul

.0113877
.0145335
.0220242
.0179635

There is no tendency for age gap to increase in blacks. But the tendency is visible for whites. Remember that the unstandardized coeff is the effect of one-unit change in age var on the wordsum. To obtain a gap between, say, 18 and 68-yrs-old, you must do 0.0179*50=0.895, i.e., a difference of almost one word correct. The age gap for the first category of survey year is 0.01138*50=0.569. The difference is 0.33 word correct, which is more or less the same amount of the gap narrowing in black-white when you look at survey year which is about 0.30.

So, what do you think ?

----

By the way, I have detected a silly mistake in my syntax. All analyses are weighted, indeed. Except my correlation of wordsum with age. Initially, I reported it to be 0.1005. But the weighted correlation is 0.1085. So, I made this modification (in the appendix and in the text) in my latest version.

This mistake is due to the fact I'm not very used to Stata, even though I know the commands well. In SPSS, when you use option weight, all of your subsequent analyses are weighted, unless you decide to unable weight option. In Stata however, each analysis must be weighted individually (or not). So that's a big difference.
 Reply
#22
It's difficult for me to make sense of the correlations. Could you just supply B/W d-values e.g.,

Survey year group 1 (first)

Age group 1
Age group 2
Age group 3
Age group 4

Average:

Survey year group 4 (Last)

Age group 1
Age group 2
Age group 3
Age group 4

Average:
 Reply
#23
But I thought the regression would be easy to interpret, even if in unstandardized coefficients. Anyway, I attached the file. Generally, the BW gap diminished over time only when you examine the intermediate age groups (2 and 3). But the BW gap is smaller (irrespective of survey years) when the age group is younger.

The syntax I have used is :

recode age (18/27=1) (28/40=2) (41/52=3) (53/69=4), generate(agegroup)

by bw1 agegroup year4, sort : summarize wordsum age [aweight = weight] if age<70

Black-White d gap
by survey year

agegroup1

year1 0.6440
year2 0.5585
year3 0.4721
year4 0.7106

agegroup2

year1 0.6610
year2 0.7089
year3 0.5908
year4 0.5726

agegroup3

year1 0.8831
year2 0.6103
year3 0.7265
year4 0.7368

agegroup4

year1 0.8914
year2 0.6657
year3 0.9255
year4 0.8280

Black-White d gap
averaged by age group within survey year

year1 0.7699
year2 0.6358
year3 0.6787
year4 0.7120


Attached Files
.xls   wordsum GSS black-white d gap by age group by survey year.xls (Size: 13.5 KB / Downloads: 454)
 Reply
#24
1) Capitalize Wordsum throughout, it's a proper name.

2) "The right procedure should be to use a tobit regression (for an introduction, see, McDonald & Moffitt, 1980)."

Can you add a brief explanation of why tobit regression is the right procedure to use here?

3) "Since the year 2000, the GSS begins to ask whether the respondent is hispanic or not"

Began to ask. Capitalize Hispanic.

4) "The variable educ has values going from 0 to 20."

Are the values years or what?

5) It would be better to describe the variables and their values in a table.

6) "The full result is available"

--> "the full results are available"

7) "with no subsequent convergent since"

--> convergence

8) "the ceiling effect in the white sample suggests that the black-white difference (but not the changes) was under-estimated"

Why is the d-value change correctly estimated in your models despite the ceiling effect? Elaborate.

9) "In early cohorts, there was a strong ceiling effect in the white sample, but this effect progressively disappears in the successive cohorts."

Does it disappear, or just diminish?

10) "Generally, there is some indication that the black-white gap has been underestimated in early cohorts. And by the same token, the magnitude of the gap narrowing. But at the same time, the white trend could have been even flatter or turned out to be somewhat dysgenic."

This passage is obscure for me. Elaborate a bit.

11) You mention that before 2000 Hispanics were not disaggregated from non-Hispanics in the GSS. This seems to be a potentially important confound for black-white differences. What's the effect on the b-w gap of including or not including Hispanics since 2000?

12) In the references list, remove "Notes and Shorter Communications" from the name of Lynn's paper.

13) In Figures 2 and 3, could you draw lines between the data points? At least those two figures should be in the main text. It doesn't matter if they cannot be smoothly embedded in the main text because it's a much greater nuisance to have to scroll to the end of the paper to look at them. Remember that figures showing the main results are THE MOST IMPORTANT THING about a paper because they are what most readers look at first, and the only thing many readers look at.
 Reply
#25
MH in fact fixed many of those things. It is just that instead of uploading the revision paper (or revision in progress), he sent it to me in email. This meant that Dalliard spent time reviewing an older draft for no reason. It would be proper if the author would update the revision on OSF as soon as a new one is available so reviewers (who work for free, remember) don't waste their time.
 Reply
#26
Dalliard :

1) Ok

2) I added the paragraph already in my version 2. But I hesitate to post it before I have reply from Chuck. He wants me to explain why the use of cohort or year makes a difference. But I have no clue, so I have not added discussion about that. Note that Huang & Hauser didn't attempt to explain such difference.

3) Chuck asked me to do it. And I capitalized all names.

4) Yes. I will add the information.

5) Should I delete these paragraphs and make a summary table instead ? I'm not sure how I should present it. Let me think about it.

6) Ok

7) ok

8 ) I'm afraid you're right. I don't know why I have written this. I should have said the magnitude of d gap and the narrowing are both under-estimated. I will make the changes.

9) It has diminished. But the difference is still meaningful. I have posted the graphs here :
http://openpsych.net/forum/showthread.ph...79#pid1879

10) It is the white score distribution that is "censored" at the right tail, i.e., ceiling effect. I don't see such right censoring for blacks. So, I think the white true score is under-estimated. But as I said, it's only in the early cohorts that you see the ceiling effect among whites. There is only a modest ceiling effect in late cohorts. So, if ceiling effect diminishes over time, this necessarily implies that the scores for whites in early cohorts have been under-estimated compared to late cohorts. Consequently, the BW narrowing is under-estimated.

11) I did not say that race before 2000 did not disaggregate whites from hispanics (perhaps I should add a note about that). In footnote 8, I have written "According to the GSS codebook, the "white" category in variable "race" (before the year 2000) includes Mexicans, Spaniards and Puerto Ricans "who appear to be white"." Of course, you can think that misclassification has occurred. I also believe that, but I have the impression that it won't make a big difference. If I'm not mistaken, most mexicans, spaniards and puerto ricans don't appear white at all. Although I can't tell for sure; I don't live in the U.S.

That said, I have created two variables bw1, and bw. You see in the syntax that I use bw1 because it removes all hispanics from the category "white" (after survey year 2000). When I used bw instead of bw1, I remembered that nothing really changed. But you're right that I hadn't said anything about it in the limitation section. I will make the changes.

12) It was like this when I cited it in google scholar, I have hesitated, but perhaps you're right.

13) Personally, I don't mind scrolling down to the end to look at the graphs. But since you and Emil asked me to do that, I will do it. But I maintain I don't understand the bother. For the lines in the graphs, I have tried, but I don't know how to do that. I will search more today, and if I find a way, I will make the graphs with the lines.


-------

Emil

I sent you the version 2 (although today I have added more things to it) because I wanted to know which version you prefer. The one with the graphs at the end of the paper or the other version ? Because I couldn't find a way to make "resize" without deterioration of quality, although I think they are still acceptable.
 Reply
#27
(2014-Oct-11, 20:36:25)Meng Hu Wrote: But the BW gap is smaller (irrespective of survey years) when the age group is younger.


Which explains why you get different results when using (a) survey year controlling for age and (b) survey year minus age. It's not clear which better represents the true cohort effect, which would best be indexed by looking at same age persons at widely different times. Generally, in my opinion, the GSS provides no clear evidence that the B/W difference has substantially diminished across time. The time periods for which you can actually match age groups show only a small decrease in difference. And H&H's method is confounded by the presence of an age x BW d interaction. Add a brief discussion of this.

I don't see any material problems regarding statistical analysis. Post an updated draft so that we can double check for language problems.
 Reply
#28
I attach the new doc file. The pdf is at OSF. (Look for version 2).

https://osf.io/hiuzk/files/

As well as the new xls file (with the Stata syntax as well; look for version 2). It contains the analysis with bw variable instead of bw1 variable, as requested by Dalliard. No meaningful change (only 0.1 word correct).

I have made lot of modifications in the text, and notably rewritten the sentences as suggested by Chuck. I fixed language issues, and added reference about Hunter & Schmidt (2004) as requested by Emil, explanations about what tobit model does, discussion about the potential effect of racial misclassification (hispanics and whites together or apart for years 2000+), and the absence of test of normal and homoskedastic errors for tobit regression.

But the major change is in tables 5-6. Initially, i used tobit for looking at the changes among low and high scoring groups. But after reflexion, it is preferable to use OLS. After all, if tobit is used to estimate the true population value of Y when the data is censored (either clustering or ceiling/floor effect), the goal of my analyses on the subsamples (high and low ability score) is just to examine the behavior in these parts of the entire group. I'm not interested in the true population value of Y (which, i have probably forgotten), then tobit is not the best choice. So I used OLS instead. But if you look at the new xls and the old one, you see there is a difference in the parameters (mean score and size of gap) but no difference in the behavior of the trend lines.

By the way, I have emailed lot of people, notably Sean Reardon and Derek Neal, concerning the ambiguity between cohort and period effect in my analyses. No one has responded. If you know someone else I can contact, feel free to tell me so. Furthermore, I have emailed several econometricians about my use of tobit regression but I didn't have lot of comment, except the one from Jeong-han Kang (but I'm still debating with him). I have also emailed MacCallum concerning dichotomization of continuous var for logistic regression. No answer, but I have already decided to remove my analysis anyway.

I have re-emailed Huang, and asked him what SD he has used to compute d values, because what he termed pooled across-years and pooled within-years is not clear to me. He said he used the "SD of the entire sample" (this is his own words). By this, I think he just did something like "summarize wordsum" in stata. That is to say, not disaggregated by race or cohort or year.

Then, i asked Paul Sackett for his opinion. He said he prefers the use of pooled SD, and does not recommend SD of entire sample or population. Either you should use black and white pooled SD or the majority group SD (i.e., whites). However, he agrees with Huang that it is better to use a "fixed" SD because it's invariant over time. Instead, I have used mean(white)-mean(black) in cohort1 divided by SD obtained in cohort1, and did the same for the other categories. Even though Sackett won't recommend it, I think you should remember he did not gave me argument, only his opinion. And I think his opinion is seriously wrong.

Just think about it. The SD in wordsum declines over time. What has caused this effect ? If the gain in wordsum, especially for blacks, is due to environment, one can guess that the lowest IQs in the black group has gained more than the highest IQs in the black group (this is true to a lesser extent in the GSS data). Automatically, this reduces variance in IQ, hence the SD.

If you use the SD of the entire sample (2.11) which is similar to the SD of the earliest years and cohorts, you're trying to express the contemporaneous black-white difference in terms of the environments in the past, not the environments today. Today, there is less overlap in score distribution because of reduced SD. This reduced SD is closely tied to the diminished score difference. If you want to examine the contemporaneous raw difference, you must use the contemporaneous SD as well. That makes sense.

The choice of the SD is crucial. I have uploaded the new xls file, using my former method and the one preferred by Huang. Using the latter, Huang is correct. There is a larger BW gap closing in both survey year and cohort, although the effect is much stronger with cohort. Using my method, there is (virtually) no gap narrowing in terms of survey year.

For this reason, I added my explanation in the text about why I used within-year pooled SD, instead of the SD for the entire time period (1974-2012).

Also, I want to take this occasion to say that most of the times, the econometricians talk about data "clustering" in the dependent var to justify tobit models. In wordsum, there is no "clustering" but instead what appears to be a "truncation" because the frequency at lower end is extremely low but the frequency is much higher at the higher end, i.e., there is no perfect symmetry. But even in that case, you can (should) use tobit. See below.
http://www.ats.ucla.edu/stat/stata/outpu..._Tobit.htm

Finally...

Chuck, if you think H&H has problem due to confounding with age*BW-d interaction, my analysis would also suffer from this bias, because H&H also controlled for age, even though they used dummy vars of the age variable. Their method and mine are very similar. They have however used more SES variables, such as parental educ, but I didn't include them because I think I have enough, and because, more importantly, the sample size would be greatly reduced. As I said in the article, the main problem with H&H is the low sample size for blacks.

Dalliard, this is off-topic, but if you're not aware of it, I should tell you that your name does not appear in your paper with John, and your reply to Kaplan is still invisible. See here for the explanation.
http://www.openpsych.net/forum/showthrea...46#pid1846

EDIT:

I just detected two mistakes in the new version.

Quote:A next analysis will be conducted using OLS regression in a subsample having low wordsum scores (0-5) and another subsample having high Wordsum scores (5-10).

The first wordsum is not capitalized.

Quote:The tobit model relies heavily on normality and homoscedasticity in the underlying latent variable

The sentence would be clearer if rewritten as "in the underlying latent dependent variable".

I will make the changes in a later version.


Attached Files
.odt   An update on the secular narrowing of the black-white gap in the Wordsum vocabulary test (1974-2012).odt (Size: 94.89 KB / Downloads: 510)
 Reply
#29
(2014-Oct-14, 02:22:25)Meng Hu Wrote: Chuck, if you think H&H has problem due to confounding with age*BW-d interaction, my analysis would also suffer from this bias, because H&H also controlled for age, even though they used dummy vars of the age variable. Their method and mine are very similar.


Yes, your "cohort" analysis is confounded by an age x d interaction. The d-values which you have provided quite clearly show this.
 Reply
#30
(2014-Oct-14, 05:48:45)Chuck Wrote: Yes, your "cohort" analysis is confounded by an age x d interaction. The d-values which you have provided quite clearly show this.


To be clearer: Either (a) explain why I'm wrong about this or (b) briefly acknowledge the issue in your paper.
 Reply
 Previous 1 2 3 4 5 7 Next   
 
 
Forum Jump:

Users browsing this thread: 2 Guest(s)