2016-Apr-09, 22:57:37

1) "The structure of the survey is as following:"

... was as follows:

2) Likert should be capitalized.

3) "There are several measures of inter-rater consistency. Perhaps the simplest is to calculate the mean correlation between raters. Figure 1 shows the distribution of rater intercorrelations."

No explanation is given what is being correlated here.

4) "One method is to correlate the estimates with the real values (Jussim, 2012, p. 205)."

I would clarify, "... to correlate participants' estimates of group values with the real group values..."

5) "Using the Pearson correlation as the measure of accuracy, the raters' accuracy scores are .78 and .89, and their inter-correlation is .51 (individual-level)."

Isn't that .51 just the inter-rater correlation which tells us nothing about accuracy? What does it do there? Very confusing. Also I'd use values in that example that yield accuracy scores more similar to those in the real data (.78 and .89 are much higher than real accuracy scores).

6) "Figure 2 shows the distribution of Pearson correlations."

Clarify, e.g., "... of Pearson correlations, each data point being a correlation between a participant's estimates of group values and the real group values." (Assuming I get this right.)

7) Regarding Table 2, why do you report correlations between individual accuracy measures rather than the accuracy measures themselves? I would merge section 5.2 to the beginning of section 5 so that at least mean/median individual accuracy measures would be presented before correlations between them.

8) "(systematic error; (Jensen, 1980))"

(systematic error; Jensen, 1980)

9) "Because the population of Danish was an extreme outlier"

the native Danish population

10) "In reviewing the paper (http://openpsych.net/forum/showthread.ph...88#pid3888), Peter Frost (http://evoandproud.blogspot.dk/)"

I don't think there's a need to link to the forum comment. At least the link should be placed in a footnote. There's no reason to link to Frost's blog.

11) "Rarely is it considered that they instead reflect group differences"

... that they may instead reflect genuine group differences

... was as follows:

2) Likert should be capitalized.

3) "There are several measures of inter-rater consistency. Perhaps the simplest is to calculate the mean correlation between raters. Figure 1 shows the distribution of rater intercorrelations."

No explanation is given what is being correlated here.

4) "One method is to correlate the estimates with the real values (Jussim, 2012, p. 205)."

I would clarify, "... to correlate participants' estimates of group values with the real group values..."

5) "Using the Pearson correlation as the measure of accuracy, the raters' accuracy scores are .78 and .89, and their inter-correlation is .51 (individual-level)."

Isn't that .51 just the inter-rater correlation which tells us nothing about accuracy? What does it do there? Very confusing. Also I'd use values in that example that yield accuracy scores more similar to those in the real data (.78 and .89 are much higher than real accuracy scores).

6) "Figure 2 shows the distribution of Pearson correlations."

Clarify, e.g., "... of Pearson correlations, each data point being a correlation between a participant's estimates of group values and the real group values." (Assuming I get this right.)

7) Regarding Table 2, why do you report correlations between individual accuracy measures rather than the accuracy measures themselves? I would merge section 5.2 to the beginning of section 5 so that at least mean/median individual accuracy measures would be presented before correlations between them.

8) "(systematic error; (Jensen, 1980))"

(systematic error; Jensen, 1980)

9) "Because the population of Danish was an extreme outlier"

the native Danish population

10) "In reviewing the paper (http://openpsych.net/forum/showthread.ph...88#pid3888), Peter Frost (http://evoandproud.blogspot.dk/)"

I don't think there's a need to link to the forum comment. At least the link should be placed in a footnote. There's no reason to link to Frost's blog.

11) "Rarely is it considered that they instead reflect group differences"

... that they may instead reflect genuine group differences