In reply to Noah's post.

Yes, it is because your method uses the total sample SD, whereas it is customary to use the pooled SD (i.e. the average of the SD within each group weighted by the group sizes). I replicated your number using the total sample SD.

I'd like that you report the normal d value (pooled SD-based). The whole point of using standardized mean differences is to get measures that are comparable across analyses and studies that do not use the same units. As far as I know, it is most common to use the pooled SD and this is the default method unless otherwise specified in statistical programs/languages. Thus, to make your result most comparable with other studies, you should use the pooled SD.

Please include a paragraph/sentence mentioning that you tried using ordered logistic modeling (or whatever you deem appropriate) and that the results were similar. I agree with your experiences.

I tried all the possible OLS models with your predictors. I always do this unless it is impossible because there are too many predictors, e.g. >15; the number of models to try is 2^p-1, so with 6 predictors, it's only 127 models to try. This takes a few seconds even without using parallel computing.

The best model according to BIC was:

Muslim + any death + part of anti-ISIS + gdp + unemploy

which had adjusted R2 of .77. The best model with only one of the intervention predictors has adjusted R2 of .67, so there is a bit of evidence that using multiple intervention predictors is superior.

In general however, this approach tends to overfit models by using too many predictors. One could use lasso regression with cross-validation to get more robust results. I did this for you. Cross-validation has a random component, so I ran it 500 times and summarized the results. The results indicated that all the predictors were useful predictors.

The last row is the proportion of runs that produced a zero coefficient for that predictor, i.e. found it to be useless. As can be seen, the worst predictor was found to be useless only 4.6% of the time. Based of this, I'd tentatively (because of the sample size) conclude that it is best to use all the predictors together.

Noah Carl Wrote:I calculated d-values by simply taking the regression coefficients from OLS models in which the dependent variable (namely, terrorism threat level) had been standardised. In other words, B1 in the following model was used as an estimate of d:

terrorism_threat_level_z-score = alpha + B1(any_deaths_Iraq_Afghanistan)

When computing Cohen's d directly (using the esize command in Stata), I get the same results as Emil. I presume the discrepancies arise from different methods of calculating the pooled standard deviation. I chose to use the OLS method, rather than calculating Cohen's d directly, because the results are more comparable with the conditional standardised differences obtained from the multivariate models (Tables 1-3). I will report the exact d-values in the paper, if preferred.

Yes, it is because your method uses the total sample SD, whereas it is customary to use the pooled SD (i.e. the average of the SD within each group weighted by the group sizes). I replicated your number using the total sample SD.

I'd like that you report the normal d value (pooled SD-based). The whole point of using standardized mean differences is to get measures that are comparable across analyses and studies that do not use the same units. As far as I know, it is most common to use the pooled SD and this is the default method unless otherwise specified in statistical programs/languages. Thus, to make your result most comparable with other studies, you should use the pooled SD.

Noah Carl Wrote:In my experience, logistic and probit regression (binary and ordered) nearly always produce highly similar point estimates (average effects) to OLS, so I prefer to use the latter, given its greater simplicity and ease of interpretation. But I can note in the paper that results were similar when using ordered logistic regression.

Please include a paragraph/sentence mentioning that you tried using ordered logistic modeling (or whatever you deem appropriate) and that the results were similar. I agree with your experiences.

Noah Carl Wrote:My mistake. Would you suggest that I tried utilising an additional variable, namely number of major military interventions in the Middle East, ranging from 0-4?

I tried all the possible OLS models with your predictors. I always do this unless it is impossible because there are too many predictors, e.g. >15; the number of models to try is 2^p-1, so with 6 predictors, it's only 127 models to try. This takes a few seconds even without using parallel computing.

The best model according to BIC was:

Muslim + any death + part of anti-ISIS + gdp + unemploy

which had adjusted R2 of .77. The best model with only one of the intervention predictors has adjusted R2 of .67, so there is a bit of evidence that using multiple intervention predictors is superior.

In general however, this approach tends to overfit models by using too many predictors. One could use lasso regression with cross-validation to get more robust results. I did this for you. Cross-validation has a random component, so I ran it 500 times and summarized the results. The results indicated that all the predictors were useful predictors.

Code:

`muslim15 any_1 deaths2 part_1 gdp_log unemp14 ineq0911`

mean 0.431 0.266 0.107 0.507 0.105 0.228 0.072

median 0.432 0.262 0.108 0.508 0.101 0.225 0.072

sd 0.010 0.072 0.004 0.008 0.041 0.043 0.005

mad 0.004 0.057 0.002 0.003 0.036 0.034 0.002

fraction_zeroNA 0.000 0.014 0.000 0.000 0.046 0.000 0.000

The last row is the proportion of runs that produced a zero coefficient for that predictor, i.e. found it to be useless. As can be seen, the worst predictor was found to be useless only 4.6% of the time. Based of this, I'd tentatively (because of the sample size) conclude that it is best to use all the predictors together.