Hello all,
I need to interpret a significant p-value that has a confidence interval (CI) for the estimate that crosses 0 (see below).
Usually, I place more importance on CI.
Knowing that MCMC was conducted in this model, should I place more importance on the CI or the p-value?
MLwiN (version: 2.36) multilevel model (Binomial)
N min mean max N_complete min_complete mean_complete max_complete
iID 60 1 23.05 77 55 1 20.36364 74
Estimation algorithm: MCMC Elapsed time : 12.08s
Number of obs: 1120 (from total 1383) Number of iter.: 5000 Chains: 1 Burn-in: 500
Bayesian Deviance Information Criterion (DIC)
Dbar D(thetabar) pD DIC
1415.712 1401.830 13.881 1429.593
Coef. Std. Err. z Pr(>|z|) [95% Cred. Interval] ESS
Intercept 0.79868 0.35035 2.28 0.02263 * 0.16612 1.50846 28
...
MalnutritionPYes 0.31611 0.16009 1.97 0.04832 * -0.01020 0.62371 663
...
Thank you so much!
Adrian
Confidence Intervals vs. Significant p-values
-
- Posts: 61
- Joined: Fri Oct 23, 2009 1:49 pm
Re: Confidence Intervals vs. Significant p-values
Note that one-sided Bayesian p-values are another option which can be helpful; these can be toggled on/off in R2MLwiN via z.ratio. There is an example in Section 6 of the R2MLwiN article (http://dx.doi.org/10.18637/jss.v072.i10) recently published in the Journal of Statistical Software; as the article states, these return the proportion of chain iterations which are of the opposite sign to the chain mean (the coefficient estimate). I.e. they use the same information as the credible intervals (insofar as they both use the values of the chain iterations), but just emphasise a different element of that information.
E.g.:
...would yield the following output (truncated):
So the proportion of chain iterations which were below zero for the Intercept was 0.444, whilst none of the 5000 iterations were below zero for standlrt.
Compare this with (note that z.ratio = TRUE is actually the default here, but just including it to be explicit):
...which would yield (again, truncated):
Best wishes,
Richard
E.g.:
Code: Select all
library(R2MLwiN)
data("tutorial")
F3 <- normexam ~ 1 + standlrt + (1 | school) + (1 | student)
print(standlrtMCMC <- runMLwiN(Formula = F3, data = tutorial, estoptions = list(EstM = 1)), z.ratio = FALSE)
Code: Select all
The fixed part estimates:
Coef. Std. Err. pMCMC(1-sided) [95% Cred. Interval] ESS
Intercept 0.00514 0.04246 0.444 -0.07933 0.08896 211
standlrt 0.56321 0.01250 0 0.53862 0.58791 4060
Compare this with (note that z.ratio = TRUE is actually the default here, but just including it to be explicit):
Code: Select all
print(standlrtMCMC, z.ratio = TRUE)
Code: Select all
The fixed part estimates:
Coef. Std. Err. z Pr(>|z|) [95% Cred. Interval] ESS
Intercept 0.00514 0.04246 0.12 0.9036 -0.07933 0.08896 211
standlrt 0.56321 0.01250 45.05 0 *** 0.53862 0.58791 4060
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Richard