www.cmm.bristol.ac.uk/forum

Dear all,

I want to compare the model fit of several different specifications of my 3-level logistic regression model fitted by the MCMC algorithm using runmlwin. I know that I have to compare the Bayesian DIC values across the models, but how can I tell whether the difference is statistically significant? In one of the former threads on this forum (https://www.cmm.bristol.ac.uk/forum/vie ... =952#p2486) George Leckie wrote that: "the DIC statistics are effectively the same, they differ by less than 1 point (see below). Differences of 5 or more are typically taken as meaningful differences in model fit". I wonder whether there is any other, more accurate test for significance of a difference in Bayesian DIC other than this rule of thumb than can be easily implemented in Stata/runmlwin? Thanks in advance for your advice.

Regards,

Zuzanna

Hi Zuzanna,
The DIC is an information criterion like the AIC and BIC in classical statistics and basically such quantities do not have an underlying statistical distribution and so the concept of significance doesn't work for such measures. It is therefore just the case that rules of thumb are used - clearly a smaller DIC is better but due to the stochastic nature of an MCMC algorithm there will be some noise in the values hence the comment by George.
Hope this helps,
Bill.

Dear Bill,

Thanks a lot for the clarification.

Best regards,
Zuzanna