Higher level variation in Logistic multilevel regression

[Frankfeng]

Hi,

I'm running a two-level Logistic multilevel regression: individuals nested within provinces. I study the effects of individual level predictors (i.e age, gender, education) and province level predictors (i.e. GDP, mean income) on self-rated health (binary outcome). I notice that the variations in level 2 (between province) have some strange outcome that I'm not sure how to interpret. From the null model (without any predictors), the variation is 0.252 (SE=0.093); when I only put the province level predictors into the null model (Model 1), the variation decreases to 0.198 (0.075); and then I put the individual level predictors into Model 1, the variation increases to 0.238(0.092).

My question is why the variation increase when I enter the individual level predictors? and how to interpret it?

Thank you!
Frank

[billb]

Hi Frank,
The level 2 variation in a binary response model is on a different scale (the logistic scale) from the response (the probability scale) and it is well known that when one adds
predictors to a model this can have the effect of rescaling parameters so that the underlying binomial assumption remains true - it is therefore not advised to look at changes
in the higher level variation between models as an indication of improved fit and instead one should simply look at the individual predictors and their se and perform a Wald test
or if using MCMC look at the DIC diagnostic.
Best wishes,
Bill.

[shanekav]

I have just encountered this issue also. Can I recommend that Snijders & Bosker 2012 Multilevel Analysis p. 307 provides a good explanation of this. Also, Bauer 2009, 'A note on comparing the estimates of models for cluster-correlated or longitudinal data with binary or ordinal outcomes' provides a method to put the coefficient values on the same scale.

Hope that helps.

Shane