Specifying orthogonal in Hierarchical Centring decreases ESS

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
KazimovHH
Posts: 14
Joined: Mon Apr 09, 2018 11:56 am

Specifying orthogonal in Hierarchical Centring decreases ESS

Post by KazimovHH »

Dear Community,

I have specified the orthogonal option in discrete models based on Browne's (2012) recommendation.

In order to increase Effective Sample Sizes of the contextual variable in the contextual-effects model, I have added hierarchical centering in MCMC as follows:

Code: Select all

* Run the model
	quietly runmlwin vote cons $controlVars Inteff_c Exteff_c dependency_c, level2(cntry_n: cons) level1(ind:) ///
	discrete(distribution(binomial) link(logit) denom(denomb) pql2)
* Define Binomial hierarchical centering algorithm
	matrix b = e(b)
	matrix V = e(V)
		
	runmlwin vote cons $controlVars Inteff_c Exteff_c dependency_c, level2(cntry_n: cons) level1(ind:) ///
	discrete(distribution(binomial) link(logit) denom(denomb)) ///
	mcmc(orth hcen(2) seed(1) burnin(1000) chain(10000)) or initsb(b) initsv(V) nopause nogroup
After running the model, I diagnose the MCMC trajectories, the context variable is not converged (small size of ESS = 33):
With Orthogonal
With Orthogonal
orthogonal.png (310.25 KiB) Viewed 5339 times
Subsequently, I have removed the orthogonal option and the ESS of the parameter increased (8221) and the parameter was converged:
Without Orthogonal
Without Orthogonal
orthogonal.png (307.39 KiB) Viewed 5339 times
I would greatly appreciate if you address these questions:

1. Is it advisable to remove orthogonal option in the discrete model when hierarchical centering is specified?

2. If I remove the orthogonal option from three out of eight discrete models, is Bayesian DIC still applicable to test hypotheses across all models?

3. In MLwiN MCMC Manual (25.4 Binomial example in practice), ESS is increasing with better convergence when the orthogonal option is specified after hierarchical centering. In my case, ESS is decreasing when orthogonal specified. What may be the reason(s) behind this?

Looking forward,
Regards
billb
Posts: 157
Joined: Fri May 21, 2010 1:21 pm

Re: Specifying orthogonal in Hierarchical Centring decreases ESS

Post by billb »

Hi KazimovHH,
Thanks for the email. Hard to know what the right answer here as you have 2 techniques (Orthogonal parameterisation and Hierarchical centering) which do different things and can both improve mixing for particular model features so the answer will depend on your particular dataset.
Orthogonal parameterisation improves things when you have correlated predictor variables as it removes that correlation fitting the model to a transformed set of predictors and then transforming back the estimates that appear.
Hierarchical centering will improve the mixing of higher level predictors at the level you are centering as in effect it pushes the estimation of these predictor variables up the hierarchy and in a non-normal model this has the additional advantages of being both quicker and also using a Normal Gibbs update rather than the MH used for the binary response.
One challenge of combining the procedures is that when you orthogonalise the transformed variables will generally no longer be higher level variables as they will be a transformation that combines variables and so this might remove the benefit of hierarchical centering. One thing to try is to put all the level 2 variables into the model first as from memory the order of transformation will be the same as the order of the variables. This might mean that more of the transformed variables stay at the higher level and thus have improved mixing.
Let me know if this helps,
Best wishes,
Bill.
KazimovHH
Posts: 14
Joined: Mon Apr 09, 2018 11:56 am

Re: Specifying orthogonal in Hierarchical Centring decreases ESS

Post by KazimovHH »

Dear Bill,

Your suggestion to put L2 variables first in the model worked very well.
Thank you very much for the suggestion and help.

Best Regards,
Rza
Post Reply