MlWin gives different results to Stata/R

Welcome to the forum for MLwiN users. Feel free to post your question about MLwiN software here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Remember to check out our extensive software FAQs which may answer your question: http://www.bristol.ac.uk/cmm/software/s ... port-faqs/
Post Reply
KatrijnDelar
Posts: 11
Joined: Mon Aug 17, 2015 1:43 pm

MlWin gives different results to Stata/R

Post by KatrijnDelar » Mon Aug 17, 2015 5:03 pm

I carried out a multilevel analysis in MlWin, using the Markov Chain Monte Carlo estimation procedure. The MCMC estimation procedure is used to take into account the cross-classified structure (see attachment).

I calculated the same model in R and Stata. Some of the fixed effects are quite similar, but the effects regarding temporal variation (age and cohort) and parental education differ (as you can see in the table: see attachment).

Does anyone have an explanation?
It is noteworthy that there is a very high correlation between cohort (level-2 characteristic) and age (level-1 characteristic): r = 0.97 . However, this is normally not a problem for the type of analysis I conducted, that is a hierarchical age-period-cohort analysis, as carried out by Andrew Bell (Life-course and cohort trajectories of mental health in the UK, 1991-2008 – A multilevel age-period-cohort analysis)

----------


(a) Stata – command: xtmixed health gender ib0.Parentaleducation c.education##c.cohort c.education##c.age || _all: R.cntry|| _all: R.cohorts|| periodcountry:, ml var
(b) R – command: model1_MCMC <- MCMCglmm (health ~1 + gender + factor(Parentaleducation) + education + age + cohort + education*cohort + education*age, random = ~ cntry + periodcountry + cohorts, nitt=20000, burnin = 2000, data = data)
Attachments
Results.docx
(15.04 KiB) Downloaded 329 times
Cross classification.docx
(16.93 KiB) Downloaded 289 times

ChrisCharlton
Posts: 1111
Joined: Mon Oct 19, 2009 10:34 am

Re: MlWin gives different results to Stata/R

Post by ChrisCharlton » Tue Aug 18, 2015 9:02 am

Which version of MLwiN are you using (there was a bug in 2.33 that would cause incorrect results with MCMC if the data contained missing values)? Have you checked the MCMC convergence criteria to ensure that the chains are stable and have been run for long enough? Could you provide further details of how the model is set up in MLwiN, for example providing a screenshot of the model along with information on the options that you have set. Alternatively syntax for runmlwin (http://www.bristol.ac.uk/cmm/software/runmlwin/) or R2MLWiN (http://www.bristol.ac.uk/cmm/software/r2mlwin/) would be useful.

KatrijnDelar
Posts: 11
Joined: Mon Aug 17, 2015 1:43 pm

Re: MlWin gives different results to Stata/R

Post by KatrijnDelar » Tue Aug 18, 2015 9:47 am

Dear Chris,

I’m using version 2.31 of MlWin and the data contained no missing values.
The MCMC convergence criteria were met (ESS > 400 and the trajectories look fine).
I added a screenshot and further information as attachment.

Many thanks,
Katrijn
Attachments
Model and additional information.docx
(79.98 KiB) Downloaded 237 times

andrewjdbell
Posts: 16
Joined: Mon Jun 03, 2013 3:19 pm

Re: MlWin gives different results to Stata/R

Post by andrewjdbell » Tue Aug 18, 2015 12:30 pm

Hello Katrijn,

Interesting. I don't have a simple answer I'm afraid, but a few things to check:

Have you centred the variables consistently across the model - might make a difference when the interactions go in if what zero means is different.

The exact zero on the education*age interaction is suspicious to me - if you increase the decimal places, is it still exactly zero? If so, that suggests it is exactly collinear with some of the other variables. Different software often deal with exact multicollinearity differently (although I would have thought R and Stata would give some kind of warning...).

I'd want to know a bit more about the data - at the moment are you modelling it like repeated-cross-sectional data, as opposed to panel data. Are the individuals followed between panels (panel data) or is there a new survey of individuals in each country each year (RCS data)? At present they are modelling the data like the latter; if the former they would need an additional observation level beneath the individual level.

Based on their code, it looks like the R model was only run fro 20k iterations, but the MLwiN model for 200k (or it might just be a typo...) Have they checked the R model for convergence?

That's all the ideas I have at the moment - happy to look at a dataset to see if I can work out the problem. Not used R much so may not be very helpful in that regard though.

Cheers,
Andy

Post Reply