I carried out a multilevel analysis in MlWin, using the Markov Chain Monte Carlo estimation procedure. The MCMC estimation procedure is used to take into account the crossclassified structure (see attachment).
I calculated the same model in R and Stata. Some of the fixed effects are quite similar, but the effects regarding temporal variation (age and cohort) and parental education differ (as you can see in the table: see attachment).
Does anyone have an explanation?
It is noteworthy that there is a very high correlation between cohort (level2 characteristic) and age (level1 characteristic): r = 0.97 . However, this is normally not a problem for the type of analysis I conducted, that is a hierarchical ageperiodcohort analysis, as carried out by Andrew Bell (Lifecourse and cohort trajectories of mental health in the UK, 19912008 – A multilevel ageperiodcohort analysis)

(a) Stata – command: xtmixed health gender ib0.Parentaleducation c.education##c.cohort c.education##c.age  _all: R.cntry _all: R.cohorts periodcountry:, ml var
(b) R – command: model1_MCMC < MCMCglmm (health ~1 + gender + factor(Parentaleducation) + education + age + cohort + education*cohort + education*age, random = ~ cntry + periodcountry + cohorts, nitt=20000, burnin = 2000, data = data)
MlWin gives different results to Stata/R

 Posts: 11
 Joined: Mon Aug 17, 2015 1:43 pm
MlWin gives different results to Stata/R
 Attachments

 Results.docx
 (15.04 KiB) Downloaded 329 times

 Cross classification.docx
 (16.93 KiB) Downloaded 289 times

 Posts: 1111
 Joined: Mon Oct 19, 2009 10:34 am
Re: MlWin gives different results to Stata/R
Which version of MLwiN are you using (there was a bug in 2.33 that would cause incorrect results with MCMC if the data contained missing values)? Have you checked the MCMC convergence criteria to ensure that the chains are stable and have been run for long enough? Could you provide further details of how the model is set up in MLwiN, for example providing a screenshot of the model along with information on the options that you have set. Alternatively syntax for runmlwin (http://www.bristol.ac.uk/cmm/software/runmlwin/) or R2MLWiN (http://www.bristol.ac.uk/cmm/software/r2mlwin/) would be useful.

 Posts: 11
 Joined: Mon Aug 17, 2015 1:43 pm
Re: MlWin gives different results to Stata/R
Dear Chris,
I’m using version 2.31 of MlWin and the data contained no missing values.
The MCMC convergence criteria were met (ESS > 400 and the trajectories look fine).
I added a screenshot and further information as attachment.
Many thanks,
Katrijn
I’m using version 2.31 of MlWin and the data contained no missing values.
The MCMC convergence criteria were met (ESS > 400 and the trajectories look fine).
I added a screenshot and further information as attachment.
Many thanks,
Katrijn
 Attachments

 Model and additional information.docx
 (79.98 KiB) Downloaded 237 times

 Posts: 16
 Joined: Mon Jun 03, 2013 3:19 pm
Re: MlWin gives different results to Stata/R
Hello Katrijn,
Interesting. I don't have a simple answer I'm afraid, but a few things to check:
Have you centred the variables consistently across the model  might make a difference when the interactions go in if what zero means is different.
The exact zero on the education*age interaction is suspicious to me  if you increase the decimal places, is it still exactly zero? If so, that suggests it is exactly collinear with some of the other variables. Different software often deal with exact multicollinearity differently (although I would have thought R and Stata would give some kind of warning...).
I'd want to know a bit more about the data  at the moment are you modelling it like repeatedcrosssectional data, as opposed to panel data. Are the individuals followed between panels (panel data) or is there a new survey of individuals in each country each year (RCS data)? At present they are modelling the data like the latter; if the former they would need an additional observation level beneath the individual level.
Based on their code, it looks like the R model was only run fro 20k iterations, but the MLwiN model for 200k (or it might just be a typo...) Have they checked the R model for convergence?
That's all the ideas I have at the moment  happy to look at a dataset to see if I can work out the problem. Not used R much so may not be very helpful in that regard though.
Cheers,
Andy
Interesting. I don't have a simple answer I'm afraid, but a few things to check:
Have you centred the variables consistently across the model  might make a difference when the interactions go in if what zero means is different.
The exact zero on the education*age interaction is suspicious to me  if you increase the decimal places, is it still exactly zero? If so, that suggests it is exactly collinear with some of the other variables. Different software often deal with exact multicollinearity differently (although I would have thought R and Stata would give some kind of warning...).
I'd want to know a bit more about the data  at the moment are you modelling it like repeatedcrosssectional data, as opposed to panel data. Are the individuals followed between panels (panel data) or is there a new survey of individuals in each country each year (RCS data)? At present they are modelling the data like the latter; if the former they would need an additional observation level beneath the individual level.
Based on their code, it looks like the R model was only run fro 20k iterations, but the MLwiN model for 200k (or it might just be a typo...) Have they checked the R model for convergence?
That's all the ideas I have at the moment  happy to look at a dataset to see if I can work out the problem. Not used R much so may not be very helpful in that regard though.
Cheers,
Andy