runmlwin with imputed data

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
fonnyyyy
Posts: 13
Joined: Mon Jun 16, 2014 8:54 am

Re: runmlwin with imputed data

Post by fonnyyyy »

The problem lies with the imputation procedure I guess. I now switched to the -ice- command using all variables of my model for imputation. Apparently there are still missings left in the imputed datasets (M=1,...). This is why runmlwin complains on the casewise deletion. I'll try to figure out why not all missing data can be imputed.
fonnyyyy
Posts: 13
Joined: Mon Jun 16, 2014 8:54 am

Re: runmlwin with imputed data

Post by fonnyyyy »

I managed to solve above-mentioned problem. It had something to do with the auxiliary variables used in the imputation procedure and selections I make in the imputed datasets. Apparently, it had nothing to do with the -runmlwin- command. So far I succeeded to estimate a IGLS model using the imputed datasets. However, when using these results as the input for an MCMC estimation for the very same model specification (initsprevious) Stata errors, saying:
The initsprevious option is invalid: No parameters in the previous model match those specified in the current model
an error occurred when mi estimate executed runmlwin on m=1
This suggests that Stata is not storing the coefficients as it normally does. Has anyone come across a similar problem? Would it be a solution to save the results of the IGLS model in a matrix and then use the -initsb- option?
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: runmlwin with imputed data

Post by ChrisCharlton »

The initsprevious option tries to use the parameter estimates for the last model run as starting values. In the case of multiple imputation this is likely to end up being the model for the previously imputed dataset, which is not what you would want. To work around this you can either manually store the relevant matrices and use the initsb/initsv options as you suggest or store the model estimates from a suitable run and use the initsmodel option.
fonnyyyy
Posts: 13
Joined: Mon Jun 16, 2014 8:54 am

Re: runmlwin with imputed data

Post by fonnyyyy »

Thank you very much for this feedback. I could have found this in a response to a similar question: https://www.cmm.bristol.ac.uk/forum/vie ... php?t=1103. Sorry for that.

However, I am running into a next problem now: stata complains that the row vector b of initial values in initsb() is of length 108, while it expects 107. The only difference between the models are the mcmc option and the initsoptions. No variables are omitted or added and selections are the same. Does the MCMC estimation procedure omits parameters compared to the IGLS? Could it be that it has something to do with the OD-values of in the IGLS parameter matrix?
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: runmlwin with imputed data

Post by ChrisCharlton »

Yes, you are correct. For unordered multinomial models there are two OD parameters when estimating with IGLS and only one for MCMC. You can se this below:

Code: Select all

. use "http://www.bristol.ac.uk/cmm/media/runmlwin/bang.dta", clear

. runmlwin use4 cons, ///
>         level1(woman:) ///
>         discrete(distribution(multinomial) link(mlogit) denominator(cons) basecategory(4)) ///
>         nopause
 
MLwiN 3.1 multilevel model                      Number of obs      =      2867
Unordered multinomial logit response model
Estimation algorithm: IGLS, MQL1

----------------------------------
    Contrast | Log-odds
-------------+--------------------
           1 | 1 vs. 4
           2 | 2 vs. 4
           3 | 3 vs. 4
----------------------------------

Run time (seconds)   =       2.59
Number of iterations =          8
------------------------------------------------------------------------------
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
Contrast 1   |
      cons_1 |  -1.741843   .0606988   -28.70   0.000    -1.860811   -1.622876
-------------+----------------------------------------------------------------
Contrast 2   |
      cons_2 |  -1.133654   .0471976   -24.02   0.000     -1.22616   -1.041149
-------------+----------------------------------------------------------------
Contrast 3   |
      cons_3 |   -1.81033   .0625664   -28.93   0.000    -1.932958   -1.687703
------------------------------------------------------------------------------

. matrix list e(b)

e(b)[1,5]
           FP1:        FP2:        FP3:         OD:         OD:
        cons_1      cons_2      cons_3     bcons_1     bcons_2
y1  -1.7418435  -1.1336541  -1.8103304           1           1

. runmlwin use4 con, ///
>         level1(woman:) ///
>         discrete(distribution(multinomial) link(mlogit) denominator(cons) basecategory(4)) ///
>         mcmc(on) initsprevious nopause
 
MLwiN 3.1 multilevel model                      Number of obs      =      2867
Unordered multinomial logit response model
Estimation algorithm: MCMC

----------------------------------
    Contrast | Log-odds
-------------+--------------------
           1 | 1 vs. 4
           2 | 2 vs. 4
           3 | 3 vs. 4
----------------------------------

Burnin                     =        500
Chain                      =       5000
Thinning                   =          1
Run time (seconds)         =       11.8
Deviance (dbar)            =    6242.82
Deviance (thetabar)        =    6239.81
Effective no. of pars (pd) =       3.02
Bayesian DIC               =    6245.84
------------------------------------------------------------------------------
             |      Mean    Std. Dev.     ESS     P       [95% Cred. Interval]
-------------+----------------------------------------------------------------
Contrast 1   |
      cons_1 |  -1.748509   .0631694     1049   0.000    -1.876236   -1.626482
-------------+----------------------------------------------------------------
Contrast 2   |
      cons_2 |  -1.138784   .0489615     1115   0.000    -1.234721   -1.040707
-------------+----------------------------------------------------------------
Contrast 3   |
      cons_3 |  -1.816065   .0641057     1153   0.000    -1.941197   -1.690176
------------------------------------------------------------------------------

. matrix list e(b)

e(b)[1,4]
           FP1:        FP2:        FP3:         OD:
        cons_1      cons_2      cons_3     bcons_1
y1  -1.7485092  -1.1387837  -1.8160653           1
As the matrix provided to initsb() does not need to contain parameter names -runmlwin- matches the starting values by position and therefore has a problem if the number of parameters does not match. If you use initsprevious or initsmodel then it should be able to match by name and discard any parameters that do not exist in the second model (and apply starting values of zero for new parameters).
fonnyyyy
Posts: 13
Joined: Mon Jun 16, 2014 8:54 am

Re: runmlwin with imputed data

Post by fonnyyyy »

Thank you. I solved it by shortening the stored e(b_mi) matrix (i.e. losing the second OD parameter) and using the initsb option. I left out the initsv as it was a bit complicated to edit the e(V_mi) matrix. I learnt from the other forum posts that I should have an eye on the convergence diagnostics then.

Thank you very much for all follow-ups. I hope all models run well now. ;-)
fonnyyyy
Posts: 13
Joined: Mon Jun 16, 2014 8:54 am

Re: runmlwin with imputed data

Post by fonnyyyy »

Ok, the next problem I am running into is that the MCMC estimation procedure does not converge.
model did not converge on m=1
r(498);
Unfortunately this message is not really helpful. I get all estimates in mlwin, but the error pops up when the estimates are returned to stata. It does not initiate the estimation procedure on m=2.
I thought I had to specify the initsv option, which I managed to do, but this doesn't help either.

The next step is having a look at the diagnostics I guess?
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: runmlwin with imputed data

Post by ChrisCharlton »

Are you using a recent version of -runmlwin-? As MCMC converges to a distribution rather than a single value we used to not set the e(converged) value for MCMC models. Due to problems such as the one that you are seeing we later changed this so that is always set to 1 for MCMC models. You can check whether this is the case for your version by running:

Code: Select all

ereturn list
from Stata after running a model with MCMC. The list that is returned should contain e(converged) = 1, which should be what is being used by other commands to determine whether or not the model has converged.
fonnyyyy
Posts: 13
Joined: Mon Jun 16, 2014 8:54 am

Re: runmlwin with imputed data

Post by fonnyyyy »

The list says
e(converged) = 1
. And I am using the 2.30 version. Do you think an update would help? MCMC models on a regular dataset tend to converge though. I'll browse further to find a solution.
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: runmlwin with imputed data

Post by ChrisCharlton »

Sorry, I was referring to the -runmlwin- version, which can be updated with the command:

Code: Select all

. adoupdate runmlwin
From the output you provided it looks like it is already reporting that it has converged, so I don't think this will make a difference however. I am therefore not sure why -mi- is giving that error, although if you could provide a reproducible example that behaves in the same way I could investigate further.
Post Reply