3-level multinomial error:'illegal op: v_block:: mult_add()'

[julia1633]

Dear George,

Thanks for all your extremely useful suggestions. I will try to start from a simple model and then build up a more complex one based on initial values from the previous one. I will also experiment with subsamples. Will be in touch once I get some progress on this.

Kind Regards,
Julia

[julia1633]

Greetings again,

I've been making some progress with estimation of our 3-level model. The results based on using IGLS, MQL1 appear to be good. However, once I fit the model using PQL2 based on the initial values from the previous model (mql1) I got the results (which are good but slightly different form mql1 which is expected) but odd enough beneath the table there appears a message 'WARNING: IGLS algorithm failed to converge. Increase the number of iterations'. I've thought if the convergence fails, no results would be reported or there would appear either dots or odd values instead of estimates. Could you please clarify why I keep getting this message. Once I increase the number of iterations to 30 or 40 I either got odd estimates or all sort of wrror messages that looks like MLwin is running out of memory.

here is the syntax I am using.

global MLwiN_path "C:\Program Files (x86)\MLwiN v2.23\mlwin.exe"

runmlwin SUYR5JOBml2 cons age age_sq male educ_secpost educ_post busang_dum omestb_dum knowent_dum fearfail_dum l3r_hfgov l4_exconst l2_finfree l_dgdp l_gdp_pc_ppp, level3(country: cons) level2(year: cons) level1(obsno) discrete(distribution(multinomial) link(mlogit) denom(cons) base(1))

runmlwin SUYR5JOBml2 cons age age_sq male educ_secpost educ_post busang_dum omestb_dum knowent_dum fearfail_dum l3r_hfgov l4_exconst l2_finfree l_dgdp l_gdp_pc_ppp, level3(country: cons) level2(year: cons) level1(obsno) discrete(distribution(multinomial) link(mlogit) denom(cons) base(1) pql2) initsprevious maxiterations(30)

I would appreciate your advice and more specifically whether I can rely on the results of pql2 regardless the message of failed convergence.

Many thanks,
Julia

[GeorgeLeckie]

Fundamentally you have a very large data set (700,000 individuals) and an unordered categorical response variable. MLwiN and other statistical packages are always going to struggle to fit a three-level model to these data. I think you will have to work with a smaller sub sample of these data, at least while you are exploring. When you decide on the final models for your paper, you can always then try to use the whole data.

If you type help runmlwin##quasilikelihood_estimates in the Stata viewer, you will see some comments on MQL1 and PQL2

mql1, mql2, pql1 and pql2 specify the linearization technique for fitting discrete response models by
(R)IGLS. All four quasilikelihood methods are approximate: pql2 is the most accurate but the least stable
and the slowest to converge, mql1 is the least accurate but the most stable and fastest to converge. We
recommend that model exploration is conducted using mql1. For any final model, we recommend fitting the
model using pql2 (or preferably MCMC) as a two stage process. First fit the model using mql1, then refit
the model using pql2 where the initsprevious option (or initsmodel(name) or initsb(matrix)) is specified to
use the mql1 parameter estimates as the starting values for fitting the model using pql2.

So while PQL2 gives less biases estimates than MQL1 it is rather flakey and the algorithm may well fail to converge at all. If after increasing the maximum number of iterations, MLwiN still fails to converge then one alternative is to use the MCMC engine where you use the MQL1 parameter estimates as starting values for the MCMC chains. MCMC does not suffer from the same biases as the quasilikelihood methods. However, MCMC is computationally very intensive and will also be very slow to fit models to such a large data set.

Yes we do report the results even if the IGLS algorithm fails to converge, but we do so with a warning message to this effect. This is no different from Stata's offical xtmixed, xtmelogit and xtmepoisson commands where you can specify estimation options which set a maximum number of iterations. These commands will also report the estimation results at the point where you hit this maximum number of iterations even though the model hasn't fully converged. No you shouldn't report results from models which have not converged in any final analysis.

Best wishes

George

[julia1633]

Dear George,

Thanks for your advice. In fact, I followed your earlier suggestions regarding starting with fitting the most simple model and moving to the most complex one each time using estimates from the previous model as initial values for the consequent one. I identified the specification which I'd like to use to fit the final model. And it works fine with mql1. However, the problems occur when I switch to pql2; igls algorithm fails to converge. In fact, I also tried MCMC but kept getting a message 'Obey command is not available when batch excecution is PAUsed' that prevented the model running. I appreciate your advice with estimating the model on a subsample, but as you point out it makes sense at the exploring stage, but not when trying to fit the final model. I haven't seen any papers published yet which would use a randomly generated subsample for obtaining their results. It would even look more odd if we refer to our publication using the same dataset for estimating a multilevel model with a continuous dependent variable denoting growth expectations in young firms using xtmixed in STATA. It is close to what we are doing now, but now we try to look at nascent firms' employment generation expectations and this unordered categorical depednent variable works better in this case. Following this could you please advise on two things. (1) Could you please clarify if the message above that I am getting with using MCMC is anything to do with a wrong syntax that I may be using or it is attributed to something else. (2) Could you please confirm whether the only visible solution for my problems is just to work with a subsample. Hope all this makes sense to you. I look forward to hearing from you. thanks, julia
----
runmlwin SUYR5JOBml2 cons age age_sq male educ_secpost educ_post busang_dum omestb_dum knowent_dum fearfail_dum l3r_hfgov l4_exconst l2_finfree l_dgdp l_gdp_pc_ppp, level3(country: cons) level2(year: cons) level1(obsno) discrete(distribution(multinomial) link(mlogit) denom(cons) base(1)) mcmc(on) initsprevious

[GeorgeLeckie]

You have ~700,000 individuals, and you want to fit a three-level three-category multinomial response model. The expanded data is therefore ~1,400,000. My gut feeling is that this is too large for MLwiN (or other packages) to handle and that you should work with a random sub-sample of your data. Many papers do perform their analyses on random samples of their data when their data are infeasibly large. I understand, however, that one of your covariates of interest is rare. Ideally you would therefore do a stratified random sample where and then use sampling weights, but this functionality is only implemented in MLwiN for continuous response variables. So yes, you are in a difficult situation.

[ChrisCharlton]

At what point did you get the message 'Obey command is not available when batch excecution is PAUsed'? I would only expect this message to occur if you tried to run another macro while MLwiN was waiting for you to choose the "resume macro" or "abort macro" buttons. If you are doing nothing more than running the macro generated by runmlwin could you use the viewfullmacro option so I can try to work out where this extra command is coming from?

[julia1633]

Thanks a lot. Chris, you were right about the reason of the error message regarding MCMC. Apparently, I've got my results using MCMC on a whole sample after 20 hrs of model running. The model has converged and the results look really good - it's what we've been expecting . So, I would like to thank both of you for your invaluable help and guidance throughout my encounter with runmlwin! Hope you don't mind if I use Forum again should I experience any problems with using MlWin in the future. Many thanks once again. Julia

[GeorgeLeckie]

You're very welcome,

Glad runmlwin is now working for you.

George