Problem with weights in the latest version of runmlwin

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
ewancarr
Posts: 18
Joined: Fri Feb 04, 2011 12:46 pm

Problem with weights in the latest version of runmlwin

Post by ewancarr »

Hi again,

I'm having problems applying weights to my models, and it seems this has been caused by recent updates (old models, that used to run fine, produce errors with the latest version(s)). For example, the following model...

Code: Select all

runmlwin affect cons, ///
         level3(country: cons) level2(wave: cons) level1(pid: , ) ///
         discrete(distribution(binomial) link(logit) denominator(cons) pql2) ///
         nopause
...runs fine. However, if I add weights at level-1 (with the option "weightvar"), I get error messages:

Code: Select all

runmlwin affect cons, ///
         level3(country: cons) level2(wave: cons) level1(pid: , weightvar(weight)) ///
         discrete(distribution(binomial) link(logit) denominator(cons) pql2) ///
         nopause
Cannot specify random part options at level 1 unless random part predictor variables are specified
r(198);

This is strange because I'm getting this error even when running old models/data (that ran perfectly several months ago).
  • Is there something obvious I'm missing? (in the above syntax, for example?)
  • Did anything change in recent versions of MLwiN/runmlwin with regard to how weights are handled?
  • Any other ideas?
Many thanks,

Ewan
--
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Problem with weights in the latest version of runmlwin

Post by ChrisCharlton »

You can turn off this error message by searching for the following text in runmlwin.ado:

Code: Select all

			if `numrp`l'vars' == 0 {
				if "`diagonal'"~="" || "`elements'"~="" || "`reset'"~="" || "`residuals'"~="" || "`weightvar'"~="" || "`mmids'"~="" || "`mmweights'"~="" || "`carids'"~="" || "`carweights'"~="" || "`parexpansion'"~="" {
					display as error "Cannot specify random part options at level `l' unless random part predictor variables are specified"
					exit 198
				}
			}
and removing

Code: Select all

|| "`weightvar'"~=""
If your model fits correctly with this change we will work out what the correct check should be.
ewancarr
Posts: 18
Joined: Fri Feb 04, 2011 12:46 pm

Re: Problem with weights in the latest version of runmlwin

Post by ewancarr »

OK, that seems to have worked. Stata did initially complain that "weight is held to more precision than MLwiN can handle" and that I should use " recast float weight, force" to change from double to float. Having recast the weight variable the model ran fine.

The results:

Code: Select all

----------------------------------------
    Variable |  No weight     Weighted 
-------------+--------------------------
FP1          |
        cons |  -.3706347   -.36349323  
-------------+--------------------------
RP3          |
   var(cons) |   .0031633    .00220827  
-------------+--------------------------
RP2          |
   var(cons) |  .28862122    .29177937  
-------------+--------------------------
RP1          |
var(bcons_1) |          1            1  
----------------------------------------
Thanks!

Ewan
--
GeorgeLeckie
Site Admin
Posts: 432
Joined: Fri Apr 01, 2011 2:14 pm

Re: Problem with weights in the latest version of runmlwin

Post by GeorgeLeckie »

Hi Ewan that is great,

Two words of caution

(1) Ideally you should fit final models by MCMC as quasilikelihood methods are biased and in some situations these biases can be large. However, unfortunately sampling weights are not implemented in MCMC. So if it is important to weight your analyses then you should check that PQL2 gives similar results to MCMC when doing an unweighted analysis.

(2) My colleagues have done thorough testing of sampling weights for continuous response variables in MLwiN,

http://www.bristol.ac.uk/cmm/software/s ... hatweights

but there isn't much information out there for using sampling weights with discrete response models in MLwiN. It would therefore be wise to compare your answers to those provided by another package. You could , for example, do this for a small sample of your data and compare with -gllamm-.

Best wishes

George
ewancarr
Posts: 18
Joined: Fri Feb 04, 2011 12:46 pm

Re: Problem with weights in the latest version of runmlwin

Post by ewancarr »

Thanks!
(1) Ideally you should fit final models by MCMC
I do, and my data does require weights, so the inability to use weights in MCMC has always been a concern. I tend to report both the weighted PQL2 and unweighted MCMC results, and generally, there isn't much variation in results.

If the results were different, however, what could I do? PQL2 allows weights but is biased; MCMC isn't quasilikelihood, so is less biased, but doesn't allow weights (so is biased...!). It seems there isn't an optimal solution to this yet. I guess the question is: which source of bias is more problematic: bias from PQL2 or bias from unweighted analyses?
You could , for example, do this for a small sample of your data and compare with -gllamm-.
That's a good idea. Am I right in thinking that xtmelogit doesn't allow weights? (I'm more familiar with the xt syntax than gllamm).

Many thanks,

Ewan
--
GeorgeLeckie
Site Admin
Posts: 432
Joined: Fri Apr 01, 2011 2:14 pm

Re: Problem with weights in the latest version of runmlwin

Post by GeorgeLeckie »

Hi Ewan,

That's great you clearly understand all the issues of PQL2 vs. MCMC for discrete response models with sampling weights

xtmixed allows sampling weights (Stata 12 onwards), xtmelogit does not

George
tulshisaha
Posts: 1
Joined: Fri Jun 19, 2020 9:21 pm

Re: Problem with weights in the latest version of runmlwin

Post by tulshisaha »

Hi I have similar problem when I use weight both at level 1 and level 2 level. And getting the following error:
. quietly runmlwin pyaud5 cons liquorden, ///
> level2(FIPS_STATE: cons, weightvar(MLM_WEIGHT_STATE_SCALED)) ///
> level1(idnum:, weightvar(audweight)) ///
> discrete(distribution(binomial) link(logit) denom(cons) pql2) ///
> pause

.
. * Fit the previous model by MCMC for a burnin of 1000 iterations followed
. runmlwin pyaud5 cons liquorden, ///
> level2(FIPS_STATE: cons, residuals(u) weightvar(MLM_WEIGHT_STATE_SCALED)) ///
> level1(idnum:,weightvar(audweight)) ///
> discrete(distribution(binomial) link(logit) denom(cons)) ///
> mcmc(burnin(1000) chain(10000)) initsprevious nopause nogroup

Weights are only valid for univariate models estimated using (R)IGLS
r(198);


end of do-file


r(198);
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Problem with weights in the latest version of runmlwin

Post by ChrisCharlton »

Unfortunately the MCMC algorithm implemented in MLwiN does not support weighting, so if you want to fit these models you will need to stick to using (R)IGLS estimation. Note also that weighting for discrete models in MLwiN has not been thoroughly tested (see http://www.bristol.ac.uk/cmm/software/s ... ights.html).
Post Reply