Order of variables?

[michaellawton]

Hi,

I have just noticed that the way I order the variables in my multilevel model (repeated measures) can lead to very small but slight adjustments in

1. Log likelihood
2. Parameter estimates (plus std errors)
3. The number of iterations to convergence.

It is a negligible difference but it still bothers me. Should this be happening?

Many thanks
Michael

[GeorgeLeckie]

Hi Michael,

That sounds odd and it would be good to get to the bottom of it.

Could you post the runmlwin output for your model twice, once specified each way so we can take a look at the differences

Best wishes

George

[michaellawton]

Hi here is one model:

Code: Select all

. runmlwin score cons fp1 fp6 ageonset fp1ageonset fp6ageonset  relapse_var  gender , level2(patid: cons fp1 fp6) level1(newids: cons fp1  )  nopause maxiterations(1250)
 
MLwiN 2.24 multilevel model                     Number of obs      =      1872
Normal response model
Estimation algorithm: IGLS

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
          patid |      406          1        4.6         26
-----------------------------------------------------------

Run time (seconds)   =       1.04
Number of iterations =         13
Log likelihood       = -2569.0691
Deviance             =  5138.1382
------------------------------------------------------------------------------
       score |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        cons |   1.323798   .6758441     1.96   0.050    -.0008321    2.648428
         fp1 |   .1227989   .0882338     1.39   0.164    -.0501361    .2957339
         fp6 |  -.4999869   .5871845    -0.85   0.394    -1.650847    .6508735
    ageonset |   .0356616   .0174237     2.05   0.041     .0015117    .0698115
 fp1ageonset |   .0020272   .0028564     0.71   0.478    -.0035714    .0076257
 fp6ageonset |   .0102426   .0178876     0.57   0.567    -.0248164    .0453016
 relapse_var |   .1746091   .0902812     1.93   0.053    -.0023389     .351557
      gender |  -.3471408   .1971454    -1.76   0.078    -.7335387    .0392572
------------------------------------------------------------------------------

------------------------------------------------------------------------------
   Random-effects Parameters |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: patid               |
                   var(cons) |    3.16518   .4899186      2.204958    4.125403
               cov(cons,fp1) |  -.2643705   .0606884     -.3833175   -.1454235
                    var(fp1) |   .0532954   .0116529      .0304561    .0761348
               cov(cons,fp6) |  -.3283667   .4002215     -1.112786    .4560531
                cov(fp1,fp6) |    -.23276    .074663     -.3790967   -.0864232
                    var(fp6) |   2.305654   .5452258      1.237031    3.374277
-----------------------------+------------------------------------------------
Level 1: newids              |
                   var(cons) |   .5237606   .0411513      .4431054    .6044157
               cov(cons,fp1) |  -.0066677   .0024093     -.0113899   -.0019455
                    var(fp1) |   .0001007   .0001186     -.0001317    .0003332
------------------------------------------------------------------------------

Here is another model with the same data but the (independent) variables listed in a different order:

Code: Select all

. runmlwin score cons  ageonset fp1ageonset fp6ageonset fp1 fp6   relapse_var  gender , level2(patid: cons fp1 fp6) level1(newids: cons fp1  )  nopause maxiterations(1250)
 
MLwiN 2.24 multilevel model                     Number of obs      =      1872
Normal response model
Estimation algorithm: IGLS

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
          patid |      406          1        4.6         26
-----------------------------------------------------------

Run time (seconds)   =       0.89
Number of iterations =         11
Log likelihood       = -2569.0776
Deviance             =  5138.1553
------------------------------------------------------------------------------
       score |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        cons |   1.322037   .6758115     1.96   0.050    -.0025289    2.646604
    ageonset |   .0356989   .0174239     2.05   0.040     .0015486    .0698491
 fp1ageonset |   .0020431   .0028527     0.72   0.474     -.003548    .0076343
 fp6ageonset |   .0101395   .0178706     0.57   0.570    -.0248861    .0451652
         fp1 |   .1221076   .0881018     1.39   0.166    -.0505687     .294784
         fp6 |  -.4957939    .586572    -0.85   0.398    -1.645454    .6538662
 relapse_var |   .1746632   .0902913     1.93   0.053    -.0023046     .351631
      gender |  -.3471212   .1971438    -1.76   0.078    -.7335159    .0392735
------------------------------------------------------------------------------

------------------------------------------------------------------------------
   Random-effects Parameters |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: patid               |
                   var(cons) |   3.166923   .4898657      2.206804    4.127042
               cov(cons,fp1) |  -.2643535    .060627     -.3831803   -.1455267
                    var(fp1) |   .0531886   .0116431      .0303685    .0760087
               cov(cons,fp6) |  -.3291243     .40044     -1.113972    .4557237
                cov(fp1,fp6) |    -.23223   .0746553     -.3785518   -.0859082
                    var(fp6) |   2.303365   .5456064      1.233996    3.372734
-----------------------------+------------------------------------------------
Level 1: newids              |
                   var(cons) |    .523794   .0411529      .4431358    .6044522
               cov(cons,fp1) |  -.0066686   .0024094     -.0113908   -.0019463
                    var(fp1) |   .0001007   .0001186     -.0001317    .0003332
------------------------------------------------------------------------------

[ChrisCharlton]

One possible reason for this behaviour is that the two models are converging from different directions and are considered to have converged before reaching exactly the same point.

The default tolerance in MLwiN (and runmlwin) is 10e-2, which corresponds to a 1% change in the value of the estimate from one iteration to the next. If all parameters for an iteration are within this tolerance from the previous iteration then the model is considered to have converged.

You can test whether this is the reason for the behaviour you are seeing by setting the tolerance() estimation option to a higher number and seeing whether the two sets of results become closer.