The difference of the results of "mixed" and "runmlwin"

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
umitotakao
Posts: 7
Joined: Fri Jun 07, 2019 3:06 pm

The difference of the results of "mixed" and "runmlwin"

Post by umitotakao »

Hi,

I have been analyzing a set of panel data with the mixed-effects model and noticed that the results are slightly different between "mixed" and "runmlwin".
Generally, the standard errors for the "mixed" are slightly bigger than those of "runmlwin", but the coefficients are also slightly different.

The commands I used are as written below:
mixed depvar [a list of within and between independent variables] || cntrynum: , vce(robust)
runmlwin depvar [a list of within and between independent variables], level2(cntrynum: cons) level1(year: cons) nopause fpsandwich rpsandwich

I would appreciate it if someone could explain the difference. Thank you.
GeorgeLeckie
Site Admin
Posts: 432
Joined: Fri Apr 01, 2011 2:14 pm

Re: The difference of the results of "mixed" and "runmlwin"

Post by GeorgeLeckie »

Dear Umitotakao,

Thank you for your query. I'm afraid I don't have a good answer for you. However, at least in the example below, mixed and runmlwin provide very similar model-based standard errors to one another and equally they provide very similar cluster-robust standard errors to one another. So the discrepancies you mention are rather small in the big scheme of things if that provides reassurance.

Best wishes

George

Code: Select all

. use http://www.bristol.ac.uk/cmm/media/runmlwin/tutorial, clear

. tabulate schgend

     School |
     gender |      Freq.     Percent        Cum.
------------+-----------------------------------
   mixedsch |      2,169       53.44       53.44
     boysch |        513       12.64       66.08
    girlsch |      1,377       33.92      100.00
------------+-----------------------------------
      Total |      4,059      100.00

. generate boyschool = (schgend==2)

. generate girlschool = (schgend==3)

. 
. * Model 1 - runmlwin, model-based standard errors
. runmlwin normexam cons standlrt girl boyschool girlschool, ///
>   level2(school: cons) level1(student: cons) nopause 
 
MLwiN 3.04 multilevel model                     Number of obs      =      4059
Normal response model (hierarchical)
Estimation algorithm: IGLS

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
         school |       65          2       62.4        198
-----------------------------------------------------------

Run time (seconds)   =       1.94
Number of iterations =          4
Log likelihood       = -4662.7132
Deviance             =  9325.4264
------------------------------------------------------------------------------
    normexam |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        cons |  -.1681502   .0539988   -3.11    0.002     -.273986   -.0623144
    standlrt |   .5599642   .0124436   45.00    0.000     .5355753    .5843532
        girl |   .1672281   .0340818    4.91    0.000     .1004289    .2340273
   boyschool |   .1776196   .1107521    1.60    0.109    -.0394504    .3946897
  girlschool |   .1589596   .0872538    1.82    0.068    -.0120547    .3299738
------------------------------------------------------------------------------

------------------------------------------------------------------------------
   Random-effects Parameters |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school              |
                   var(cons) |   .0811054   .0161794      .0493944    .1128164
-----------------------------+------------------------------------------------
Level 1: student             |
                   var(cons) |   .5622733    .012581      .5376149    .5869316
------------------------------------------------------------------------------

. estimates store m1

. 
. * Model 2 - runmlwin, cluster-robust standard errors
. runmlwin normexam cons standlrt girl boyschool girlschool, ///
>   level2(school: cons) level1(student: cons) fpsandwich rpsandwich nopause
 
MLwiN 3.04 multilevel model                     Number of obs      =      4059
Normal response model (hierarchical)
Estimation algorithm: IGLS

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
         school |       65          2       62.4        198
-----------------------------------------------------------

Run time (seconds)   =       0.77
Number of iterations =          4
Log likelihood       = -4662.7132
Deviance             =  9325.4264
------------------------------------------------------------------------------
    normexam |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        cons |  -.1681502   .0543405   -3.09    0.002    -.2746556   -.0616448
    standlrt |   .5599642   .0191179   29.29    0.000     .5224939    .5974346
        girl |   .1672281    .028825    5.80    0.000     .1107321    .2237241
   boyschool |   .1776196    .091065    1.95    0.051    -.0008644    .3561037
  girlschool |   .1589596   .0898003    1.77    0.077    -.0170457    .3349649
------------------------------------------------------------------------------

------------------------------------------------------------------------------
   Random-effects Parameters |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school              |
                   var(cons) |   .0811054   .0163392      .0490811    .1131298
-----------------------------+------------------------------------------------
Level 1: student             |
                   var(cons) |   .5622733   .0194074      .5242355     .600311
------------------------------------------------------------------------------

. estimates store m2

. 
. * Model 3 - mixed, model-based standard errors
. mixed normexam standlrt girl boyschool girlschool || school:

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log likelihood = -4662.7132  
Iteration 1:   log likelihood = -4662.7132  

Computing standard errors:

Mixed-effects ML regression                     Number of obs     =      4,059
Group variable: school                          Number of groups  =         65

                                                Obs per group:
                                                              min =          2
                                                              avg =       62.4
                                                              max =        198

                                                Wald chi2(4)      =    2093.27
Log likelihood = -4662.7132                     Prob > chi2       =     0.0000

------------------------------------------------------------------------------
    normexam |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    standlrt |   .5599641   .0124436    45.00   0.000     .5355752    .5843531
        girl |   .1672282   .0340818     4.91   0.000      .100429    .2340273
   boyschool |   .1776197   .1107533     1.60   0.109    -.0394529    .3946922
  girlschool |   .1589596   .0872548     1.82   0.068    -.0120567    .3299759
       _cons |  -.1681504   .0539994    -3.11   0.002    -.2739873   -.0623134
------------------------------------------------------------------------------

------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
school: Identity             |
                  var(_cons) |   .0811077   .0165468      .0543761    .1209805
-----------------------------+------------------------------------------------
               var(Residual) |   .5622731   .0125854      .5381393    .5874891
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 346.77        Prob >= chibar2 = 0.0000

. estimates store m3

. 
. * Model 4 - mixed, cluster-robust standard errors
. mixed normexam standlrt girl boyschool girlschool || school:, vce(robust)

Performing EM optimization: 

Performing gradient-based optimization: 

Iteration 0:   log pseudolikelihood = -4662.7132  
Iteration 1:   log pseudolikelihood = -4662.7132  

Computing standard errors:

Mixed-effects regression                        Number of obs     =      4,059
Group variable: school                          Number of groups  =         65

                                                Obs per group:
                                                              min =          2
                                                              avg =       62.4
                                                              max =        198

                                                Wald chi2(4)      =     883.35
Log pseudolikelihood = -4662.7132               Prob > chi2       =     0.0000

                                (Std. Err. adjusted for 65 clusters in school)
------------------------------------------------------------------------------
             |               Robust
    normexam |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    standlrt |   .5599641   .0192667    29.06   0.000     .5222021    .5977261
        girl |   .1672282   .0290494     5.76   0.000     .1102925    .2241639
   boyschool |   .1776197   .0917738     1.94   0.053    -.0022538    .3574931
  girlschool |   .1589596   .0904992     1.76   0.079    -.0184156    .3363348
       _cons |  -.1681504   .0547635    -3.07   0.002    -.2754848   -.0608159
------------------------------------------------------------------------------

------------------------------------------------------------------------------
                             |               Robust           
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
school: Identity             |
                  var(_cons) |   .0811077   .0171974      .0535278    .1228978
-----------------------------+------------------------------------------------
               var(Residual) |   .5622731   .0195671       .525201    .6019619
------------------------------------------------------------------------------

. estimates store m4

. 
. * Compare results
. esttab m1 m2 m3 m4, wide se

--------------------------------------------------------------------------------------------------------------------------------
                      (1)                          (2)                          (3)                          (4)                
                                                                           normexam                     normexam                
--------------------------------------------------------------------------------------------------------------------------------
main                                                                                                                            
cons               -0.168**      (0.0540)       -0.168**      (0.0543)                                                          
standlrt            0.560***     (0.0124)        0.560***     (0.0191)        0.560***     (0.0124)        0.560***     (0.0193)
girl                0.167***     (0.0341)        0.167***     (0.0288)        0.167***     (0.0341)        0.167***     (0.0290)
boyschool           0.178         (0.111)        0.178        (0.0911)        0.178         (0.111)        0.178        (0.0918)
girlschool          0.159        (0.0873)        0.159        (0.0898)        0.159        (0.0873)        0.159        (0.0905)
_cons                                                                        -0.168**      (0.0540)       -0.168**      (0.0548)
--------------------------------------------------------------------------------------------------------------------------------
RP2                                                                                                                             
var(cons)          0.0811***     (0.0162)       0.0811***     (0.0163)                                                          
--------------------------------------------------------------------------------------------------------------------------------
RP1                                                                                                                             
var(cons)           0.562***     (0.0126)        0.562***     (0.0194)                                                          
--------------------------------------------------------------------------------------------------------------------------------
lns1_1_1                                                                                                                        
_cons                                                                        -1.256***      (0.102)       -1.256***      (0.106)
--------------------------------------------------------------------------------------------------------------------------------
lnsig_e                                                                                                                         
_cons                                                                        -0.288***     (0.0112)       -0.288***     (0.0174)
--------------------------------------------------------------------------------------------------------------------------------
N                    4059                         4059                         4059                         4059                
--------------------------------------------------------------------------------------------------------------------------------
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001

. 
end of do-file
tonyadams
Posts: 6
Joined: Fri Jun 16, 2023 9:24 am

Re: The difference of the results of "mixed" and "runmlwin"

Post by tonyadams »

The differences in results between the "mixed" command and "runmlwin" can arise due to the default settings and estimation algorithms used by each command may differ or the computational approaches employed by the two commands can also contribute to differences in results.
fnf
Post Reply