Hi,
I have been analyzing a set of panel data with the mixed-effects model and noticed that the results are slightly different between "mixed" and "runmlwin".
Generally, the standard errors for the "mixed" are slightly bigger than those of "runmlwin", but the coefficients are also slightly different.
The commands I used are as written below:
mixed depvar [a list of within and between independent variables] || cntrynum: , vce(robust)
runmlwin depvar [a list of within and between independent variables], level2(cntrynum: cons) level1(year: cons) nopause fpsandwich rpsandwich
I would appreciate it if someone could explain the difference. Thank you.
The difference of the results of "mixed" and "runmlwin"
-
umitotakao
- Posts: 7
- Joined: Fri Jun 07, 2019 3:06 pm
-
GeorgeLeckie
- Site Admin
- Posts: 432
- Joined: Fri Apr 01, 2011 2:14 pm
Re: The difference of the results of "mixed" and "runmlwin"
Dear Umitotakao,
Thank you for your query. I'm afraid I don't have a good answer for you. However, at least in the example below, mixed and runmlwin provide very similar model-based standard errors to one another and equally they provide very similar cluster-robust standard errors to one another. So the discrepancies you mention are rather small in the big scheme of things if that provides reassurance.
Best wishes
George
Thank you for your query. I'm afraid I don't have a good answer for you. However, at least in the example below, mixed and runmlwin provide very similar model-based standard errors to one another and equally they provide very similar cluster-robust standard errors to one another. So the discrepancies you mention are rather small in the big scheme of things if that provides reassurance.
Best wishes
George
Code: Select all
. use http://www.bristol.ac.uk/cmm/media/runmlwin/tutorial, clear
. tabulate schgend
School |
gender | Freq. Percent Cum.
------------+-----------------------------------
mixedsch | 2,169 53.44 53.44
boysch | 513 12.64 66.08
girlsch | 1,377 33.92 100.00
------------+-----------------------------------
Total | 4,059 100.00
. generate boyschool = (schgend==2)
. generate girlschool = (schgend==3)
.
. * Model 1 - runmlwin, model-based standard errors
. runmlwin normexam cons standlrt girl boyschool girlschool, ///
> level2(school: cons) level1(student: cons) nopause
MLwiN 3.04 multilevel model Number of obs = 4059
Normal response model (hierarchical)
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
school | 65 2 62.4 198
-----------------------------------------------------------
Run time (seconds) = 1.94
Number of iterations = 4
Log likelihood = -4662.7132
Deviance = 9325.4264
------------------------------------------------------------------------------
normexam | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | -.1681502 .0539988 -3.11 0.002 -.273986 -.0623144
standlrt | .5599642 .0124436 45.00 0.000 .5355753 .5843532
girl | .1672281 .0340818 4.91 0.000 .1004289 .2340273
boyschool | .1776196 .1107521 1.60 0.109 -.0394504 .3946897
girlschool | .1589596 .0872538 1.82 0.068 -.0120547 .3299738
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school |
var(cons) | .0811054 .0161794 .0493944 .1128164
-----------------------------+------------------------------------------------
Level 1: student |
var(cons) | .5622733 .012581 .5376149 .5869316
------------------------------------------------------------------------------
. estimates store m1
.
. * Model 2 - runmlwin, cluster-robust standard errors
. runmlwin normexam cons standlrt girl boyschool girlschool, ///
> level2(school: cons) level1(student: cons) fpsandwich rpsandwich nopause
MLwiN 3.04 multilevel model Number of obs = 4059
Normal response model (hierarchical)
Estimation algorithm: IGLS
-----------------------------------------------------------
| No. of Observations per Group
Level Variable | Groups Minimum Average Maximum
----------------+------------------------------------------
school | 65 2 62.4 198
-----------------------------------------------------------
Run time (seconds) = 0.77
Number of iterations = 4
Log likelihood = -4662.7132
Deviance = 9325.4264
------------------------------------------------------------------------------
normexam | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
cons | -.1681502 .0543405 -3.09 0.002 -.2746556 -.0616448
standlrt | .5599642 .0191179 29.29 0.000 .5224939 .5974346
girl | .1672281 .028825 5.80 0.000 .1107321 .2237241
boyschool | .1776196 .091065 1.95 0.051 -.0008644 .3561037
girlschool | .1589596 .0898003 1.77 0.077 -.0170457 .3349649
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
Level 2: school |
var(cons) | .0811054 .0163392 .0490811 .1131298
-----------------------------+------------------------------------------------
Level 1: student |
var(cons) | .5622733 .0194074 .5242355 .600311
------------------------------------------------------------------------------
. estimates store m2
.
. * Model 3 - mixed, model-based standard errors
. mixed normexam standlrt girl boyschool girlschool || school:
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0: log likelihood = -4662.7132
Iteration 1: log likelihood = -4662.7132
Computing standard errors:
Mixed-effects ML regression Number of obs = 4,059
Group variable: school Number of groups = 65
Obs per group:
min = 2
avg = 62.4
max = 198
Wald chi2(4) = 2093.27
Log likelihood = -4662.7132 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
normexam | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
standlrt | .5599641 .0124436 45.00 0.000 .5355752 .5843531
girl | .1672282 .0340818 4.91 0.000 .100429 .2340273
boyschool | .1776197 .1107533 1.60 0.109 -.0394529 .3946922
girlschool | .1589596 .0872548 1.82 0.068 -.0120567 .3299759
_cons | -.1681504 .0539994 -3.11 0.002 -.2739873 -.0623134
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
school: Identity |
var(_cons) | .0811077 .0165468 .0543761 .1209805
-----------------------------+------------------------------------------------
var(Residual) | .5622731 .0125854 .5381393 .5874891
------------------------------------------------------------------------------
LR test vs. linear model: chibar2(01) = 346.77 Prob >= chibar2 = 0.0000
. estimates store m3
.
. * Model 4 - mixed, cluster-robust standard errors
. mixed normexam standlrt girl boyschool girlschool || school:, vce(robust)
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0: log pseudolikelihood = -4662.7132
Iteration 1: log pseudolikelihood = -4662.7132
Computing standard errors:
Mixed-effects regression Number of obs = 4,059
Group variable: school Number of groups = 65
Obs per group:
min = 2
avg = 62.4
max = 198
Wald chi2(4) = 883.35
Log pseudolikelihood = -4662.7132 Prob > chi2 = 0.0000
(Std. Err. adjusted for 65 clusters in school)
------------------------------------------------------------------------------
| Robust
normexam | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
standlrt | .5599641 .0192667 29.06 0.000 .5222021 .5977261
girl | .1672282 .0290494 5.76 0.000 .1102925 .2241639
boyschool | .1776197 .0917738 1.94 0.053 -.0022538 .3574931
girlschool | .1589596 .0904992 1.76 0.079 -.0184156 .3363348
_cons | -.1681504 .0547635 -3.07 0.002 -.2754848 -.0608159
------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Robust
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
school: Identity |
var(_cons) | .0811077 .0171974 .0535278 .1228978
-----------------------------+------------------------------------------------
var(Residual) | .5622731 .0195671 .525201 .6019619
------------------------------------------------------------------------------
. estimates store m4
.
. * Compare results
. esttab m1 m2 m3 m4, wide se
--------------------------------------------------------------------------------------------------------------------------------
(1) (2) (3) (4)
normexam normexam
--------------------------------------------------------------------------------------------------------------------------------
main
cons -0.168** (0.0540) -0.168** (0.0543)
standlrt 0.560*** (0.0124) 0.560*** (0.0191) 0.560*** (0.0124) 0.560*** (0.0193)
girl 0.167*** (0.0341) 0.167*** (0.0288) 0.167*** (0.0341) 0.167*** (0.0290)
boyschool 0.178 (0.111) 0.178 (0.0911) 0.178 (0.111) 0.178 (0.0918)
girlschool 0.159 (0.0873) 0.159 (0.0898) 0.159 (0.0873) 0.159 (0.0905)
_cons -0.168** (0.0540) -0.168** (0.0548)
--------------------------------------------------------------------------------------------------------------------------------
RP2
var(cons) 0.0811*** (0.0162) 0.0811*** (0.0163)
--------------------------------------------------------------------------------------------------------------------------------
RP1
var(cons) 0.562*** (0.0126) 0.562*** (0.0194)
--------------------------------------------------------------------------------------------------------------------------------
lns1_1_1
_cons -1.256*** (0.102) -1.256*** (0.106)
--------------------------------------------------------------------------------------------------------------------------------
lnsig_e
_cons -0.288*** (0.0112) -0.288*** (0.0174)
--------------------------------------------------------------------------------------------------------------------------------
N 4059 4059 4059 4059
--------------------------------------------------------------------------------------------------------------------------------
Standard errors in parentheses
* p<0.05, ** p<0.01, *** p<0.001
.
end of do-file