Number of groups varies between 2- and 3-level multivariate MLM

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
erikruzek
Posts: 12
Joined: Tue May 21, 2019 3:57 pm

Number of groups varies between 2- and 3-level multivariate MLM

Hi all,

I am running a multivariate MLM that has repeated observations (level 1) of teachers (level 2) within schools (level 3). The data has 550 non-missing observations on the two outcomes for 65 teachers in 23 schools.

Variable | Obs Mean Std. Dev. Min Max
-----------------+---------------------------------------------------------
ELA_overall | 550 2.472286 .7009207 .5555556 3.888889
CLASS_over~l | 550 4.394666 .6439461 2.739683 6.023016

There was not an occasion level variable, so I created one that does not repeat across teachers using

Code: Select all

``````sort id
gen occ_c = _n ``````
The two level model is as follows:

Code: Select all

``````sort id occ_c
runmlwin (ELA_overall cons, eq(1)) (CLASS_overall cons, eq(2)), ///
level2(id: (cons, eq(1)) (cons, eq(2))) ///
level1(occ_c: (cons, eq(1)) (cons, eq(2))) nopause correlations //gives you corrs vs. covariances
``````
The number of groups is as follows (total number of obs = 550):

Code: Select all

``````-----------------------------------------------------------
|   No. of       Observations per Group
Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
id       |       65        1            8.5         17
-----------------------------------------------------------``````

I then run the 3-level model, asking for residuals at each level:

Code: Select all

``````sort sch1 id occ_c
runmlwin (ELA_overall cons, eq(1)) (CLASS_overall cons, eq(2)) , ///
level3(sch1: (cons, eq(1)) (cons, eq(2)), residuals(uS)) ///
level2(id: (cons, eq(1)) (cons, eq(2)), residuals(uT)) ///
level1(occ_c: (cons, eq(1)) (cons, eq(2)), residuals(r)) nopause correlations //gives you corrs vs. covariances
``````
This model reports an additional two id groups, and the number of observations per id differs slightly (total number of obs = 550):

Code: Select all

``````-----------------------------------------------------------
|   No. of       Observations per Group
Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
sch1     |       23       1            23.9         60
id       |       67       1              8.2         17
-----------------------------------------------------------``````
Any ideas what might be going on?

Thanks,
Erik
ChrisCharlton
Posts: 1252
Joined: Mon Oct 19, 2009 10:34 am

Re: Number of groups varies between 2- and 3-level multivariate MLM

This would normally happen if the same teacher id occurred within more than one school id. What happens if you run the first model with the data sorted as in the second model?
erikruzek
Posts: 12
Joined: Tue May 21, 2019 3:57 pm

Re: Number of groups varies between 2- and 3-level multivariate MLM

Thanks, Chris. You identified the problem exactly. I have two teachers who each taught in two different schools. Technically that means teachers and schools are cross-classified. However, with only two of 65 teachers in this situation, I'm curious whether you think it makes sense to model the cross-classification?
ChrisCharlton
Posts: 1252
Joined: Mon Oct 19, 2009 10:34 am

Re: Number of groups varies between 2- and 3-level multivariate MLM

I can't really advise on that (although a quick search turned up https://doi.org/10.3389/fpsyg.2016.00328, which might be relevant), technically you should probably fit the cross-classified model but it may not change the results much. Chapter 18 of the replication materials for the MLwiN user guide (see https://www.bristol.ac.uk/cmm/software/ ... /examples/) demonstrates fitting a cross-classified model with IGLS, but it is easiest to set up if you use MCMC instead. Maybe try running the models as both nested and cross-classified and see what differences it makes to the results?
erikruzek
Posts: 12
Joined: Tue May 21, 2019 3:57 pm

Re: Number of groups varies between 2- and 3-level multivariate MLM

Thanks, Chris. Knowing the CC exists and it is so easily handled in MLwiN, I agree it makes sense to model it correctly. I did so and variance and covariance estimates at each of the levels are quite similar, and as a result, the number of groups being reported is exactly aligned with expectations.