Page 1 of 1

Number of groups varies between 2- and 3-level multivariate MLM

Posted: Wed Sep 25, 2019 9:16 pm
by erikruzek
Hi all,

I am running a multivariate MLM that has repeated observations (level 1) of teachers (level 2) within schools (level 3). The data has 550 non-missing observations on the two outcomes for 65 teachers in 23 schools.

Variable | Obs Mean Std. Dev. Min Max
-----------------+---------------------------------------------------------
ELA_overall | 550 2.472286 .7009207 .5555556 3.888889
CLASS_over~l | 550 4.394666 .6439461 2.739683 6.023016


There was not an occasion level variable, so I created one that does not repeat across teachers using

Code: Select all

sort id 
gen occ_c = _n 
The two level model is as follows:

Code: Select all

sort id occ_c
runmlwin (ELA_overall cons, eq(1)) (CLASS_overall cons, eq(2)), ///
	level2(id: (cons, eq(1)) (cons, eq(2))) ///
	level1(occ_c: (cons, eq(1)) (cons, eq(2))) nopause correlations //gives you corrs vs. covariances
The number of groups is as follows (total number of obs = 550):

Code: Select all

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
       id       |       65        1            8.5         17
-----------------------------------------------------------

I then run the 3-level model, asking for residuals at each level:

Code: Select all

sort sch1 id occ_c
runmlwin (ELA_overall cons, eq(1)) (CLASS_overall cons, eq(2)) , ///
	level3(sch1: (cons, eq(1)) (cons, eq(2)), residuals(uS)) ///
	level2(id: (cons, eq(1)) (cons, eq(2)), residuals(uT)) ///
	level1(occ_c: (cons, eq(1)) (cons, eq(2)), residuals(r)) nopause correlations //gives you corrs vs. covariances
This model reports an additional two id groups, and the number of observations per id differs slightly (total number of obs = 550):

Code: Select all

-----------------------------------------------------------
                |   No. of       Observations per Group
 Level Variable |   Groups    Minimum    Average    Maximum
----------------+------------------------------------------
      sch1     |       23       1            23.9         60
      id       |       67       1              8.2         17
-----------------------------------------------------------
Any ideas what might be going on?

Thanks,
Erik

Re: Number of groups varies between 2- and 3-level multivariate MLM

Posted: Thu Sep 26, 2019 9:01 am
by ChrisCharlton
This would normally happen if the same teacher id occurred within more than one school id. What happens if you run the first model with the data sorted as in the second model?

Re: Number of groups varies between 2- and 3-level multivariate MLM

Posted: Thu Sep 26, 2019 9:01 pm
by erikruzek
Thanks, Chris. You identified the problem exactly. I have two teachers who each taught in two different schools. Technically that means teachers and schools are cross-classified. However, with only two of 65 teachers in this situation, I'm curious whether you think it makes sense to model the cross-classification?

Re: Number of groups varies between 2- and 3-level multivariate MLM

Posted: Fri Sep 27, 2019 9:19 am
by ChrisCharlton
I can't really advise on that (although a quick search turned up https://doi.org/10.3389/fpsyg.2016.00328, which might be relevant), technically you should probably fit the cross-classified model but it may not change the results much. Chapter 18 of the replication materials for the MLwiN user guide (see https://www.bristol.ac.uk/cmm/software/ ... /examples/) demonstrates fitting a cross-classified model with IGLS, but it is easiest to set up if you use MCMC instead. Maybe try running the models as both nested and cross-classified and see what differences it makes to the results?

Re: Number of groups varies between 2- and 3-level multivariate MLM

Posted: Fri Sep 27, 2019 4:12 pm
by erikruzek
Thanks, Chris. Knowing the CC exists and it is so easily handled in MLwiN, I agree it makes sense to model it correctly. I did so and variance and covariance estimates at each of the levels are quite similar, and as a result, the number of groups being reported is exactly aligned with expectations.