Page **1** of **1**

### 3 level cross-classified model: how to set level 2 and 3?

Posted: **Fri Jun 07, 2019 3:39 pm**

by **scorpionwwj**

Dear users,

I am fitting a 3 level cross-classified model: level 1 is patient ID (ca.40000), the other 2 levels are physician ID (ca.7000) and region ID where patients live in (ca.700). Outcome variable is binary so this is a logistic model. Below is the code:

Code: Select all

```
runmlwin y sex age cons, level3(region:cons) level2(physician:cons) level1(id) discrete(distribution(binomial) link(logit) denominator(cons)) or nopause
runmlwin y sex age cons, level3(region:cons) level2(physician:cons) level1(id) discrete(distribution(binomial) link(logit) denominator(cons)) or mcmc(on) initsprevious nopause nogroup
```

But if I switch the level 2 and level 3 like below:

Code: Select all

```
runmlwin y sex age cons, level3(physician:cons) level2(region:cons) level1(id) discrete(distribution(binomial) link(logit) denominator(cons)) or nopause
runmlwin y sex age cons, level3(physician:cons) level2(region:cons) level1(id) discrete(distribution(binomial) link(logit) denominator(cons)) or mcmc(on) initsprevious nopause nogroup
```

I got quite different results, especially the random effects.

It would be great if anyone could help. Thank you indeed!

Best,

wj

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Mon Jun 10, 2019 8:59 am**

by **ChrisCharlton**

In order to fit a cross-classified model you need to add

**cc** to the mcmc options, otherwise the model will be fitted as hierarchical. This should be reflected in group summary information in the model output. Note that even after doing so the results from switching the levels around, as in your example, will be slightly different as the IGLS results are used as starting values and these will still assume a hierarchical structure. You can find examples of specifying cross-classified models within runmlwin in chapter 15 of the MCMC manual examples, available at

https://www.bristol.ac.uk/cmm/software/ ... /examples/.

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Fri Jun 14, 2019 4:38 pm**

by **scorpionwwj**

Dear Chris,

thank you indeed for your reply. It is really helpful.

Best regards,

wj

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Fri Jun 14, 2019 7:47 pm**

by **scorpionwwj**

ChrisCharlton wrote: ↑Mon Jun 10, 2019 8:59 am

In order to fit a cross-classified model you need to add

**cc** to the mcmc options, otherwise the model will be fitted as hierarchical. This should be reflected in group summary information in the model output. Note that even after doing so the results from switching the levels around, as in your example, will be slightly different as the IGLS results are used as starting values and these will still assume a hierarchical structure. You can find examples of specifying cross-classified models within runmlwin in chapter 15 of the MCMC manual examples, available at

https://www.bristol.ac.uk/cmm/software/ ... /examples/.

Dear Chris,

I have one further question. I have modified the code according to your reply:

Code: Select all

```
runmlwin y sex age cons, level3(region:cons) level2(physician:cons) level1(id) discrete(distribution(binomial) link(logit) denominator(cons)) or nopause
runmlwin y sex age cons, level3(region:cons) level2(physician:cons) level1(id) discrete(distribution(binomial) link(logit) denominator(cons)) or mcmc(cc on) initsprevious nopause nogroup
```

But the results of random-effect parameters are a bit weird: the higher level variation is 0.08 for level 3 (region), 1308 for level 2 (physician) which is too big. I have also tried two 2-level models, one with region as 2nd level, the other one with physician as 2nd level, the higher level variation for the 2 models are 0.08 and 0.18, respectively. I am not sure if the random effect estimates in the cross-classified model, especially for the physician level are correct.

Thank you in advance for your reply.

Best,

wj

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Mon Jun 17, 2019 9:11 am**

by **ChrisCharlton**

Have you checked the MCMC chain and convergence diagnostics? It may be that you have to run the model for more iterations before it converges to a distribution. I would also suggest removing the **nopause** option to check within MLwiN via the **Model->Hierarchy Viewer** window that you have the expected number of units within each classification.

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Tue Jul 02, 2019 5:12 pm**

by **scorpionwwj**

ChrisCharlton wrote: ↑Mon Jun 17, 2019 9:11 am

Have you checked the MCMC chain and convergence diagnostics? It may be that you have to run the model for more iterations before it converges to a distribution. I would also suggest removing the

**nopause** option to check within MLwiN via the

**Model->Hierarchy Viewer** window that you have the expected number of units within each classification.

Dear Chris,

thank you very much for your reply.

First, I have removed nonpause option and checked in the "Hierarchy Viewer" and the number of units within each classification is correct.

Second, I tried to increase iterations with the option "mcmc(cc burnin(5000) chain(10000))", but the random effect at physician level was even bigger (ca.4500), then I increased further the iterations by "mcmc(cc burnin(100000) chain(500000))", the random effect at physician level turned to 26500. And the random effect at region level was always stable (0.08).

Then, I checked the trajectories within MLwin, attached are the figures for region level and physician level, respectively. The figure for region level looks fine, but for physician level it is odd. I noticed that the mean value is abnormally large, but the median value is quite reasonable (as same as in the 2-level model with patient and physician, ca.0.18). I am not sure about the reason, but do you think if I could use median instead of mean for random effect at physician level?

Thank you indeed.

Best,

wj

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Wed Jul 03, 2019 11:44 am**

by **ChrisCharlton**

The spikes in the chains do look odd. It may be that there is something unusual in your data causing this (for example units with mostly or all zeros or ones). We would be willing to take a closer look if this data is something that you are able to share. If so could you send it to Bill Browne, who implemented the MCMC algorithm, using the email address on

https://www.bristol.ac.uk/cmm/team/?

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Thu Jul 04, 2019 12:10 pm**

by **scorpionwwj**

Dear Chris,

thank you for your reply. Yes, I agree that the reason might be that both region and physician levels have many units and three might be quite a lot of units with zeros and ones, especially at the physician level, around 1500 of 7000 physicians have only one patient.

Unfortunately, I could not share the data as they are from the insurance company and it is not allowed to share it without permission.

What would you think if I use median instead of mean? Or do you have any further suggestions?

Thank you indeed.

Best,

wj

### Re: 3 level cross-classified model: how to set level 2 and 3?

Posted: **Fri Jul 05, 2019 8:37 am**

by **ChrisCharlton**

I asked Bill whether he could give any further advice without seeing the data and the response was as follows:

This isn't behaviour that we have seen before and so therefore likely to be something to be specific to their dataset, given how often these models are fitted, which is hard to diagnose without seeing the dataset.

A couple of things

- Do they have any missing data as they could then listwise delete the missing data and see if that fixes things in case either runmlwin or MLwiN itself is getting confused by missing data.
- They could in MLwiN try the MLwiN->WinBUGS convertor and then (i) try running the model in WinBUGS and/or (ii) Look at the data generated for WinBUGS to see if there is anything unusual e.g. any weird values etc.