95% prediction interval for new obs from same cluster populations

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
RachaelHughes
Posts: 8
Joined: Thu Jun 13, 2019 10:33 am

95% prediction interval for new obs from same cluster populations

Post by RachaelHughes » Tue Nov 26, 2019 10:13 am

Hello,

Using MCMC I have fitted a multivariate normal 3-level cross-classified model with random intercepts at each level. In my data, surgeons have conducted eye-operations, where a patient may have had a single eye operation or two separate eye operations conducted by the same surgeon or different surgeons. The outcome of interest is a 3-component continuous measure of vision. Of interest is if the presence of pre-existing condition diabetes (yes/no) affects the outcome of the surgery.

I want to calculate a 95% interval that tells me the range of values I would see among new eye operations conducted on individuals from the same population of patients and operated on by the same population of surgeons. So, I want to calculate a 95% interval for new eye-operations on people with diabetes and a 95% interval for new eye-operations on people without diabetes.

My thinking on how to do calculate a 95% interval is as follows:
1) After convergence, run the MCMC chain for say 1000 iterations.

2) For each iteration:
(a) Draw a random intercept at level 3 and at level 2 (from a multivariate normal distribution) using the current estimates of the random covariance matrix.
(b) Use the random draws from (a) and current estimates of the fixed effects to generate a new predicted value (e.g., fixed effects + random effect draws)

3) Use the 2.5% and 97.5% of the distribution of predicted values in step 2 as my 95% prediction interval.

Is this a sensible approach? I can see that it accounts for uncertainty in the fixed-effect coefficients and random variance parameters but not the residual variance. Is there a simpler approach?

Thanks in advance
Rach

billb
Posts: 114
Joined: Fri May 21, 2010 1:21 pm

Re: 95% prediction interval for new obs from same cluster populations

Post by billb » Tue Nov 26, 2019 12:40 pm

Hi Rachael,
If you were hoping to get 95% intervals for individuals rather than for the average individual you will also need to have level 1 in your predictions I would expect.
Best wishes,
Bill.

RachaelHughes
Posts: 8
Joined: Thu Jun 13, 2019 10:33 am

Re: 95% prediction interval for new obs from same cluster populations

Post by RachaelHughes » Tue Nov 26, 2019 1:50 pm

Thanks Bill for your reply. I suspected my idea wasn't quite right.

So to predict for individuals should I amend step 2 (in particular part b) as follows:

2) For each MCMC iteration:
(a) Draw a random intercept at level 3 and at level 2 (from a multivariate normal distribution) using the current estimates of the random covariance matrix.
(b) Draw predicted values from a Normal distribution with
mean=Fixed + random realisation draws from (a)
variance=level 1 residual variance
Best wishes
Rach

billb
Posts: 114
Joined: Fri May 21, 2010 1:21 pm

Re: 95% prediction interval for new obs from same cluster populations

Post by billb » Tue Nov 26, 2019 1:57 pm

Hi Rachael,
Just generate a level 1 residual (using the level 1 variance) and add it to the prediction and you should be sorted.
Bill.

Post Reply