Page 1 of 1

Predicted Probability for cross-classified ML model

Posted: Wed Aug 16, 2017 7:04 pm
by vivian1234
Hi,

I would like to calculate the population-averaged probability for a cross-classified ML model.
I have read this post: https://www.cmm.bristol.ac.uk/forum/vie ... 23fb#p3867,
2. Depends on whether you want cluster-specific or population-averaged inference. If you want to make predictions for a specific respondent or interviewer then you need to plug in their predicted random effect values (i.e., cluster-specific inference). If you want to calculate the prediction averaging over the respondent and interviewer effects then you need to probably go down a simulation-based route (i.e., population average inference)

3. Yes you are correct. Only including the fixed-part of the model in the prediction is equivalent to setting all the random effects to zero. Whether this is good thing to do depends on what type of inference you want. I suspect you want population-average inference unless you are trying to illustrate the heterogeneity in the predicted probability across clusters. Setting the random effects to zero will give you something similar if you have a low degree of clustering, especially if the probability is close to 0.5, But if you don't have this then setting the random effects to zero will typically give much more extreme predictions (further away from 0.5) than the population average.

4. If you want to do this then you need to calculate population-averaged probabilities. See response to 3.

How can I do a simulation-based route (point 2) using R2MLwin to obtain a PA probability? Any ideas would be great.

Thanks a lot.

Vivian

Re: Predicted Probability for cross-classified ML model

Posted: Wed Aug 23, 2017 10:42 am
by billb
Hi Vivian,
I suspect this is pretty challenging for a cross-classified scenario. Lemma module 7 has a section on PA for 2-level binomial and the simulation approach (section 7.4) and my guess is that to do cross-classified equivalent calculations would require an additional level of looping in the simulations which I hope gives you some pointers.
Regards,
Bill.