Page 1 of 1
getting credible intervals around postestimations
Posted: Tue Dec 08, 2020 10:38 pm
by becciaa1992
Hello! I am fitting a null two-level multilevel logistic model using the following example code:
Code: Select all
runmlwin outcome cons , ///
level2(group: cons) ///
level1(id:) ///
discrete(distribution(binomial) link(logit) denominator(denominator) pql2) ///
rigls maxiterations(100) ///
nopause
runmlwin outcome cons , ///
level2(group: cons, residuals(u, savechains("m1u.dta", replace))) ///
level1(id:) ///
discrete(distribution(binomial) link(logit) denominator(denominator)) ///
mcmc(burnin(5000) chain(50000) thinning(50) ///
savechains("m1beta.dta", replace)) initsprevious ///
nopause
* pull residuals
egen pickone = tag(group)
sort u0
generate u0rank = sum(pickone)
sort group
list group u0 u0se u0rank if pickone==1
I am interest in comparing specific level 2 groups using odds ratios and odds difference, is it possible to get credible intervals around these estimates in runmlwin? Thanks!
Re: getting credible intervals around postestimations
Posted: Tue Dec 08, 2020 10:59 pm
by ChrisCharlton
I think that you should be able to do this by loading the .dta file where you have saved the parameter chains and then running the
mcmcsum command on the variable that corresponds to the parameter of interest. For details on the options for this command run:
from within Stata.
Re: getting credible intervals around postestimations
Posted: Wed Dec 09, 2020 10:22 pm
by becciaa1992
Thank you for your reply! I have tried mcmcsum, and I get the following error:
Code: Select all
Estimate results must come from runmlwin
Furthermore, when I load the .dta file where the parameter chains I stored, the only variables included are: iteration, deviance, FP1_cons, RP2_var_cons_, and OD_bcons_1; I don't see the residuals.
Re: getting credible intervals around postestimations
Posted: Wed Dec 09, 2020 10:44 pm
by becciaa1992
To give you some more context: After I fit the models and pull residuals:
Code: Select all
runmlwin outcome cons , ///
level2(group: cons) ///
level1(id:) ///
discrete(distribution(binomial) link(logit) denominator(denominator) pql2) ///
rigls maxiterations(100) ///
nopause
runmlwin outcome cons , ///
level2(group: cons, residuals(u, savechains("m1u.dta", replace))) ///
level1(id:) ///
discrete(distribution(binomial) link(logit) denominator(denominator)) ///
mcmc(burnin(5000) chain(50000) thinning(50) ///
savechains("m1beta.dta", replace)) initsprevious ///
nopause
* pull residuals
egen pickone = tag(group)
sort u0
generate u0rank = sum(pickone)
sort group
list group u0 u0se u0rank if pickone==1
I would like to calculate an OR between two groups by exponentiating their residuals, and then get a credible interval around that.
Re: getting credible intervals around postestimations
Posted: Thu Dec 10, 2020 12:30 pm
by ChrisCharlton
In your estimation command you requested that the residual chains at level-2 are stored in the file m1u.dta, so this is what you will need to open in order to calculate what you need. When running the mcmcsum on variables from the currently loaded dataset you need to specify the variables option when running the command, otherwise it will look for the chains saved as a mata matrix.
Re: getting credible intervals around postestimations
Posted: Thu Dec 10, 2020 2:48 pm
by becciaa1992
thank you so much! so that makes sense, although it looks like I can only use mcmcsum (with the variables option) if I use all the chains; when I try to use it on a subset (e.g., on the residual values for one group) I get an output that looks like this:
. mcmcsum u if group==1, variables
------------------------------------------------------------------------------
| Mean Std. Dev. ESS P [95% Cred. Interval]
-------------+----------------------------------------------------------------
policy | 0 0 0 45.000 0 0
------------------------------------------------------------------------------
So I'm assuming that means I cannot get credible intervals around group-specific effects?
Re: getting credible intervals around postestimations
Posted: Thu Dec 10, 2020 3:01 pm
by becciaa1992
Actually, I can get credible intervals around a group-specific effect using this code:
Code: Select all
bysort group (iteration): egen p_mean = mean(p)
bysort group (iteration): egen p_lo = pctile(p), p(2.5)
bysort group (iteration): egen p_hi = pctile(p), p(97.5)
where p is a variable for the group specific-predicted probability of the outcome (calculated from the residuals).
But then I run into issues when trying to compare two groups.
Re: getting credible intervals around postestimations
Posted: Thu Dec 10, 2020 3:20 pm
by ChrisCharlton
mcmcsum should work with subsets, so it may be a bug if that isn't working for you. A workaround would be to remove rows that do not correspond to the desired group before running the command, although the alternative code that you identified should be equivalent to what mcmcsum is doing for the mean and credible intervals anyway.
Re: getting credible intervals around postestimations
Posted: Thu Dec 10, 2020 8:50 pm
by becciaa1992
got it to work, thank you so much for your help!