MCMC: calculating estimates, CrIs for combinations of terms/interactions

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
amaald21
Posts: 6
Joined: Wed Nov 24, 2021 3:55 am

MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by amaald21 »

Hello,

I'm wondering if there is a way to calculate estimates and credible intervals for different combinations of terms (including interaction terms) if the model was run using runmlwin and MCMC?

I'm looking to create a table like the one in this article (Table 1): https://doi.org/10.1093/ije/dyr218
which would present, for example, odds ratios and credible intervals for the combination of Term A + Term B + Interaction(A+B)

I would typically do this in Stata using the 'lincom' command, but it looks like lincom doesn't work because the estimates/interaction terms are variables, not e-class results, in the saved chains file.

Is there a way of doing this, or a lincom-type command that would work with the saved chains?

Thanks very for your help, much appreciated!
Amanda
ChrisCharlton
Posts: 1348
Joined: Mon Oct 19, 2009 10:34 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by ChrisCharlton »

I think that you should be able to do this with the normal variable manipulation commands such as generate and egen. Once you have the new variable representing the result of the calculation you should be able to get the credible intervals using the summarize command, or the mcmcsum provided with runmlwin.
amaald21
Posts: 6
Joined: Wed Nov 24, 2021 3:55 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by amaald21 »

Hi Chris,

Thanks so much - that makes perfect sense! I've tried this out using generate and mcmcsum and have got the combinations of terms and their credible intervals.

Is there a way to get the p-value as an r-class result too?

Thank you!
Amanda
ChrisCharlton
Posts: 1348
Joined: Mon Oct 19, 2009 10:34 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by ChrisCharlton »

It should be fairly easy to modify mcmcsum to do this. You can see where the r-class is filled in on lines 535-552 or 410-427 of mcmcsum. To add Bayesian p-values you just need to decide whether you want the mean, median or mode version and add an extra line to return this, i.e.

Code: Select all

return scalar pvalue         = `pvalmean'
If you prefer more conventional p-values you can calculate these from the estimate and variance as usual.
amaald21
Posts: 6
Joined: Wed Nov 24, 2021 3:55 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by amaald21 »

Thanks Chris. I've tried this and it seems to work, but the p-values are only to one significant figure (they display as '0'). Could you please advise how I might alter the ado so that I can get the p value to a few more significant figures e.g. 0.001?

Thank you!
Amanda
ChrisCharlton
Posts: 1348
Joined: Mon Oct 19, 2009 10:34 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by ChrisCharlton »

The p-values reported here are one-tailed bayesian p-values, and essentially give the proportion of the chain that crosses zero (or one if the variable is exponentiated and it is aware of this). You can find the code for this in runmlwin_mcmcdiag.ado, and is as follows:

Code: Select all

		local compval 0
		if "`eform'" ~= "" {
			local compval 1
		}
		
		if `mean' > `compval' {
			quietly count if `parameter' < `compval'
		}
		else {
			quietly count if `parameter' > `compval'
		}		
		local pvalmean = r(N) / `N'
I suggest testing with a variable that contains both positive and negative numbers to check whether this is why you are only getting values of zero.
grippingfossil
Posts: 9
Joined: Fri Jan 13, 2023 2:58 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by grippingfossil »

I am able to fix my bug. funny shooter 2
Many thanks!
ikvkntz644
Posts: 1
Joined: Tue Dec 26, 2023 7:46 am

Re: MCMC: calculating estimates, CrIs for combinations of terms/interactions

Post by ikvkntz644 »

It seems like you're working with multilevel models using the runmlwin command in Stata and seeking to calculate estimates and credible intervals for combinations of terms, including interaction terms, when using Markov Chain Monte Carlo (MCMC) methods. If I understand correctly, you're looking for a way to obtain estimates similar to those presented in Table 1 of the provided article.

In the context of multilevel modeling with MCMC, Stata's runmlwin command may not directly provide the same functionalities as the lincom command for combinations of terms. However, you can achieve similar results by manually calculating the relevant quantities from the saved chains.

Here are steps you might consider: car games

Extract Chains:

Make sure you have saved the chains using the save option in runmlwin. This will create a dataset containing the chains for each parameter.
Combine Chains:

Combine the saved chains dataset with your original dataset, linking on the ID variable. This allows you to work with both your model results and your original data in the same dataset.
Calculate the Combinations:

Create new variables that represent the combinations of terms and interaction terms you are interested in. This may involve straightforward multiplication or combination of existing variables.
Calculate Estimates:

For each iteration in the MCMC chains, calculate the estimates of interest using the combinations you created.
Summarize Results:

Use Stata commands like summarize or ci to obtain summary statistics and credible intervals for the estimates across the MCMC iterations.
Post Reply