Multinomial logistic model variance covariance matrix

Welcome to the forum for runmlwin users. Feel free to post your question about runmlwin here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Go to runmlwin: Running MLwiN from within Stata >> http://www.bristol.ac.uk/cmm/software/runmlwin/
Post Reply
feifeibu
Posts: 11
Joined: Thu Oct 02, 2014 11:02 am

Multinomial logistic model variance covariance matrix

Post by feifeibu »

Hi all,

I have a two-level multinomial model. After fitting my model using MQL1, I want it fit it by MCMC. However I get the following error message:
“error while obeying batch file C:\user… at line number 2183: MCMC 0 500 1 5.8 10 C1411] C1412] 2 2 2 1 1 6”

After reading the materials online, I realize this is due to the negatively defined covariance matrix, and that I need to manually alter the priors. So instead of initsprevious, I need to use initsb() initsv(). But here is the problem.

Using the help file example as an illustration (it doesn’t have any problem with covariance matrix of course):

Code: Select all

use http://www.bristol.ac.uk/cmm/media/runmlwin/bang, clear
generate lc1 = (lc==1)
generate lc2 = (lc==2)
generate lc3plus = (lc>=3)
runmlwin use4 cons lc1 lc2 lc3plus, level2(district: cons) ///
level1(woman)  ///
discrete(distribution(multinomial) link(mlogit) denom(cons) basecategory(4)) /// 
nopause
Then I fit the model by MCMC:

Code: Select all

matrix b = e(b)
matrix V = e(V)
runmlwin use4 cons lc1 lc2 lc3plus, level2(district: cons) ///
level1(woman)  ///
discrete(distribution(multinomial) link(mlogit) denom(cons) basecategory(4)) /// 
mcmc(on) initsb(b) initsv(V)  ///
nopause

Here I haven’t tried to modify the priors yet, which is supposed to come at next step. But I got the following error message:

Code: Select all

“The row vector b of initial values in initsb() is of length 22, but -runmlwin- expects the row vector to be of length 21.”
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Multinomial logistic model variance covariance matrix

Post by ChrisCharlton »

The behaviour that you are seeing is due to (for various reasons) the number of parameters in a multinomial model being different between IGLS and MCMC estimation and therefore the starting values not matching up.

To get around this you can use the initsmodel option which rather than matching the parameters by position instead uses the parameter names. For the example model that you provided the syntax would be as follows:

Code: Select all

use http://www.bristol.ac.uk/cmm/media/runmlwin/bang, clear
generate lc1 = (lc==1)
generate lc2 = (lc==2)
generate lc3plus = (lc>=3)

runmlwin use4 cons lc1 lc2 lc3plus, level2(district: cons) ///
level1(woman) ///
discrete(distribution(multinomial) link(mlogit) denom(cons) basecategory(4)) ///
nopause

estimates store iglsmodel

runmlwin use4 cons lc1 lc2 lc3plus, level2(district: cons) ///
level1(woman) ///
discrete(distribution(multinomial) link(mlogit) denom(cons) basecategory(4)) ///
mcmc(on) initsmodel(iglsmodel) ///
nopause
Alternatively you would have to modify the initial value matrices so that the ones used for MCMC only have one bcons parameter.
feifeibu
Posts: 11
Joined: Thu Oct 02, 2014 11:02 am

Re: Multinomial logistic model variance covariance matrix

Post by feifeibu »

Thank you very much for your help, Chris! It works! Just to be sure I get it right, still using the the help file example:

Code: Select all

use http://www.bristol.ac.uk/cmm/media/runmlwin/bang, clear
generate lc1 = (lc==1)
generate lc2 = (lc==2)
generate lc3plus = (lc>=3)

runmlwin use4 cons lc1 lc2 lc3plus, level2(district: cons) ///
level1(woman)  ///
discrete(distribution(multinomial) link(mlogit) denom(cons) basecategory(4)) ///
nopause
matrix drop _all
matrix b = e(b)
matrix V = e(V)
matrix b=b[1, 1..19]
matrix V=V[1..19, 1..19]

runmlwin use4 cons lc1 lc2 lc3plus, level2(district: cons) ///
level1(woman) ///
discrete(distribution(multinomial) link(mlogit) denom(cons) basecategory(4)) /// 
mcmc(on) initsb(b) initsv(V) ///
nopause
Then runmlwin works perfectly. But is this what you mean?

Then I tried to modify the variance matrix for my own model. But unfortunately I still get the error message saying "Prior variance matrix is not positive definite(MCMC)". Could you please take a look at the attachment? Thanks very much!

Feifei
Attachments
mlwin.docx
(134.55 KiB) Downloaded 484 times
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Multinomial logistic model variance covariance matrix

Post by ChrisCharlton »

Yes, you have correctly understood what I was suggesting. The random part covariance matrices used in the prior/initial values when running the MCMC model have to be invertible. Looking at the document that you sent you appear to have variances of zero in your Omega_v covariance matrix, which is likely to prevent this being the case. I would suggest changing this value, and then checking the Omega_v portion of your initial values matrix with the cholesky Stata matrix function (see http://www.stata.com/help.cgi?matrix+functions). If this doesn't generate an error message then the matrix should be accepted by MLwiN.
feifeibu
Posts: 11
Joined: Thu Oct 02, 2014 11:02 am

Re: Multinomial logistic model variance covariance matrix

Post by feifeibu »

Thank you for your reply, Chris. Following your suggestion, I tried to alter the 0 covariances in the b matrix. But my matrix algebra knowledge is rather limited, so it is not clear to me how I can alter the corresponding elements in the V matrix. Could you give an example? Thanks a lot!
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Multinomial logistic model variance covariance matrix

Post by ChrisCharlton »

As far as I know the random part of the V matrix isn't used by the MLwiN MCMC algorithm, so you should be able to leave this as it is (I believe that the fixed part of the V matrix is used in the model adapting if this occurs).
feifeibu
Posts: 11
Joined: Thu Oct 02, 2014 11:02 am

Re: Multinomial logistic model variance covariance matrix

Post by feifeibu »

Thanks, Chris.
I manually changed the variance and covariance in the b matrix.

Code: Select all

matrix drop _all			 
matrix b = e(b)
matrix V = e(V)
*
matrix b=b[1, 1..21]
matrix b[1,12]=-0.127
matrix b[1,13]=0.14
matrix b[1,14]=-0.16
matrix b[1,15]=0.202
matrix b[1,19]=-0.236
matrix b[1,20]=0.939
matrix V=V[1..21, 1..21]
The omega_v matrix seems to be fine (please see the attachment). But I still get the same error: Prior variance matrix is not positive definite (MCMC). Any idea? Thank you very much for your help :)
Attachments
mlwin2.docx
(75.59 KiB) Downloaded 453 times
ChrisCharlton
Posts: 1351
Joined: Mon Oct 19, 2009 10:34 am

Re: Multinomial logistic model variance covariance matrix

Post by ChrisCharlton »

It looks as if your chosen matrix is still not positive-definite:

Code: Select all

. mat a = (0.259, -0.235, 0.117, -0.127, -0.113 \ -0.235, 0.476, -0.139, 0.140, -0.153 \ 0.117, -0.139, 0.067, -0.160, -0.174 \ -0.127, 0.140, -0.160, 0.202, -0.236 \ -0.113, -0.153, -0.174, -0.236, 0.939)

. mat list a

symmetric a[5,5]
       c1     c2     c3     c4     c5
r1   .259
r2  -.235   .476
r3   .117  -.139   .067
r4  -.127    .14   -.16   .202
r5  -.113  -.153  -.174  -.236   .939

. mat b = cholesky(a)
matrix not positive definite
r(506);
You will need to make more changes to your matrix so that the above function no longer gives an error.
feifeibu
Posts: 11
Joined: Thu Oct 02, 2014 11:02 am

Re: Multinomial logistic model variance covariance matrix

Post by feifeibu »

Finally, get it work (by checking the determinants) !!!
Thank you for your help :)
Post Reply