Detecting significant results

Welcome to the forum for MLwiN users. Feel free to post your question about MLwiN software here. The Centre for Multilevel Modelling take no responsibility for the accuracy of these posts, we are unable to monitor them closely. Do go ahead and post your question and thank you in advance if you find the time to post any answers!

Remember to check out our extensive software FAQs which may answer your question: http://www.bristol.ac.uk/cmm/software/s ... port-faqs/
Post Reply
SabineKatzdobler
Posts: 23
Joined: Sat Sep 05, 2020 10:22 am

Detecting significant results

Post by SabineKatzdobler »

Good day,

At the moment, I am trying to fit a multilevel "empty model" with an ordered category response variable and I can only see effect sizes and standard errors in the model results. My question involves, where can I see exactely if the effect sizes are statistically significant (two-tailed p-value) or not without involving MCMC method? I would rather prefer MQL and PQL.

Kind regards,

Sabine
billb
Posts: 157
Joined: Fri May 21, 2010 1:21 pm

Re: Detecting significant results

Post by billb »

Hi Sabine,
The estimates from MQL / PQL are quasi-likelihood so you cannot do a standard likelihood-based test to get a P value. You can do an approximate Wald test via the intervals and tests window but bear in mind this assumes normality for the variables so is only an approximation.
Hope that helps,
Bill.
SabineKatzdobler
Posts: 23
Joined: Sat Sep 05, 2020 10:22 am

Re: Detecting significant results

Post by SabineKatzdobler »

Good day,

thanks for your fast response. I have a little follow-up question: Where can I see the p-value regarding the independent variables in the regression equation to determine if the independent variable is significant or not?

Kind regards,

Sabine
billb
Posts: 157
Joined: Fri May 21, 2010 1:21 pm

Re: Detecting significant results

Post by billb »

Hi Sabine,
Not sure I follow your question - the interval and tests window will give you a P value if we are still talking about the same model but you need to tell it what Wald test to perform (there is something about this window in the help system and an example in the binary response chapter of the users guide). You could also Store the model by clicking Store on the Equations window and giving it a name. Then if you choose Manage stored models that will allow you in the options to choose P value for each coefficient and they will be displayed but once again this is assuming normality of the parameters.
Best wishes,
Bill.
SabineKatzdobler
Posts: 23
Joined: Sat Sep 05, 2020 10:22 am

Re: Detecting significant results

Post by SabineKatzdobler »

Hi Bill,

thanks for your quick response.

For both of my questions I had the p-value for each coefficient in mind. Additionally, my variables are not normally distributed. What can you recommend for testing the effect on each coefficient on an ordered categorical response variable regarding its significance? Furthermore, I would not like to interpret a MCMC-model.
billb
Posts: 157
Joined: Fri May 21, 2010 1:21 pm

Re: Detecting significant results

Post by billb »

Hi Sabine,
I am afraid outside of MCMC you are stuck with the approximate Wald test (which essentially assumes normality) of equivalently the P values spat out by the Store model window which does likewise.
Best wishes,
Bill.
SabineKatzdobler
Posts: 23
Joined: Sat Sep 05, 2020 10:22 am

Re: Detecting significant results

Post by SabineKatzdobler »

Hi Bill,

thank you - I will analyse the results of IGLS, RIGLS and MCMC.

Kind regards,

Sabien
Post Reply