Hiya,
I've hit a wall with my multilevel models, which I was hoping someone could help me with. It's almost certainly because I'm using MLWin wrong, so with any luck, someone will be able to point out my obvious mistake.
I have monthly inflation indices for a series of categories of goods for the 12 Government Office Regions and nations of the UK. So my data is structured as 12 GORS > 13 Categories > 84 monthly measurements.
The descriptives suggest that the monthly measurements vary by both category and GOR, so I expect MLWin to tell me something like - month accounts for most of the variance, category most of the remaining variance, GOR for a small but noticeable amount.
If I run a 2-level model with GOR as level 2 and month as level 1, I get roughly that - GOR accounts for about 5% of the variance, month for about 95%.
However, if I make the model 3-level, with GOR as level 3, Category as level 2 and month as level 1, the variance due to GOR disappears. It all goes to the Category, along with some of the variance previously due to month. GOR is given a variance of 0.0000 with a standard error of 0.0000.
Unless I'm interpreting the results incorrectly, I'm being given an answer that makes no sense. I think my data is sorted correctly, and the hierarchy viewer shows what I expect it to.
Is there something extra I should be taking account of in the 3-level model that I wouldn't in the 2-level model? Does my data need to be structured differently?
Any help would be greatly appreciated.
Variance disappearing in shift from 2- to 3-level model
-
Mattoftheday
- Posts: 3
- Joined: Wed Jun 20, 2012 12:30 pm
Re: Variance disappearing in shift from 2- to 3-level model
Hi Mattoftheday,
your email puzzles me a little. When you say you have a structure of 12 > 13 > 84 do you in fact mean 12 GORS with 12*13 Categories within the 12 i.e. categories within GOR or do you mean you have a cross-classified structure with GOR crossed by Category ? Either way these sorts of errors often occur when the data is incorrectly sorted i.e. it is not sorted in line with the data structure with the all of each GOR clustered together then within each GOR each category clustered together - the hierarchy viewer window can help here although you sound like you already know this. Perhaps if you clarify your structure that will help.
Regards,
Bill Browne.
your email puzzles me a little. When you say you have a structure of 12 > 13 > 84 do you in fact mean 12 GORS with 12*13 Categories within the 12 i.e. categories within GOR or do you mean you have a cross-classified structure with GOR crossed by Category ? Either way these sorts of errors often occur when the data is incorrectly sorted i.e. it is not sorted in line with the data structure with the all of each GOR clustered together then within each GOR each category clustered together - the hierarchy viewer window can help here although you sound like you already know this. Perhaps if you clarify your structure that will help.
Regards,
Bill Browne.
-
Mattoftheday
- Posts: 3
- Joined: Wed Jun 20, 2012 12:30 pm
Re: Variance disappearing in shift from 2- to 3-level model
Dear Bill,
Thank you for taking the time to help me with this, it's very kind of you. (Sorry for my own delay in replying, I was at the RGS conference.)
The structure is that each GOR has the same set of 13 categories, but each with their own independent set of measurements. So London will have indices of prices for 'Food', for 'Clothing' and so on; the South East will also have indices of prices for 'Food', for 'Clothing; but the indices are independent of one another. So the variance in an index score can come from one of three places - the month in which it is recorded (an index score in month 84 is likely to be different from one in month 1), from the category (an index score for 'Food' in month 84 is likely to be different from one for 'Clothing' in month 84) or from the region (an index score for 'Food' in month 84 in London is likely to be different from an index score for 'Food' in month 84 in the South East).
I think the sorting I have is correct. In the three level model (GOR top level, Category in between, Month at the lowest level), the hierarchy viewer shows 12 GORs, 13 Categories and 84 months, in the two level model (GOR at the top and Month at the bottom) it shows 12 GORS and 1092 months (1092 being 13 by 84). Even without considering the 2-level model, the results of the 3-level look wrong to me, there should be some variance arising from GOR, as the index scores are not uniform across different regions.
Thank you for taking the time to help me with this, it's very kind of you. (Sorry for my own delay in replying, I was at the RGS conference.)
The structure is that each GOR has the same set of 13 categories, but each with their own independent set of measurements. So London will have indices of prices for 'Food', for 'Clothing' and so on; the South East will also have indices of prices for 'Food', for 'Clothing; but the indices are independent of one another. So the variance in an index score can come from one of three places - the month in which it is recorded (an index score in month 84 is likely to be different from one in month 1), from the category (an index score for 'Food' in month 84 is likely to be different from one for 'Clothing' in month 84) or from the region (an index score for 'Food' in month 84 in London is likely to be different from an index score for 'Food' in month 84 in the South East).
I think the sorting I have is correct. In the three level model (GOR top level, Category in between, Month at the lowest level), the hierarchy viewer shows 12 GORs, 13 Categories and 84 months, in the two level model (GOR at the top and Month at the bottom) it shows 12 GORS and 1092 months (1092 being 13 by 84). Even without considering the 2-level model, the results of the 3-level look wrong to me, there should be some variance arising from GOR, as the index scores are not uniform across different regions.
Re: Variance disappearing in shift from 2- to 3-level model
Hi Matt,
I am still a little puzzled. If as you say there are 13 categories for each GOR then a nested 3 level model should have 12 level 3 with 12*13 categories = 156 level 2 units. The only way that categories should be 13 would be if you had a cross-classified model. Am happy to take a look if you want to send me the dataset.
I am also puzzled as to why your '3 level model' is not really a 4 level model with observations at the bottom level? I don't understand how your 2 models have differing numbers of level 1 units. Perhaps you are quoting the range of the values e.g. 1..13 rather than total 156?
Best wishes,
Bill.
I am still a little puzzled. If as you say there are 13 categories for each GOR then a nested 3 level model should have 12 level 3 with 12*13 categories = 156 level 2 units. The only way that categories should be 13 would be if you had a cross-classified model. Am happy to take a look if you want to send me the dataset.
I am also puzzled as to why your '3 level model' is not really a 4 level model with observations at the bottom level? I don't understand how your 2 models have differing numbers of level 1 units. Perhaps you are quoting the range of the values e.g. 1..13 rather than total 156?
Best wishes,
Bill.
-
Mattoftheday
- Posts: 3
- Joined: Wed Jun 20, 2012 12:30 pm
Re: Variance disappearing in shift from 2- to 3-level model
Hiya Bill,billb wrote:The only way that categories should be 13 would be if you had a cross-classified model. Am happy to take a look if you want to send me the dataset.
Right, it sounds like I'm completely misunderstanding the way my data should be structured. Which at least explains what's going wrong. I will take you up on your kind offer if that's all right, that would be very helpful. Thank you very much for this.
With thanks and best wishes,
Matt