TIMSS - plausible values

[cbokhove]

Dear sirs,

I did see a question that resembles mine (PISA, http://www.cmm.bristol.ac.uk/forum/view ... 000a7#p425) but alas no answer.

1. First plausible values. In TIMSS there are five plausible values for Maths achievement, lets call them M1-5. There is a lot of literature saying that averaging them or choosing the first one gives a good score estimate but gives wrong estimates for standard error (right?). So one procedure would be to model with M1-5 separately and then combine the results. However, based on some pointers here and there, it has been suggested I can take the P.V.'s as repeated measure as separate level in my multilevel model. Any pointers on how to do this?

2. This has to be combined with weightings. Is this 'just' a case of using the procedure in the pdf document on weightings AFTER having fit the total model.

Best wishes,

Christian

[Harvey Goldstein]

The PVs are essentially multiple imputations. These should be combined using what ar eknown as Rubins rules and there should be guidance from TIMSS on this. You cannot treat them in any sense as a 2 level structure. If you want to do multilevel multiple imputation using individual records you can try realcom impute
Harvey Goldstein

[cbokhove]

Thanks for your response. It's clear now how I have to combine 5 separate models using Rubin's rules. I think I am right in assuming that this will have to be done outside MLwin (or write a macro for it). With TIMSS data there are two more issues. One is weights. That is clear following the separate pdf on weighting in MLwin. A second one is the jackknife procedure, pairing schools. I have read in at least one article that taking into account the hierrachical nature of data, as done in a multilevel model, provides a good estimate for the jackknife (and BRR) procedure. However, I'd rather have more confirmations as with plausible values I've seen almost everything suggested (average them and use, take the first, consider them repeated measures). Is there any information on jackknife procedures with MLwin?

[Fiunrekio]

In large-scale assessments such as TIMSS and PISA, plausible values (PVs) should be treated as multiple imputations rather than averaged or analyzed as repeated measures. Each plausible value represents a random draw from a student’s posterior ability distribution, so the correct procedure is to run the multilevel model separately for each plausible value and then combine the parameter estimates using Rubin’s rules.