presentations with videos
'LEMMA'
Learning Environment for
Multilevel Methodology and Applications
April 2005 - September 2008
See also Lemma II (Lemma continued)
Multilevel Modelling on-line learning course
Topics covered
- Using quantitative data in research
- Introduction to quantitative data analysis
- Multiple regression
- Multilevel structures & classifications
- Introduction to multilevel modelling
- Regression Models for Binary Responses
- Multilevel Models for Binary Responses
Target audience
- Anyone with an interest in multilevel modelling!
- From beginners to advanced quantitative researchers
- Other quantitative methods trainers
Features
- Free to registered users
- Free training version of MLwiN
- Quizzes for self-evaluation
LEMMA system design
Sacha Brostoff (Navigation & Interaction design), Hilary Browne (Visual & Interaction design), Fiona Steele, Tony Fielding, Jon Rasbash, Becky Pillinger (Content & Information design).
The Lemma Research Project
LEMMA is one of the nodes of the ESRC-funded NCRM (National Centre for Research Methods) whose mission is to provide a strategic focal point for the identification, development and delivery of an integrated national research, training and capacity-building programme.
Lemma Goals
This application seeks to establish an interdisciplinary node specialising in the analysis of data with complex structure that mirrors substantive research questions. Such complex structure includes household and family data, contextual, neighbourhood and area effects, spatial analytical models, longitudinal data structures, event-duration models, and mover-stayer models.
This node will draw on the interdisciplinary research strengths of the University of Bristol that are found across the social sciences.
The proposal has three inter-related elements.
- important and much-needed methodological developments in the specification and estimation of multilevel models which represent a further step-change in the capacity of these models to handle realistic complexity; in particular developments will be made in the analysis of non-hierarchical structures, complex dependencies between structures and latent-class models; all these developments will be implemented in appropriate software. An unrestricted version of MLwiN software is freely available to UK academics for the lifetime of the project (currently scheduled to finish on 30 September 2008)
- a variety of integrated flagship projects using this methodology to research important social science questions; these will not only serve to demonstrate the methodological developments but will also be substantively important in their own right. Additionally our experience is that synergy is important; consequently we anticipate that further methodological insights and challenges will result from tackling real-world questions;
- extensive capacity building and research training in the analysis of data with complex structure; this builds on the experience of the applicants in delivering a range of training materials both in a face-to-face format and via the web. This proposal will fund a range of workshops aimed at awareness raising and capacity building. Most importantly a multilevel modelling virtual learning environment will be established which is designed to initiate, develop, and support geographically-dispersed researchers who have a variety of needs using ‘models of learning’ that have been found to be effective for a virtual environment; it will include a carefully graded training repository (from basic to advanced) based on substantive research and a variety of data sources, as well as software, master-classes, and facilities for discussion groups and on-line networking. Focus groups and a user group formed from the ESRC constituency will be used in the design of workshops and the learning environment. The resultant training repository will become the world’s leading archive on the analysis of social-science data with complex structure.
This node works collaboratively with other nodes and the hub in ensuring an integrated approach to the development and dissemination of methodological expertise within the UK.
Selected LEMMA Presentations and Papers
See also - Other special topics (multilevel modelling and/or MLwiN)
- Residuals - an Introduction, slides with commentary
- Modelling the effect of Pupil Mobility on School differences in educational achievement
- What are the barriers to using more complex quantitative methods?
- Significance Testing: Powerpoint presentation with audio commentary and MLwiN demo
- Fitting a two-level model
- Contextual Effects: The Multilevel Latent Covariate Model: A New, More Reliable Approach to Group-Level Effects in Contextual Studies
- Multilevel Structural Equation Models for the Analysis of Comparative Data on Educational Performance
Residuals - An Introduction
Watch slide presentation with spoken commentary by Rebecca Pillinger
Modelling the effect of Pupil Mobility on School differences in educational achievement
Authors:
Download paper: Modelling the impact of Pupil Mobility on School differences in educational achievement, Journal of the Royal Statistical Society. Series A, Statistics in Society (2007) 170, Part 4, pp. 941-954
Value added analyses of school performance typically assign students to the school where they take the outcome test or examination. A particular problem with this is that this ignores the fact that many pupils move school between the time that they take the prior test or examination and the time that they take the outcome test. For example, for GCSE outcome in England where an 11-year old test score is used to adjust for prior achievement, up to 40% may have changed school in some areas. Ignoring this will tend to reduce the amount of variation estimated at the school level and distort inferences in terms of school effectiveness. The paper demonstrates this effect, using multiple membership models with data from English Local Authorities provided by the National Pupil database. It also shows that rank orderings of schools are almost unaffected when the proper analysis is carried out.
What are the barriers to using more complex quantitative methods?
Powerpoint or PDF presentation
by Sally Thomas
- Many researchers in the UK have complex questions and limited statistical modelling expertise
- This means they either use simple but inappropriate techniques that they can understand or they use complex appropriate techniques that they do not understand
- The use of simple techniques leads to the criticism that these approaches are too reductionist to ever capture social reality in any meaningful way
- The attempt to use complex techniques without the necessary background is demoralising for practitioners and leads to bad research
- But lack of suitable training and training materials
Example: Evaluating School Effectiveness in Lancashire LEA using Value Added Measures
Download Powerpoint presentation
Download same presentation in PDF form
by Sally Thomas
- UK context
- What is value added?
- Key issues in school effectiveness research
Significance Testing
Powerpoint presentation with audio commentary and MLwiN demo by Kelvyn Jones
Tests for coefficients of individual variables: eyeballing standard errors, Wald tests, and calculating p-values using the tail areas screen in MLwiN. Tests for comparing models: the Likelihood Ratio Test and the Deviance Information Criterion.
Multilevel Discrete-time Event History Analysis Workshop materials
by Fiona Steele
Powerpoint presentation, (0.5 mb)
Exercises, macros and worksheetsWorkshop data
The practical exercises used data derived from the National Child Development Study and the 1970 British Birth Cohort Study. These datasets are available for download via the UK Data Archive.
- Multilevel Event History Analysis Training Datasets - Go to data-archive web site to download
For further event history materials see Fiona Steele's project: Multilevel Multiprocess Models for Partnership and Childbearing Event Histories
Introductory 2 level MLwiN task
by Sally Thomas
This worksheet and dataset describes and introduces a simple 2 level model using educational data to examine school effectiveness. It is used as part of the course materials for the Graduate School of Education 10 credit masters unit: Advanced Quantitative Modelling Techniques in Education EDUCM5509
Contextual Effects
by Oliver Lüdtke - Max Planck Institute for Human Development, Berlin , Herbert W. Marsh - Oxford University, Alexander Robitzsch - Institute for Educational Progress, Berlin Ulrich Trautwein - Max Planck Institute for Human Development, Berlin Tihomir Asparouho - Muthén & Muthén Bengt Muthén - University of California, Los Angeles
Multillevel contextual studies look at the effects of group-level (L2; eg, schools, classes, families, neighbourhoods, etc) characteristics on individual-level (L1) measures. For example, (L2) school-level could be estimated by aggregating (L1) student-level scores to get a school-average value -- a manifest (single-indicator) measure of the corresponding latent school characteristic (eg, school-average ability or SES, or school climate) . However, the psychometric properties of the school-average value depend on the nature of the construct, the extent of agreement among students within each school, the number of students responding from each school (and perhaps the number of students in each school), and the number of schools. Particularly when the number of students within each school, the number of schools, and extent of agreement among students (intraclass correlations) is small, the manifest school-average might be a very unreliable indicator of the latent (population) value and result in biased estimates of contextual effects and standard errors.
In CFA/SEM we use multiple indicators (eg items) to infer a latent construct that is corrected for unreliability using a classical measurement and domain sampling rationale (based on the extent of agreement among multiple indicators and the number of indicators) . Here we describe -- with mathematical derivation, simulation, and selected "real data" applications -- new multilevel approaches to infer latent L2 constructs that are purged for unreliability due to sampling variability (based on the extent agreement among L1 students within each L2 school, and the number of students within each school), and critically evaluate their appropriateness in different situations.
Multilevel Structural Equation Models for the Analysis of Comparative Data on Educational Performance
Authors:Harvey Goldstein, Gérard Bonnet, Thierry Rocher Ministère de l’Education Nationale, de l’Enseignement Supérieur et de la Recherche, Direction de l’Évaluation et de la Prospective, Paris
The Programme for International Student Assessment comparative study of reading performance among 15-year-olds is reanalyzed using statistical procedures that allow the full complexity of the data structures to be explored. The article extends existing multilevel factor analysis and structural equation models and shows how this can extract richer information from the data and provide better fits to the data. It shows how these models can be used fully to explore the dimensionality of the data and to provide efficient, single-stage models that avoid the need for multiple imputation procedures. Markov Chain Monte Carlo methodology for parameter estimation is described.