Web Resources for Multilevel Modelling
Compiled by Kelvyn Jones, Myles Gould and SV Subramanian.
Their list of resources is designed as an organized meta-site, whereby other resources for multilevel modelling on the web can be accessed. A specific resource is listed only once, so you may have to search through this page to find what you want.
The multilevel mailing list is a key general resource as it is searchable; it represents many years of accumulated questions and answers.
Another vital resource is provided by the UCLA Academic Technology Services who maintain data and worked examples in a number of different software packages for a number of different multilevel textbooks.
Books and related downloads and materials
A selection here, but for a full list, go to our useful books page
A taster of Goldstein’s classic text in its 3rd edition on multilevel statistical models (Goldstein H, 2003, Multilevel statistical models, London, Arnold Publishers) is available here
A previous version of this text can also be downloaded at www.ats.ucla.edu/stat/examples/msm_goldstein/default.htm
Supplementary material for Tom Snijders and Roel Bosker textbook – Snijders T, Bosker R, 1999 Multilevel analysis: an introduction to basic and advanced multilevel modeling, London, Sage, including updates and corrections, data sets used in examples, with set-ups for running the examples in MLwiN and in HLM, and an introduction to MLwiN can be found at stat.gamma.rug.nl/snijders/mlbook1.htm
Supplementary material for Joop Hox’s textbook – Hox J, 2002, Multilevel analysis: techniques and applications, Mahwah, NJ, Lawrence Erlbaum – can be found at www.ats.ucla.edu/stat/examples/ma_hox/default.htm
The complete content of Hox J, 1995, Applied multilevel analysis,
Amsterdam: TT-Publikaties can be downloaded at
www.fss.uu.nl/ms/jh/publist/amaboek.pdf at Hox’s website
Supplementary material to Skrondal, A. and Rabe-Hesketh, S. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models. Chapman & Hall/CRC. can be found at www.gllamm.org/
Supplementary material to Garrett Fitzmaurice, Nan Laird, James Ware (2004) Applied Longitudinal Analysis Wiley is to be found at biosun1.harvard.edu/~fitzmaur/ala/
Supplementary material to Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence by Judy Singer and John Wilett (2003) Oxford University Press, can be found at: gseacademic.harvard.edu/~alda/
For those interested in the analysis of spatial data, there is Andrew B. Lawson, William J. Browne, Carmen L. Vidal Rodeiro (2003) Disease Mapping with WinBUGS and MLwiN, Wiley and its associated website http://seis.bris.ac.uk/~frwjb/dm.html
To read a comparison of multilevel modelling with traditional
approaches to running ANOVA, regression, and logistic regression with
memories/events being "nested" within people/testing session see
Wright DB, 1998, Modelling clustered data in autobiographical memory
research: the multilevel approach, Applied Cognitive Psychology,
12, 339-357 at:
www.fiu.edu/~dwright/pdf/multil.pdf
To keep up to date with developments in the field have a look at the downloadable Multilevel Newsletters
Reference lists about multilevel modelling
Wolfgang Ludwig-Mayerhofer’s annotated references on multilevel modelling: www.lrz-muenchen.de/~wlm/wlmmule.htm
There is a structured list of references (based on different types of model) at the HLM website www.ssicentral.com/hlm/references.html#r7
Software in general
The CMM maintains reviews of some of the packages available for multilevel modelling. These reviews contain syntax for fitting a range of multilevel models to example datasets.
If you want to see how a particular model can be fitted in particular software, there are the developing resources at UCLA www.ats.ucla.edu/stat/examples/
For those wishing to analyse longitudinal data, software instructions in
a wide range of programs is provided by UCLA to accompany the textbook Singer JD,
Willett JB, 2003 Applied longitudinal data analysis: modeling change and event
occurrence, New York, Oxford University Press, at:
www.ats.ucla.edu/stat/examples/alda/
Training associated with software packages
A growing amount of web-based (or at least downloadable) training materials are being developed. We have organized this section by the particular software that is being used, and rather arbitrarily separated commercial software from the freeware that follows
| Software | Training Material Description |
|---|---|
| aML | Can be used to fit a range of multilevel models but has specific features for fitting multi-process or simultaneous equation models to hierarchical data where predictor variables may be non-random or endogenous, and other types of models used by economists such as a multilevel Heckman selection model: www.applied-ml.com/product/multiprocess.html |
| HLM | The official site gives guidance at
www.ssicentral.com/hlm/examples.html There is very good introductory material on how to set up the models by Information Technology Services at the University of Texas Jason Newsom’s Multilevel Regression course that uses HLM, but covers a lot of other ground too (eg Distinguishing between random and fixed: variables, effects, and coefficients; comparison of estimators, and kinks to SPSS Mixed) |
| MLwiN | You can download a version of the software, data and training materials from our Learning Environment for Multilevel Methods and Applications site (LEMMA). These materials can be supplemented by the freely available MLwiN manuals, which are a course in themselves. A special MCMC manual discusses MCMC estimation in MLwiN in full. |
| Mplus | This software allows structural equation modeling, multilevel modelling and mixture modelling; their web site, www.statmodel.com has training downloads and examples. |
| SAS | Judy Singer has a pdf download that shows how to fit multilevel
models in PROC MIXED; it is very well written. UCLA has implemented the Singer example in other software (eg R\Splus, HLM, MLwiN, SPSS) www.ats.ucla.edu/stat/paperexamples/singer/default.htm C.J. Anderson has a lot of material for her course online at www.ed.uiuc.edu/courses/EdPsy490CK/ Data and SAS related material are available for Applied Longitudinal Analysis by Garrett Fitzmaurice, Nan Laird, James Ware at biosun1.harvard.edu/~fitzmaur/ala/ The code and data to fit the models contained in SAS System for Mixed Models (1996) by RC Littell, GA Milliken, WW Stroup, and RD Wolfinger, is to be found at http://ftp.sas.com/samples/A55235 |
| SPSS | A useful discussion of the Linear Mixed Models procedure in SPSS Advanced Models
is to be found at http://www.spss.com/downloads/Papers.cfm (search for Linear mixed-effects modeling in SPSS) also an HTML downloadable tutorial based on a set of case studies John Painter provides a clear guide on how to fit multilevel models using SPPS mixed www.unc.edu/~painter/ Another brief demonstration of SPSS Mixed in action is to be found at  step.psy.cmu.edu/materials/spss/mixed.doc |
Freeware
There are a number of programs that are available at low or nil cost; some of these are general (like R), others are more specific but can have special features that make them particularly attractive; we have tried to identify these special features below. We have also pointed to some appropriate training resources.
| Software | Description |
|---|---|
| BAYESX | Has a number of distinctive features including handling structured (correlated) and/or unstructured (uncorrelated) effects of spatial covariates (geographical data) and unstructured random effects of unordered group indicators. It allows non-parametric relationships between the response and the predictors (generalized additive models) and does this for continuous and discrete outcomes, it can manipulate and display geographical maps www.stat.uni-muenchen.de/~lang/bayesx/bayesx.html |
| BUGS | Bayesian inference Using Gibbs Sampling is really a
flexible language that allows the fitting of a very wide range
of models using MCMC methods; this is a very rich site developed
by the MRC Biostatistics Research Unit in Cambridge which has
lots of freely downloadable software and detailed manuals
www.mrc-bsu.cam.ac.uk/bugs |
| GeoBUGS | GeoBUGS is an add-on to BUGS that has been developed by a team at Imperial College to fit spatial models and produce a range of maps as output www.mrc-bsu.cam.ac.uk/bugs/winbugs/geobugs.shtml |
| GLLAMM | GLLAMM usefully undertakes multilevel latent class and
factor analysis, adapative quadrature to derive the full likelihood with
discrete and normal response, and has facilities for fitting non-parametric
models in which the distribution at the higher level can be non-normal
(you need STATA to run this software; preferably STATA 8); this software
is particularly useful for the models listed above, but can be slow
to converge. This site is also a rich one with growing number of downloads of
lectures and papers showing how the approach can be used in practice
www.gllamm.org/ You can download materials from a multilevel modelling course taught at Lancaster University which includes examples of using GLLAMM for continuous and discrete responses. |
| MIX | MIX are a set of stand-alone programs that fit a number of specific models including mixed-effects linear regression, mixed-effects logistic regression for nominal or ordinal outcomes, mixed-effects probit regression for ordinal outcomes, mixed-effects Poisson regression, and mixed-effects grouped-time survival analysis. They have a common interface, and importantly they calculate the likelihood directly so allowing comparison of the change in deviance for nested models. The are versions for Windows as well as for PowerMac and Solaris www.uic.edu/~hedeker/mix.html |
| R | R is a complete system for statistical computation and graphics, it can be seen as an Open Source implementation of the S language which in turn underlies the S-Plus software. It is distributed freely under the GNU General Public License and can be used for commercial purposes. It operates across a very wide range of platforms. The latest version and documentation can be obtained via CRAN, the Comprehensive R Archive Network cran.r-project.org/ |
| R/S | Normal-theory models are fitted in R using lme and nlme
functions described in full in ‘Mixed-effects models in S and
S-PLUS&lsquo by J. C. Pinheiro and D. M. Bates (2000),
there is an additional support for this book at
cm.bell-labs.com/cm/ms/departments/sia/project/nlme/ for discrete responses there is the function glmmPQL which is discussed in the 4th edition of Modern applied statistics with S W. N. Venables and B. D. Ripley; the book also covers normal theory models; there is online support for the book at www.stats.ox.ac.uk/pub/MASS4/ Jeff Gill maintains a website that provides help, tutorials and references for those who want to use R psblade.ucdavis.edu/ |
Useful software and macros
| Software | Description |
|---|---|
| PreML | There is a very useful utility written so as to export an SPSS file into a MLwiN worksheet, it is down-loadable from Tom Snijders' webpage stat.gamma.rug.nl/snijders/PreML.inc |
| Diagnostics | Tom Snijders’ homepage contains a set of MLwiN macros for producing diagnostics and for fitting a social network model stat.gamma.rug.nl/snijders/mlnmac.htm |
| PINT | For determining appropriate required sample sizes and power in a two-level model; there is a manual |
| OD | Downloadable software and manual are available from www.wtgrantfoundation.org/resources/overview/research_tools |
| DismapWin | is a public domain software for the statistical analysis of epidemiological maps; it allows the analysis of unobserved heterogeneity using mixture models; the program offers a Poisson regression approach which links disease and exposure data www.personal.reading.ac.uk/~sns05dab/Software.html |
| PROC TRAJ | is a SAS® procedure, written by Bob Jones, that fits a discrete mixture model to longitudinal data, and thereby implements Nagel’s group trajectory model; a very useful site for this type of model with downloads of papers is www.andrew.cmu.edu/user/bjones/ |
Experts’ Websites
Douglas Bates who developed the LME and NLME functions in R and S-plus has a website at www.stat.wisc.edu/~bates/bates.html
Bill Browne (who has made major contributions to the MCMC component of MLwiN) has a large number of downloadable papers at seis.bris.ac.uk/~frwjb/bill.html
David Draper’s home page has a lot of material about the Bayesian approach to hierarchical models www.cse.ucsc.edu/~draper/
Tony Fielding has useful material on ordered categorical variables, endogeneity and instrumental variables including MLwiN macros on his personal page
Andrew Gelman has lots of downloadable papers and presentations on multilevel modelling with a strong Bayesian flavour www.stat.columbia.edu/~gelman/
Harvey Goldstein, who is the instigator of the MLwiN software has a number of downloadable papers at his personal website
Don Hedeker who has been behind the MIX set of programs has lecture transparencies and class notes on longitudinal analysis at tigger.uic.edu/~hedeker/
Joop Hox’s webpage has papers, programs and lectures to download at www.geocities.com/joophox/
Alastair Leyland has extensively used multilevel modelling in public health
Bengt Muthen who is the developer of Mplus which allows multilevel factor analysis has a site at www.gseis.ucla.edu/faculty/muthen/muthen3.htm
Jason Newsom’s multilevel page has discussion of topics like centering, and how to distinguish between fixed and random effects www.upa.pdx.edu/IOA/newsom/mlrclass/default.htm
David Rogosa’s hompage has useful links to his course Education 260 on Popular Methods (including multilevel modeling, and causal inference) and Education 351 on Longitudinal analysis www.stanford.edu/~rag/
Steve Raudenbush’s LAMMP website has publications and pre-prints and links to the projects he is currently working on www-personal.umich.edu/~rauden/
Tom Snijders homepage stat.gamma.rug.nl/snijders/multilevel.htm
Subramanian’s research papers on using multilevel models in social epidemiology and health as well training resources related to the concepts and application of multilevel statistical methods can be found at www.hsph.harvard.edu/faculty/SVSubramanian.html
Examples of multilevel modeling
For an interesting discussion about what multilevel models can (and cannot do) see the interchange at www.stat.columbia.edu
For the use of multilevel models in social network analysis, see stat.gamma.rug.nl/snijders/socnet.htm
Tutorials in MCMC estimation
MCMC estimation is increasingly being used to estimate complex models; there are number of sites with really helpful resources to get you started:
Simon Jackman’s Estimation and Inference via Markov chain Monte Carlo: a resource for social scientists tamarama.stanford.edu/mcmc/
Jeff Gill’s homepage is a mine of information in this area, it includes some down-loadable chapters from his 2002 book Bayesian Methods for the Social and Behavioral Sciences which is to be thoroughly recommended psblade.ucdavis.edu/
There is a useful website for David Spiegelhalter, Keith Abrams and Jonathan Myles (2003) Bayesian approaches to clinical trials and health-care evaluation, Wiley; it contains downloads for the examples that use WinBugs and Excel worksheets that allow simple analysis of odds-ratio and hazard ratio models on the basis of normal priors and likelihoods www.mrc-bsu.cam.ac.uk/bayeseval/
Sujjit Sahu’s tutorial on MCMC www.maths.soton.ac.uk/staff/Sahu/utrecht/
Harold Lehmann Bayesian Communication webpage prototypes Bayesian analysis on-line www.hopkinsmedicine.org/Bayes/PrimaryPages/Index.cfm
A Brief Introduction to Graphical Models and Bayesian Networks is to be found at www.ai.mit.edu/~murphyk/Bayes/bayes.html
For software to determine sample-size requirements using prior opinion see Lawrence Joseph’s Bayesian software site www.medicine.mcgill.ca/epidemiology/Joseph/
To keep up to date in this area, you can visit the MCMC preprint service www.statslab.cam.ac.uk/~mcmc/
Causal analysis
Introductory sites
Christopher Winship’s Counterfactual Causal Analysis in Sociology website provides a good introduction to developments in this area www.wjh.harvard.edu/~cwinship/cfa.html
Harvard School of Public Health PROGRAM ON CAUSAL INFERENCE in Epidemiology and Allied Sciences www.hsph.harvard.edu/causal/index.htm
Experts on causal analysis
Judea Pearl’s home-page has a large number of downloads of lectures and papers ayes.cs.ucla.edu/jp_home.html
Guido Imbens homepage ideas.repec.org/e/pim4.html
David Harding’s homepage www-personal.umich.edu/~dharding/
Software for causal analysis with observational data
for R-based matching software which uses a wide range of techniques see Gary King’s site sekhon.berkeley.edu/matching/
there is a SPPS syntax file for propensity scoring available at John Painter’s site www.unc.edu/~painter/SPSSsyntax/propen.txt
facilities in R for Multivariate and Propensity Score Matching Software written by Jasjeet Sekhon sekhon.berkeley.edu/matching/
and Stata programs for ATT estimation based on propensity score matching www.sobecker.de/pscore.html
Multilevel modelling and causal analysis
The “Columbia group on Bayesian statistics, multilevel modelling, causal inference, and social networks” have a site at www.stat.columbia.edu/~sam/MultilevelModeling/
There are pre-prints and publications on Steve Raudenbush’s site - search for 'causal'.
Tony Fielding has material on endogeneity and instrumental variables including MLwiN macros, at www.economics.bham.ac.uk/people/fieldingt.htm