Log into Moodle

LEMMA: Multilevel Modelling online course

Our on-line multilevel modelling course is now available: 'LEMMA' (The Learning Environment for Multilevel Methodology and Applications) contains a set of graduated modules starting from an introduction to quantitative research progressing to multilevel modelling of continuous data.

"A great course. Just what I needed. Explained everything in detail. The tests were particularly good as they highlighted aspects you thought you understood but hadn't really grasped. A great way to get back into stats. Thanks" (PhD student, 2009)

Log on or register to course

Whether you are new to statistical modelling or an advanced user, we hope that you will find our materials useful. We assume that users have attended at least an introductory statistics course in the past, but Modules 1 and 2 are provided as refreshers.  More experienced quantitative researchers may wish to skip to Modules 3 or 4.

Before starting to work through the materials, we strongly encourage you to test your current understanding of statistics by taking our prerequisites quiz. If you find the quiz questions difficult, we suggest that you refresh your knowledge by working through Modules 1 and 2. On the other hand, those who find the questions easy may prefer to go straight to Module 3 (Multiple regression) or Module 4 (Multilevel structures and classifications). You will also find quiz questions throughout the modules to allow you to assess your understanding of the material.

The system is under continuous development, and will be extended to include further modules on advanced multilevel modelling and applications of multilevel modelling to data from different contexts.  Other planned enhancements include overview videos for each module and a more extensive glossary.

Log in or register for the course.

help icon FAQs or frequently-asked questions about the LEMMA course

What topics are covered?

There are currently seven modules available:

  1. Using quantitative data in research
  2. Introduction to quantitative data analysis
  3. Multiple regression
  4. Multilevel structures and classifications
  5. Introduction to multilevel modelling
  6. Regression Models for Binary Responses
  7. Multilevel Models for Binary Responses

Further modules will be added as they are developed. 

1. Using quantitative data in research: concepts and definitions

In Module 1 we look at quantitative research and how we collect data, in order to provide a firm foundation for the analyses covered in later modules. The aims of Module 1 are:

2. Introduction to quantitative data analysis

The aim of this module is to give an overall view of some the principles of effective data analysis. The focus is on how we summarise data to uncover patterns of explanation and relationships between variables. Some key statistical vocabulary is introduced and concepts are illustrated by example. You will learn about the following:

3. Multiple Regression

Multiple regression is a technique used to study the relationship between an outcome variable and a set of explanatory or predictor variables.  Module 3 covers the following topics:

The ideas are illustrated in analyses of hedonistic attitudes in Europe (using data from the European Social Surveys) and of trends in educational attainment (using data from the Scottish Youth Cohort Study).

4. Multilevel structures and classifications

Multilevel modelling is designed to explore and analyse data that come from populations which have a complex structure.  This module aims to introduce:

5. Introduction to Multilevel Modelling

In the social, medical and biological sciences multilevel or hierarchical structures are the norm.   Examples include individuals nested within geographical areas or institutions (e.g. schools or employers), and repeated observations over time on individuals.  Other examples of hierarchical and non-hierarchical structures were given in Module 4.  When individuals form groups or clusters, we might expect that two randomly selected individuals from the same group will tend to be more alike than two individuals selected from different groups.  For example, children learn in classes and features of their class, such as teacher characteristics and the ability of other children in the class, are likely to influence a child's educational attainment.  Because of these class effects, we would expect test scores for children in the same class to be more alike than scores for children from different classes. Multilevel models – also known as hierarchical linear models, mixed models, random effects models and variance components models – can be used to analyse data with a hierarchical structure. Throughout this module we refer to the lowest level of observation in the hierarchy (e.g. student) as level 1, and the group or cluster (e.g. class) as level 2.

Module 5 will cover the following topics:

The ideas are illustrated in analyses of hedonistic attitudes in 20 European countries (using data from the European Social Surveys) and of between-school variation in trends in students' educational attainment (using data from the Scottish Youth Cohort Study).  The same datasets are analysed in Module 3 using multiple regression, ignoring country and school effects respectively.  In this module, we emphasise the substantive insights that can be gained from a multilevel modelling approach.

6. Regression Models for Binary Responses

In Module 3 we considered multiple linear regression models for the relationship between a continuous response variable and a set of explanatory variables which may be continuous or categorical.  Regression models need to be adapted to handle categorical response variables and, in this module, we consider methods for a particular type of categorical variable: binary or dichotomous responses, that is variables with only two categories.  Module 6 covers the following topics:

The ideas are illustrated in analyses of voting intentions in the 2004 US general election (using data from the National Annenberg Election Study) and uptake of antenatal care in Bangladesh (using data from the 2004 Bangladesh Demographic and Health Survey).

7. Multilevel Models for Binary Responses

In Module 6 we saw how multiple linear regression models for continuous responses can be generalised to handle binary responses.  In this module, we consider how these methods can be extended for the analysis of clustered binary data.  We show that many of the extensions to the basic multilevel model introduced in Module 5 – for example random slopes and contextual effects – apply also to binary responses.  However, there are some important new issues to consider in the interpretation and estimation of multilevel binary response models. Module 7 covers the following topics:

The ideas are illustrated in analyses of voting intentions in the 2004 US general election (using data from the National Annenberg Election Study) and uptake of antenatal care in Bangladesh (using data from the 2004 Bangladesh Demographic and Health Survey).

What is the target audience for the materials?  Do I need to have a strong background in statistics?

Are you a beginner in statistics?

This set of modules is not aimed at the complete beginner in quantitative analysis. We rather expect most users will have some familiarity with many basic ideas and have some prior experience of traditional elementary statistical methods, perhaps up to two variable regression and correlation methods. They will usually also have had some exposure to introductory inference such as the ideas of confidence intervals and hypothesis testing.

However, it may be that someone who has received such training may need their ideas refreshing. It may also be the case that the ideas of statistical uncertainty and inference were not fully understood on the first exposure. Module 1 and Module 2 therefore provide selective introductions to quantitative research and data analysis, with a focus on key topics that will help to contextualise the ultimate task of learning about multilevel models.

If you have a good understanding of multiple regression – including the treatment of categorical explanatory variables and interaction effects – you may wish to skip to Module 4. Confident users of multiple regression, but who have not used MLwiN before, should start with the practical for Module 3.

More advanced modules will be published as they are written.

Preliminary reading

Modules 1 and 2 are designed to refresh the concepts, definitions and techniques of introductory quantitative data analysis.  We expect that users will already have some familiarity with many basic ideas and have some prior experience of traditional elementary statistical analysis.  More in-depth treatments of the material covered in Modules 1 and 2 can be found in the following on-line resources and texts.

Online resources

We have come across the following sites which we think are useful.  Please note, however, that they are not connected with the Centre for Multilevel Modelling.

http://www.socialresearchmethods.net/kb/contents.php
This is the Research Methods Knowledge Base and is a comprehensive web-based text book that introduces all of the topics in a typical introductory course in social research methods, including quantitative analysis and statistical inference.

http://cast.massey.ac.nz/login.html
CAST includes several computer-based statistics textbooks that uses interactive diagrams to help teach all the statistical concepts. There are three introductory textbooks aimed at different application areas.

http://sportsci.org/resource/stats/
New original approaches to statistics for researchers: the examples are taken from exercise and sports science, but the principles apply to all empirical sciences: The site says "If you're new to stats, most of what you read here will be a new view. But even if you have done some stats, there's plenty here that's new. For example, I've discarded most details of computation, in the hope that you will get a better understanding of the concepts. Let's leave the computations to the computers."

http://davidmlane.com/hyperstat/index.html
Hyper Stat online; an online statistics book with links to other statistics resources on the web.

http://faculty.vassar.edu/lowry/webtext.html
Concepts and Applications of Inferential Statistics is a free, full-length, and occasionally interactive statistics textbook. It is a companion site of VassarStats, Web Site for Statistical Computation. The materials on this site may be freely used for any non-commercial educational purpose.

http://www.stats.gla.ac.uk/steps/home.html
The STEPS consortium has developed problem-based modules to support the teaching of Statistics in Biology, Business, Geography and Psychology. The software is freely available to educational institutions, and can be downloaded from the Web site.

http://www.anu.edu.au/nceph/surfstat/surfstat-home/surfstat.html
Surfstat.australia: an online text in introductory Statistics

Books

Diamond, I. and Jefferies, J. (2001)  Beginning Statistics: An Introduction for Social Scientists. London: Sage.

Fielding, J. and Gilbert, N. (2006)  Understanding Social Statistics. 2nd Edition. London: Sage.

Wright, D. (1996) Understanding Statistics: An Introduction for the Social Sciences . London: Sage.

Abselson, R.P. (1995) Statistics as Principled Argument. New York: Lawrence Erlbaum.

de Vaus, D. (2002) Analysing Social Data: 50 Key Problems in Data Analysis. London: Sage.

Erickson, B.H. and Nosanchuk, T.A. (1992) Understanding Data. Buckingham: Open University Press.

Robson, C. (2002) Real World Research: A Resource for Social Scientists and Practitioner-researchers. 2nd Edition. London: Blackwell.

Other useful multilevel modelling books

Are you an experienced quantitative researcher?

If you have a good understanding of multiple regression – including the treatment of categorical explanatory variables and interaction effects – you may wish to skip to Module 4.  If you do not feel fully confident in the application and interpretation of multiple regression, we recommend that you at least answer the quiz questions for Module 3 to test your understanding.

If you are a confident user of multiple regression, but have not used MLwiN before, you should start with the practical for Module 3.

I have attended one of the Centre for Multilevel Modelling workshops.  Do the online materials cover the same topics?

Modules 4 and 5 cover the same material as the first part of our workshops, although in more detail and using different examples.  We therefore strongly recommend working through these modules after attending the workshop to consolidate what you learnt before attempting to analyse your own data.

Further modules will be added as they are developed.  These will include additional methods-based materials (e.g. single-level and multilevel logistic regression), as well as applications (e.g. the use of multilevel models in school effectiveness research).

I am planning to attend one of the Centre for Multilevel Modelling workshops.  Should I work through these materials first?

Our workshops assume an understanding and familiarity with the application of multiple regression.  If you have not used multiple regression, or feel in need of a refresher, we strongly recommend that you work through Module 3 before the workshop.  For those with less experience in statistical analysis, you may additionally find it helpful to study Modules 1 and 2.

The workshops do not assume any prior knowledge of multilevel modelling or MLwiN.  The first part of the workshop will cover much of the same material as Modules 4 and 5, although in less detail and using different examples.  To get the maximum benefit from the workshop, it would be worthwhile reading Module 4 and Module 5 (Concepts) beforehand.  If time is short, Lessons 5.1 and 5.2 of Module 5 provide an introduction to multilevel modelling.

How can I test my understanding of the material?

You will find quiz questions interspersed throughout the modules to allow you to assess your understanding of the material.

Before starting to work through the materials, we strongly encourage you to test your current understanding of statistics by taking our prerequisites quiz. On the basis of your score in this quiz, a recommendation will be given on which module to start with.

I do not have a copy of MLwiN.  How can I do the practical exercises?

You can use our LEMMA MLwiN training version or the latest version of MLwiN (MLwiN 2.02 is not compatible with the training materials). There are several options, please see our MLwiN download page for further details.

I do not use MLwiN. Will I still find the materials useful?

Yes, much of the material can be used without software, or by users of software other than MLwiN.

Modules 1, 2 and 4 do not have practical exercises, and are therefore not tied to any particular software package.

Modules 3 (Multiple Regression) and Module 5 (Introduction to Multilevel Modelling) each consist of two integrated components: concepts and practicals. In ‘concepts’, we describe the statistical models using illustrative examples from a range of disciplines; this is done without reference to any software. The ‘MLwiN practical’ component goes through, in detail, the analysis of a particular data set with MLwiN using the modelling techniques described in the Concepts component of the module.

Users of other software packages could use the 30-day evaluation version or free-to-UK academics versions of MLwiN to output data in SPSS or Stata format. In the future, we hope to extend our materials to include practical instructions for other packages.

Do I have to pay to access the materials?

No. The materials can be accessed free-of-charge and downloaded by anyone. However, we do require users to register with us first.

During registration, we collect:

We also collect information about your use of and progress through the course, to help us in our research, and to help our funders evaluate the course.

More information is available in "Why do I need to register?", and in our site's Privacy Statement.

Why do I need to register?

We require you to register mostly so we can collect data to help us to conduct and publish research into the learning of multilevel methodology and applications. We hope to study learning trajectories:

...and relate it to our users' statistical and academic background.

However, your personal data will be held and processed in strict confidence by Centre for Multilevel Modelling researchers. We uphold UK data protection laws, and much of our long professional experience has been in the analysis of sensitive educational and health data. Data will be highly aggregated before any publication, so that individual persons cannot be identified. We also collect course users' demographic, academic background, and contact details on behalf of our project funders – the ESRC (through the NCRM - National Centre for Research Methods ). We pass this data onto them so that they can evaluate our course, particularly assessing its uptake. They may also contact you in order to ask you for feedback. You have no obligation to respond to their requests.

More information is available in our Privacy Statement.

Can I use the materials in my own teaching?

YES! 

Learning materials on our site are free to use under a Creative Commons licence. We will be delighted for you to use these materials for your own (non-commercial) teaching, and ask that you cite us if you do so.

Your citation should include:

For example:

Steele, F. (2008) Module 3: Multiple Regression MLwiN Practical.  LEMMA VLE, Centre for Multilevel Modelling.  Accessed at  http://www.cmm.bris.ac.uk/lemma/course/view.php?id=13.

Touch base, give feedback, get support

Do please let us know that you're using our materials (email: lemma-help@bristol.ac.uk) – we're keen to support teachers of Multilevel Modeling, and feedback is always welcome!

Adapt the materials

Please also feel that you can adapt our materials to suit your needs. We've designed in a split between Concepts (generic information) and Practicals (examples using specific software and datasets from particular disciplines), so that there's less that you'll have to change.

Let us know (email: lemma-help@bristol.ac.uk) which Practical you intend to adapt, and we'll send you its original Microsoft Word file if required. 

...then share them here?

We hope that you will create different versions of the Practicals that we can put on the site - publicising your expertise, and drawing the attention of more people to it as the site becomes useful to wider communities. We hope that you'll make versions that use:

To ensure consistency, we may require some changes to your version before we can put it up on the site, and we do not guarantee that we will be able to host your version on our site.

Who designed the system?

The learning materials are kept in a course management system called Moodle. Moodle is an open source community project, and so has been designed by many people.

The curriculum design and key requirements for the site were by Fiona Steele, supported by colleagues, including: Jon Rasbash, Kelvyn Jones, Harvey Goldstein, Aileen Earle.

The individual learning materials are by the named authors at the start of each Module, including Fiona Steele, Tony Fielding and Rebecca Pillinger, and Jon Rasbash. Materials were edited by Fiona Steele, and sub-edited by Rebecca Pillinger.

Hilary Browne was the web-developer - doing most of the visual design, all of the CSS and HTML hacking, and key parts of the interaction design.

Sacha Brostoff did most of the navigation design, interaction design and usability testing, and a little of the visual design.

Chris Charlton is the system administrator: doing all the php, database, software customisation and web-server magic that put the course on the internet, bent Moodle to our needs, and kept it working.

Contact us

Unfortunately we do not have resources to answer queries about the content of the materials. 

However, go to our LEMMA-help enquiry form if you have questions about or problems with using the system, bug reports, or comments on materials.

Or email us on lemma-help@bristol.ac.uk to let us know you're teaching with them, or to let us know you would like to put materials on the site.

Why can't we answer questions about the content of the materials?

The materials are very detailed and designed for self-learning. In particular, you can test your understanding of the materials by answering the quiz questions in each lesson of a module. You can retake the quiz as many times as you need to, to understand.

If these materials don't work for you, it may be that brushing up your statistical foundations will help. There are links to prerequisites throughout the course. If this doesn't help, you may need some face-to-face time with someone knowledgeable in multilevel methods. There are workshops several times a year in the UK, and some of them are workshops given by us.

How do I get to the course?

Click:  Log into Moodle

After you have opened the page, you can add it to your web-browser's Favourites or Bookmarks.

When I try to open the worksheets for the LEMMA course I get an error message

Q: I'm taking the LEMMA Course and until yesterday everything was working perfectly, but today I just couldn't open the MLwiN Datafiles. Every time I try I get this message: "Run-time error '5': Invalid procedure call or argument". I've already install the latest version but it didn't work. I tried several times uninstalling and reinstalling the software, but it didn't work… Any ideas of what can I do? Many thanks in advance.

When I double click on one of the worksheets for the LEMMA course, it opens with notepad not MLwiN

When I click on the icon for 5.1.wsz in Module 5.1 I seem to get a notepad file that has a lot of blank characters, not a data file.