FAQs (frequently-asked questions) about the LEMMA course

• Is the course difficult?
• What topics are covered?
• What statistical software is used in the practicals?
• My Names window looks different to the screenshots in the training materials and I can't find all the buttons mentioned in the instructions
• Do I need to have a strong background in statistics?
  • Are you a beginner in statistics?
    • Preliminary reading
    • Online resources
    • Books
  • Are you an experienced quantitative researcher?
• I have attended one of the Centre for Multilevel Modelling workshops. Do the online materials cover the same topics?
• I am planning to attend on of the Centre for Multilevel Modelling workshops. Should I work through these materials first?
• How can I test my understanding of the material?
• I do not have MLwiN, how can I do the practical exercises?
• I do not use MLwiN, will I still find these materials useful?
• Do I have to pay to access the materials?
• Why do I need to register?
• Can I use the materials in my own teaching?
• How should I cite or reference the different modules??
• Who designed the materials?
• Why can't we answer questions about the content of the materials?
• How do I get to the course?
• When I try to open the worksheets for the LEMMA course I get an error message
• When I double click on one of the worksheets for the LEMMA course, it opens with notepad not MLwiN
• Help!, I still have a question not answered here

Is the course difficult?

Whether you are new to statistical modelling or an advanced user, we hope that you will find our materials useful. We recommend you test your current understanding of statistics by taking our prerequisites quiz.

• If you find the prerequisites quiz questions difficult, we suggest that you refresh your knowledge by working through Module 1 (Using quantitative data in research) and Module 2 (Introduction to quantitative data analysis)
• If you find the pre-requisites quiz a little tricky, particularly the later questions, then we suggest that you start with Module 3 (Multiple regression)
• If you find the pre-requisites quiz easy, then you probably do not need to study Modules 1 and 2 or the Concepts part of Module 3.
  • If you do not have any experience with the software package you will be using for the practicals (your choice of MLwiN, R or Stata), then we recommend that you work through the Practical section of Module 3 for that software package, to familiarise yourself with how it works.
  • Otherwise: (if you do have experience with the software):
    • If you have little or no experience of multilevel modelling, start at Module 4 (Multilevel Structures and Classifications).
    • If you are experienced in multilevel modelling but have never worked with discrete (categorical) response variables, start at Module 6 (Regression Models for Binary Responses).
    • If you are experienced in multilevel modelling and discrete response models, but have never combined these to work with multilevel discrete response models, you may want to start at Module 7 (Multilevel Models for Binary Responses).
• If you are familiar with multilevel models for binary responses, you may wish to start at Module 9 (Single-level and Multilevel Models for Ordinal Responses)

What statistical packages are used in the practicals?

MLwiN, R and Stata. As the MLwiN, R and Stata versions of each practical cover the same material, comparing across the practicals will help users to become familiar with the other package.

My Names window looks different to the screenshots in the training materials and I can't find all the buttons mentioned in the instructions

• I'm using MLwiN v2.18 and working through your LEMMA online training materials. What I see in the Names window looks different to the screenshot given. When the instructions ask me to press Categories I don't know what to do because I can't see this button, and when they ask me to press Copy I don't know what to do because there are two Copy buttons.
  • The training materials and manuals available on our main website and as part of our LEMMA online learning materials were all created using earlier versions of MLwiN than v2.18. In MLwiN v2.18 there were some changes to the Names window. ( Further details…)
    
    The changes affect the appearance of the Names window; some buttons have changed their location, and some buttons have also changed their name. There are also some new features available in the Names window; there is no loss of features compared to earlier versions (every facility offered by the Names window in v2.10 to v2.17 is also available in v2.18).
    
    Since the training materials use screenshots and instructions based on the earlier versions, users should be aware that:
    
    
    • screenshots of the Names window will look slightly different to what they will see when using v2.18
    • instructions may ask them to press a button which does not exist under that name in v2.18 (for example Categories which is now View under Categories), or refer to a button name that appears in two different places (for example Copy which now appears both under Data and Categories)
    • when working with category labels instructions may take users through a process that is slightly more laborious than the easiest way to achieve the same effect in v2.18

We do intend to update many of our materials, in particular the manuals and the LEMMA online learning materials. However this is a fairly large undertaking and it is anticipated that this will not happen for several months. In the meantime we ask users to be aware of these small differences between the latest version of MLwiN and the training materials, and apologise for any confusion resulting from this.

What is the target audience for the materials? Do I need to have a strong background in statistics?

Are you a beginner in statistics?

This set of modules is not aimed at the complete beginner in quantitative analysis. We rather expect most users will have some familiarity with many basic ideas and have some prior experience of traditional elementary statistical methods, perhaps up to two variable regression and correlation methods. They will usually also have had some exposure to introductory inference such as the ideas of confidence intervals and hypothesis testing.

However, it may be that someone who has received such training may need their ideas refreshing. It may also be the case that the ideas of statistical uncertainty and inference were not fully understood on the first exposure. Module 1 and Module 2 therefore provide selective introductions to quantitative research and data analysis, with a focus on key topics that will help to contextualise the ultimate task of learning about multilevel models.

If you have a good understanding of multiple regression - including the treatment of categorical explanatory variables and interaction effects - you may wish to skip to Module 4. Confident users of multiple regression, but who have not used MLwiN before, should start with the practical for Module 3.

More advanced modules will be published as they are written.

Preliminary reading

Modules 1 and 2 are designed to refresh the concepts, definitions and techniques of introductory quantitative data analysis. We expect that users will already have some familiarity with many basic ideas and have some prior experience of traditional elementary statistical analysis. More in-depth treatments of the material covered in Modules 1 and 2 can be found in the following online resources and texts.

Online resources

We have come across the following sites which we think are useful. Please note, however, that they are not connected with the Centre for Multilevel Modelling.

http://www.socialresearchmethods.net/kb/contents.php
This is the Research Methods Knowledge Base and is a comprehensive web-based text book that introduces all of the topics in a typical introductory course in social research methods, including quantitative analysis and statistical inference.

cast.massey.ac.nz/collection_public.html
CAST includes several computer-based statistics textbooks that uses interactive diagrams to help teach all the statistical concepts. There are three introductory textbooks aimed at different application areas.

http://sportsci.org/resource/stats/
New original approaches to statistics for researchers: the examples are taken from exercise and sports science, but the principles apply to all empirical sciences: The site says "If you're new to stats, most of what you read here will be a new view. But even if you have done some stats, there's plenty here that's new. For example, I've discarded most details of computation, in the hope that you will get a better understanding of the concepts. Let's leave the computations to the computers."

http://davidmlane.com/hyperstat/index.html
Hyper Stat online; an online statistics book with links to other statistics resources on the web.

http://faculty.vassar.edu/lowry/webtext.html
Concepts and Applications of Inferential Statistics is a free, full-length, and occasionally interactive statistics textbook. It is a companion site of VassarStats, Web Site for Statistical Computation. The materials on this site may be freely used for any non-commercial educational purpose.

http://www.stats.gla.ac.uk/steps/home.html
The STEPS consortium has developed problem-based modules to support the teaching of Statistics in Biology, Business, Geography and Psychology. The software is freely available to educational institutions, and can be downloaded from the Web site.

surfstat.anu.edu.au/surfstat-home/surfstat-main.html
Surfstat.australia: an online text in introductory Statistics

Books

Go to our extensive guide to multilevel modelling books >>

Brief list:

• Diamond, I. and Jefferies, J. (2001) Beginning Statistics: An Introduction for Social Scientists. London: Sage.
• Fielding, J. and Gilbert, N. (2006) Understanding Social Statistics. 2nd Edition. London: Sage.
• Wright, D. (1996) Understanding Statistics: An Introduction for the Social Sciences . London: Sage.
• Abselson, R.P. (1995) Statistics as Principled Argument. New York: Lawrence Erlbaum.
• de Vaus, D. (2002) Analysing Social Data: 50 Key Problems in Data Analysis. London: Sage.
• Erickson, B.H. and Nosanchuk, T.A. (1992) Understanding Data. Buckingham: Open University Press.
• Robson, C. (2002) Real World Research: A Resource for Social Scientists and Practitioner-researchers. 2nd Edition. London: Blackwell.

Are you an experienced quantitative researcher?

If you have a good understanding of multiple regressio - including the treatment of categorical explanatory variables and interaction effects - you may wish to skip to Module 4. If you do not feel fully confident in the application and interpretation of multiple regression, we recommend that you at least answer the quiz questions for Module 3 to test your understanding.

If you are a confident user of multiple regression, but have not used MLwiN before, you should start with the practical for Module 3.

I have attended one of the centre for multilevel modelling workshops. Do the online materials cover the same topics?

Module 4 and Module 5 cover the same material as the first part of our workshops, although in more detail and using different examples. We therefore strongly recommend working through these modules after attending the workshop to consolidate what you learnt before attempting to analyse your own data.

Further modules will be added as they are developed. These will include additional methods-based materials (e.g. single-level and multilevel logistic regression), as well as applications (e.g. the use of multilevel models in school effectiveness research).

I am planning to attend one of the centre for multilevel modelling workshops. Should I work through these materials first?

Our workshops assume an understanding and familiarity with the application of multiple regression. If you have not used multiple regression, or feel in need of a refresher, we strongly recommend that you work through Module 3 before the workshop. For those with less experience in statistical analysis, you may additionally find it helpful to study Module 1 and Module 2.

To get the maximum benefit from the workshop, it would be worthwhile reading Module 4 and Module 5 (Concepts) beforehand. If time is short, Lessons 5.1 and 5.2 of Module 5 provide an introduction to multilevel modelling.

How can I test my understanding of the material?

You will find quiz questions interspersed throughout the modules to allow you to assess your understanding of the material.

Before starting to work through the materials, we strongly encourage you to test your current understanding of statistics by taking our prerequisites quiz. On the basis of your score in this quiz, a recommendation will be given on which module to start with.

I do not have a copy of MLwiN. How can I do the practical exercises?

Download our LEMMA MLwiN training version (msi, 85.0 mb) for free. For other options such as using the full software where you can add your own data,: please see MLwiN downloads.

I do not use MLwiN. Will I still find the materials useful?

Yes, much of the material can be used without software, or by users of software other than MLwiN.

Modules 1, 2 and 4 do not have practical exercises, and are therefore not tied to any particular software package.

In Concepts, we describe the statistical models using illustrative examples from a range of disciplines; this is done without reference to any software. The Practical component goes through the analysis of a particular data set using MLwiN R and Stata and the modelling techniques described in the concepts component of the module.

Do I have to pay to access the materials?

No. The materials can be accessed free-of-charge and downloaded by anyone. However, we do require users to register with us first.

During registration, we collect:

• your name and contact information, including e-mail address.
• information regarding your personal or professional interests, statistical and academic background, demographics, and experience with our products and services.

We also collect information about your use of and progress through the course, to help us in our research, and to help our funders evaluate the course.

More information is available in "Why do I need to register?", and in our site's Privacy Statement.

Why do I need to register?

We require you to register sothat we can collect data to help us to conduct and publish research into the learning of multilevel methodology and applications. We hope to study learning trajectories:

• use of our course, and
• progress (measured by Quiz scores) in our course

...and relate it to our users' statistical and academic background.

However, your personal data will be held and processed in strict confidence by Centre for Multilevel Modelling researchers. We uphold UK data protection laws, and much of our long professional experience has been in the analysis of sensitive educational and health data. Data will be highly aggregated before any publication, so that individual persons cannot be identified. We also collect course users' demographic, academic background, and contact details on behalf of our project funders - the ESRC (through the NCRM - National Centre for Research Methods ). We pass this data onto them so that they can evaluate our course, particularly assessing its uptake. They may also contact you in order to ask you for feedback. You have no obligation to respond to their requests.

More information is available in our Privacy Statement.

Can I use the materials in my own teaching?

YES!

Learning materials on our site are free to use under a Creative Commons licence. We will be delighted for you to use these materials for your own (non-commercial) teaching, and ask that you cite us if you do so.

How should I cite or reference the different LEMMA modules?

These learning materials were created as part of a research grant and we therefore ask that you cite them if you found them useful.

Your citation should include:

• the module's author
• the module number and name
• the part of the module (Concepts, Software Practical or Quiz)
• "LEMMA VLE", the name of the project deliverable this was funded by
• "University of Bristol, Centre for Multilevel Modelling"
• a link to our web site

For example:

Steele, F. (2008) Module 3: Multiple Regression MLwiN Practical. LEMMA VLE, University of Bristol, Centre for Multilevel Modelling. Accessed at https://www.cmm.bris.ac.uk/lemma/.

Steele, F. (2010) Module 5: Introduction to Multilevel Modelling Concepts. LEMMA VLE, University of Bristol, Centre for Multilevel Modelling. Accessed at https://www.cmm.bris.ac.uk/lemma/.

Steele, F. (2010) Module 5: Introduction to Multilevel Modelling - MLwiN Practical. LEMMA VLE, University of Bristol, Centre for Multilevel Modelling. Accessed at https://www.cmm.bris.ac.uk/lemma/.

Leckie, G. (2010) Module 5: Introduction to Multilevel Modelling - Stata Practical. LEMMA VLE, University of Bristol, Centre for Multilevel Modelling. Accessed at https://www.cmm.bris.ac.uk/lemma/.

Szmaragd, C., & Leckie, G. (2011): Module 5: Introduction to Multilevel Modelling - R Practical. LEMMA VLE, University of Bristol, Centre for Multilevel Modelling. Accessed at https://www.cmm.bris.ac.uk/lemma/.

or if citing the entire course:

Centre for Multilevel Modelling (2019): Multilevel Modelling Course. LEMMA VLE, University of Bristol, Centre for Multilevel Modelling. Accessed at https://www.cmm.bris.ac.uk/lemma/

Touch base, give feedback, get support

Do please let us know that you're using our materials (email: lemma-help@bristol.ac.uk) - we're keen to support teachers of Multilevel Modeling, and feedback is always welcome!

Adapt the materials

Please also feel that you can adapt our materials to suit your needs. We've designed in a split between Concepts (generic information) and Practicals (examples using specific software and datasets from particular disciplines), so that there's less that you'll have to change.

Let us know (email: lemma-help@bristol.ac.uk) which Practical you intend to adapt, and we'll send you its original Microsoft Word file if required.

...then share them here?

We hope that you will create different versions of the Practicals that we can put on the site - publicising your expertise, and drawing the attention of more people to it as the site becomes useful to wider communities. We hope that you'll make versions that use:

• examples and data from other disciplines, and
• instructions and output from other software.

To ensure consistency, we may require some changes to your version before we can put it up on the site, and we do not guarantee that we will be able to host your version on our site.

Who designed the system?

The learning materials are kept in a course management system called Moodle. Moodle is an open source community project, and so has been designed by many people.

The curriculum design and key requirements for the site were by Fiona Steele, supported by colleagues, including: Jon Rasbash, Kelvyn Jones, Harvey Goldstein, Aileen Earle.

The individual learning materials are by the named authors at the start of each Module, including Fiona Steele, Tony Fielding and Rebecca Pillinger, and Jon Rasbash. Materials were edited by Fiona Steele, and sub-edited by Rebecca Pillinger.

Hilary Browne was the web-developer - doing most of the visual design, all of the CSS and HTML hacking, and key parts of the interaction design.

Sacha Brostoff did most of the navigation design, interaction design and usability testing, and a little of the visual design.

Chris Charlton is the system administrator: doing all the php, database, software customisation and web-server magic that put the course on the internet, bent Moodle to our needs, and kept it working.

Help

Unfortunately we do not have resources to answer queries about the content of the materials.

However, go to our help form if you have questions about or problems with using the system, bug reports, or comments on materials.

Or email us on lemma-help@bristol.ac.uk to let us know you're teaching with them, or to let us know you would like to put materials on the site.

Why can't we answer questions about the content of the materials?

The materials are very detailed and designed for self-learning. In particular, you can test your understanding of the materials by answering the quiz questions in each lesson of a module. You can retake the quiz as many times as you need to, to understand.

If these materials don't work for you, it may be that brushing up your statistical foundations will help. There are links to prerequisites throughout the course. If this doesn't help, you may need some face-to-face time with someone knowledgeable in multilevel methods. There are workshops several times a year in the UK, and some of them are workshops given by us.

How do I get to the course?

One easy way is to select this button

When I try to open the worksheets for the LEMMA course I get an error message

Q: I'm taking the LEMMA Course and until yesterday everything was working perfectly, but today I just couldn't open the MLwiN Datafiles. Every time I try I get this message: "Run-time error '5': Invalid procedure call or argument". I've already install the latest version but it didn't work. I tried several times uninstalling and reinstalling the software, but it didn't work… Any ideas of what can I do? Many thanks in advance.

• A: Has anything on your system changed between when you were able to open the worksheets and now? Examples might be other software being installed or removed. It might be worth manually clearing the MLwiN installation directory after removing it, before attempting to reinstall it. This is the directory where you chose to install it (usually C:\Program Files\MLwiN...). If you have an account with administrative privileges on that machine it might also be worth checking to see if you have the same problems when running as that user.
  • As I don't have administrator rights in my computer, it is a pain each time I need to install a software. Finally I could open the files. As I realized I can open the old example files, what I did was to save the files in my computer and then change the extension from '.wsz' to '.ws' I know it is not the best, but it worked.

When I double click on one of the worksheets for the LEMMA course, it opens with notepad not MLwiN

When I click on the icon for 5.1.wsz in Module 5.1 I seem to get a notepad file that has a lot of blank characters, not a data file.

• First note that the datasets for the LEMMA course will not open with MLwiN version 2.02: you will need the latest version, or the teaching version ( which you can download for free here). Make sure that you have one of these versions of MLwiN. If you do have the appropriate version but are experiencing this problem, then first try reinstalling your copy of MLwiN. If this fails, you can try downloading again and then reinstalling. If that doesn't work or if you cannot do this, there are two options: a work around or a fix.
  
  Option 1: Work around
  When you select the link to the datafile, choose Save to computer instead of Open. Then start MLwiN and open the data file from MLwiN's File menu.
  
  Option 2: Fix
  The fix is to reassociate the data file's .wsz extension with MLwiN (so that Windows opens them in MLwiN if you select them, or if you choose Open instead of Save to computer when downloading), which is pretty easy and takes just a minute.
  Again, when you select the link to the datafile in your browser choose Save to computer rather than Open, and save for example to My Documents. Then open up My Documents or whichever folder you have saved the file into in Windows Explorer - so you can see the data file's icon. If the cause of the problem with opening the files is the situation this fix is designed for, then it should not have the MLwiN logo on it. (If the data file does have the MLwiN logo then there is probably a different cause of the problem and this fix will probably not work).
  The next part of these instructions is written correctly for Windows XP, but probably something similar will work for Windows Vista. Right click on the data file (i.e. click with the right mouse button, not the left one that people usually use), and choose the menu option Open With, then in the submenu select Choose Program. You should get a new window with a list of programs and a tick box labelled "Always use the selected program to open this kind of file" WHICH YOU SHOULD TICK, then select MLwiN, then select OK. If MLwiN is not in the list of programs, select the Browse button and locate and open the "mlwin.exe" file. It would normally be in the folder C:\Program Files\MLwiN v2\ or C:\Program Files\MLwiN... (Note that the icon for the program should be pale blue and white, not yellow and dark blue: the program with the yellow and dark blue icon is version 2.02 which will not open the LEMMA datafiles).
  After doing this, the data file on your computer should now have the MLwiN logo, and when you double click it should open up in MLwiN, and when you open files directly from your browser, they should open in MLwiN too.
  • I didn't reinstall, but I tried the work around (didn't work), and the fix (which did).