Centre for Multilevel Modelling

• Team
  • William Browne
  • Christopher Charlton
  • Harvey Goldstein
    • Harvey Goldstein's publications
    • Multilevel Statistical Models - third edition
    • Multilevel Statistical Models - second edition
  • George Leckie
  • Rebecca Pillinger
  • Fiona Steele
  • Elizabeth Washbrook
• Research
• Learning and training
• Links
• Publications
• Software
• Advanced quantitative methods in social science and health

Harvey Goldstein's commentaries 1996 - 2005

• September 2005:
  • Some reflections on retiring from the Institute of Education after 28 years.
• July 2004:
  • The Statistics Commission has issued a report critical of the DfES methodology and public presentation of school league tables.
• February 2004:
  • The enquiry into 14-19 Mathematics chaired by Adrian Smith makes a strong, and welcome, case for strengthening mathematics in the school curriculum. It also, however, leaves unanswered some important questions and makes certain questionable recommendations. A contribution to the debate on the report is provided.
• January 2004:
  • The Higher Education Funding Council for England has issued a preliminary report on its pilot survey of undergraduate student course evaluations. A contribution to the debate.
  • Following last year's value added secondary results the government has this year, with great fanfare, released value added data for KS1 – KS2 results along with the raw, unadjusted, KS2 results. Unfortunately, this year's tables are no advance on the inadequate information that was produced before. It has also released KS2-KS3 and KS4 value added data which suffer from the same limitations.
• February 2003:
  • The final report from the University of Toronto team evaluating the National Literacy and Numeracy strategies has now been published. It extends some of the earlier work but retains the same technical weaknesses and lack of clarity as in previous reports. .
• January 2003:
  • With the publication of new secondary school league tables a significant 'value added' component has been included. While this does represent some advance, this commentary explains why we should remain sceptical about the claims being made by policy makers. There is still a widespread failure to recognise the limitations of all such performance indicators, and politicians and the media should make a much greater effort to inform the public about what it is and is not possible to infer from the data.
• June 2002:
  • A previous commentary on the DfES 2001 Education White Paper (PDF, 0.1 mb) discussed the validity of research findings on the effects of specialist school status. Since then a second report from the same source makes similar claims about specialist schools. This report suffers from many of the same inadequacies as the previous one: a detailed critique, by Ian Schagen and myself (PDF, 0.1 mb). It also includes reflections upon the use of educational research for policy. I twill be published in 'Research Intelligence' (British Educational Research Association) July 2002.
• March 2002:
  • A recent book (Education and Social Justice by Stephen Gorard) questions the received wisdom of the 1990s that educational participation and performance have become increasingly polarised as a result of market reforms pursued during that period. This review of the book points out methodological flaws in the analysis methods used.
• January 2002:
  • The University of Toronto team which is evaluating the implementation of the Government's National Literacy and Numeracy Strategy (NLNS) has now produced a second report . In contrast to the praise accorded to the first report (see July 2000 commentary below), this time there has been no press release. Onere as on for this apparent lack of interest may lie in the somewhat more critical stance taken by the present report.
• January 2002:
  • The government is fond of claiming that their education policies are 'raising standards'. Since they typically place great weight on increasing key stage test scores over time it is reasonable to assume that this provides their main criterion. One problem with using test scores is that it excludes potentially negative effects on those aspects of education that are not regularly monitored or measured, for example artistic and humanities curriculum subjects. In addition to this there is also a question mark about thew ay in which such test results themselves should be interpreted. This commentary explores the notion of 'standards' based upon testing and looks at evidence about the effects of the use of 'high stakes' testing regimes.
• October 2001:
  • The DfES 2001 Education White Paper follows the green paper (see May 2001 commentary). While continuing to promote league tables and 'targets' it also seeks to justify its policy to promote 'specialist 'schools using evidence from recent research. The commentary discusses the validity of these research findings and questions the Government's commitment to evidence based policy making (PDF, 0.1 mb)
• May 2001:
  • The DfEE Green paper "Schools: building on success" sets out Government plans for the future of the education service. Despite talking much about 'successful' schools and proposing many rewards and responsibilities for such schools, it is very vague about how such schools are to be defined. The worry is that such schools will be identified using currently available, and highly flawed, league tables. The government should come clean about how it intends to select schools. Go to commentary.
• March 2001:
  • The DfEE announces cash awards up to £25k each, for schools based upon 'improvement' in average test scores over time or a high position in key stage test score and exam league tables. This goes much further than before in raising the stakes associated with performance. The commentary points out that this policy, especially that part which attempts to reward schools for 'improvement' over time, conflicts with all the existing evidence.
• July 2000:
  • A recent report from a team based at the University of Toronto presents an interim evaluation of the implementation of the Government's National Literacy and Numeracy Strategy (NLNS). The report has been praised by the Government and is generally very positive about the strategy and the way it has been accepted by the educational community. In many respects, however, the report contains some serious deficiencies and makes some unsubstantiated assumptions. Critique of the report, together with a response to the critique from the principal author of the report and a further response to that.
• June 2000:
  • Current educational policy in the UK and elsewhere has emphasised initiatives to reduce class sizes. To a large extent these policies derive from recent research and research reviews that have claimed moderate learning gains from class size reductions. Any new research, therefore, that appears to contradict this deserves careful attention and evaluation. Two recent American reports claim that class size effects, at best, are negligible. Their arguments are critiqued in this commentary (PDF, 0.1 mb)
• April 2000:
  • Government proposals for performance related pay come into effect in September2000. They have aroused considerable controversy and this note points out some of the inconsistencies and inequities of linking pupil performance to teacher appraisal and promotion.
• March 2000:
  • OFSTED's inspection procedures, and the role of HMCI Chris Woodhead, are highly controversial. A recent book attempts a critique of these various activities, but unfortunately fails to provide a satisfactory evaluation of OFSTED's role. A review of the book appears in the British Educational Research Journal (2000, vol 26, 547-555). Review: An Inspector Calls.
  • In a follow-up to the DfEE autumn package, the Curriculum and Qualifications Authority, QCA, issued booklets intended to provide guidance on interpreting performance data obtained from Key stage assessments ('A guide to using national performance data in Primary schools, (1998), London, QCA'). This guidance, however, has some important deficiencies. (See also discussion and critique of the DfEE Autumn Package).
• December 1999:
  • The International Adult Literacy Survey, has been quoted and used by policy makers to compare literacy levels in a number of countries, including the UK (see April 1999 below). It has been subject, however, to some criticism. This commentary looks at the technical procedures used by the IALS researchers and exposes several weaknesses (PDF, 0.1 mb). It is written in a non-technical manner, but has a technical appendix which explains some of the statistical procedures used in IALS.
• May 1999:
  • In the Summer of 1999 a consultation process is planned on a proposed system of so called‘best value’ performance indicators (PIs) for all statutory Local Authority services. The views expressed in a recently issued preliminary paper appear to reflect current Government thinking. This note is intended to widen the debate from the somewhat narrow perspectives currently in operation.
• April 1999:
  • The DfEE issues a report on the state of adult literacy which discusses the 'evidence' and makes recommendations for action. This short paper looks critically at the report's claims.
• December 1998:
  • HMCI Chris Woodhead has been expressing his views about the reliability and general usefulness of National Curriculum tests for detecting changes over time and he considers them to be inferior to 'standardised' tests. He has been attacked by QCA and NFER, while finding some support from teachers. a This paper looks at the issues and sets them within the broader framework of current educational policy).
• December 1998:
  • The annual performance tables this year include a 'value added' element for KS3 - KS4, although most of the media chose not to publish this. The calculation and presentation of these results is both technically poor and substantively misleading. The results are presented as a 'pilot' yet readers are encouraged to make comparisons between individual schools, when in fact such comparisons are flawed.
• November 1998:
  • OFSTED has produced a consultation paper for a new inspection system which will involve identifying the 'best'20-30% of schools who will then get a less intensive inspection. The idea behind this betrays considerable naivete and the implementation would have damaging consequences. Select Go to more detailed comments.
  • Chris Woodhead, the head of OFSTED, has an article in the Times Educational Supplement (20/11/98) in which he claims to support the principle of value added comparisons of schools. On close re-examination it can be shown that his arguments are confused and that he is poorly informed about value added issues.
  • a response to the 'Autumn package' released by DfEE. This package, circulated to all schools, offers advice on the interpretation of Key Stage test scores and examination results, including for the first time an attempt to use 'value added' information. The response points out several weaknesses and serious omissions.
• August 1998:
  • a response to the report and recommendations in a report reviewing educational research. The report was commissioned by DfEE and carried out by a team at the Institute for Employment Studies, University of Sussex. The response questions some of the assumptions and methodology used and is particularly critical of some of the recommendations (PDF, 0.1 mb).
  • A response to a recently published survey for OFSTED by Tooley and Darby. The report of this survey claims that nom ore than a third of published educational research is 'satisfactory'. The response takes issue with the methodology used in this survey and hence the validity of its conclusions. The response suggests ways in which valid studies of educational research output can b be carried out (PDF, 0.1 mb).
• June 1998:
  • a response to a recently published critique of school effectiveness research. The critique accuses school effectiveness researchers of aiding and abetting Government policies aimed at holding schools wholly responsible for the performances of their pupils. The response argues that the critique is misdirected and fails to understand the importance of current school effectiveness work (PDF, 0.1 mb).
• October 1997:
  • a discussion of the failure of the 1997 Education White paper to properly address issues of poverty and school achievement and how it misunderstands some basic statistical issues. The 1997 Education White Paper - a failure of Standards. By Ian Plewis and Harvey Goldstein (Also published in British Journal of Curriculum and Assessment (1997), 8, 17-20.).
• September 1997:
  • a discussion of recent critiques of school effectiveness research: School effectiveness research: a band-wagon, a hijack or a journey towards enlightenment? by Harvey Goldstein and Kate Myers.
  • The teaching of Reading in 45 Inner London primary schools: a critical examination of OFSTED research. Published jointly with Peter Mortimore, this examines the evidence in a politically influential report and concludes that none of the report's major conclusions about the effectiveness of schools or teachers is supported by the research.
• August 1997:
  • A commentary on the July 1997 Education White Paper. This takes a critical look at the recommendations on performance tables and the proposals for numeracy and literacy targets.
• July 1997:
  • A commentary on the SCAA 'value added' report published July 1997: this is a longer version of an article in the TES of July 18th 1997.
  • Targets for all - a not-entirely-serious proposal for some performance targets for the DfEE. (updated February 1998)
• March 1997:
  • a critique of the report by the Labour Party Literacy Task Group.
• February 1997:
  • Misinterpreting Key Stage 1 test scores. Written jointly with Peter Mortimore, this is critical of a report from Social Market Foundation. Based on an analysis of KS1 test scores it attempts to draw strong policy conclusions about teaching methods and the publication of league tables. A 'popular' version was published in the Guardian February 25th 1997.
• January 1997:
  • a critique of the recently published report by Michael Barber et al. on the link between amounts of homework and pupil performance.
• December 1996:
  • a response to David Hargreaves' TTA lecture and subsequent discussions in 'Research Intelligence' in which he criticises the current state of educational research and researchers respond.
• September 1996:
  • the Secretary of State for Education issues a statement on baseline testing of five-year-olds.