Evidence-based reform in education
Robert E. Slavin *
* Teacher at Johns Hopkins University and University of York.
Abstract: This article proposes an approach to education reform in which government encourages schools to use programs and practices proven to be effective in rigorous evaluations. In this system, teachers and school leaders would be able to select among a variety of proven methods, and could receive funds to help them adopt and implement these programs. Government would help build up a set of proven programs for all grade levels and many subjects, and would provide support to help schools choose and implement proven programs, but schools would have freedom to choose whatever works for their students. This policy, it is argued, would promote innovation and reform without top-down mandates, building knowledge as it increases effective teaching and improved outcomes for students.
Keywords: Reforms, research, decision-making.
Reforma educativa basada en evidencias2
Síntesis: Este artículo propone un enfoque sobre la reforma educativa en el que el gobierno alienta a las escuelas a utilizar programas y prácticas cuya efectividad fue probada mediante rigurosas evaluaciones. En este sistema, profesores y líderes de la comunidad educativa podrán escoger de una gama de métodos que han sido probados y podrán recibir fondos que los ayudarán a adoptar e implementar estos programas. De este modo, el gobierno ayudaría a construir un conjunto de programas comprobados para todos los niveles educativos y para distintos sujetos, y proporcionaría apoyo para que las escuelas puedan escoger e implementar programas de efectividad comprobada, sin que disminuya la libertad de las escuelas para elegir el que consideren que funcionará mejor con sus estudiantes. Esta política, de sostenerse, promoverá la innovación y la reforma sin mandatos verticales, construyendo conocimiento al tiempo que se incrementa la enseñanza eficaz y se mejoran los resultados obtenidos por los estudiantes.
Palabras clave: Reformas; investigación; toma de decisiones.
Reforma educativa baseada em evidências3
Síntese: Este artigo propõe um enfoque sobre a reforma educativa em que o governo alenta as escolas a utilizarem programas e práticas cuja efetividade foi provada mediante rigorosas avaliações. Segundo este sistema, professores e líderes da comunidade educativa poderão escolher entre uma série de métodos que foram provados e poderão receber fundos que os ajudarão a adotar e aplicar estes programas. Deste modo, o governo ajudaria a construir um conjunto de programas comprovados para todos os níveis educativos e para diferentes sujeitos e proporcionaria apoio para que as escolas pudessem escolher e aplicar programas de efetividade comprovada, sem que diminuísse a sua liberdade para escolher o que considerem que funcionaria melhor com seus estudantes. Com esta política poderá se promover a inovação e a reforma sem mandatos verticais, construindo conhecimento, ao mesmo tempo em que se incrementa o ensino eficaz e se melhoram os resultados obtidos pelos estudantes.
Palavras-chave: Reformas; pesquisa; tomada de decisões.
1. Introduction
Imagine the following scenario. Ten years from now, education throughout the world is very different from what it is today. In every classroom and every school, teachers are using programs and practices that have been proven to be effective in rigorous experiments. Government agencies and researchers are constantly updating reviews of research on programs and practices of all sorts, for every objective, and then proactively disseminate information on what works. Government provides funding to help schools learn about and adopt proven programs. Teachers and principals who are dissatisfied with their pupils’ current levels of performance have regular opportunities to meet to review easily accessible evidence on programs likely to solve their problems, to visit schools using these programs, and to attend effective program fairs or online webinars in which they can see various programs demonstrated and explained. Ultimately, school staffs vote on which programs to adopt, and they must achieve a supermajority of 75% in favor to qualify for funding.
Because they know that proven programs are sure to be widely used, universities, research organizations, commercial companies, and government itself are engaged in extensive research and development to create and evaluate new programs more effective than those that are state-of-the-art today. New technology applications, new professional development programs, new approaches of every kind are flowing regularly from the best minds in education, who know that they will ultimately be held accountable not for how their programs look, but for how well they work in real schools. For example, if a university or research organization develops a Spanish reading program and finds it to be effective, the program could be adopted anywhere in the Spanish-speaking world.
In this new world, the role of government will be very different. Government will maintain standards, holding schools accountable for improving the performance of all children at all grade levels. However, instead of telling educators exactly what to do to achieve those goals, government will focus on helping schools adopt and implement effective practices. It will maintain the integrity of a system of evaluation and dissemination of best practices, will encourage the formation of national or regional networks of schools using practical proven models, and will provide resources to help all schools continuously assess their needs and seek proven programs to meet those needs. Government will work to ensure that new and experienced teachers are learning teaching methods most likely to make a difference for children.
Teachers will take a leading role in this new world. They will be expected and enabled to be full participants in an evidence-informed decision making process, deciding as members of school staffs on which programs the school will adopt, and then working together to ensure that whatever programs they choose to put in their school will be implemented with integrity, intelligence, and appropriate accommodations to their school’s unique needs. Because teachers will have information on various programs readily available to them in easily comprehensible forms, they will be empowered to be critical consumers of evidence and to make wise and informed choices for their children. Teachers will also be active participants in the research progress, trying out new methods, contributing ideas, and working in networks to help the entire profession move forward.
The scenario I have just described is entirely achievable. Nothing in it requires radical change in our society, government, or teaching profession, or unlikely advances in science or technology. What it does require is a commitment in government and in our profession to one simple, powerful idea: Use what works.
Evidence-based policy is the way forward in education reform. This article lays out a rationale for evidence-based reform and discusses the policy, scientific, and professional changes needed to make this vision a reality.
2. Evidence-Based Reform in Education
There is a new movement in education toward evidence-based reform, policies promoting the creation, evaluation, and widespread application of programs and practices that produce significantly better outcomes than current practices (Slavin, 2008a). Although evidence-based reform is new in education, other fields have long since embraced it. In particular, medicine, agriculture, and engineering began in the 20th century to require that innovative medicines, seeds, and machines to be put to the test before being widely adopted. The result was revolutionary progress in each of these fields, which continues today. Evidence-based reform in any area creates a dynamic of progressive improvement, in which many researchers and developers are working to replace today’s best solutions with something even more effective, confident that policy makers and the market will enthusiastically adopt proven innovations.
If education is to make significant progress in the 21st century, it must embrace evidence-based reform. However, there are great obstacles to be overcome.
For evidence-based reform to prevail, three conditions must exist.
1. There must be a broad range of proven programs in every area of education, every subject and grade level, appropriate for use in a given country or region. Evidence-based policies will not prevail if few programs have strong evidence.
2. Trusted, impartial, educator-friendly reviews of research must be available, to enable educators and policymakers to know which specific programs and practices have been proven to work in rigorous evaluations.
3. Government agencies must provide resources to schools to enable them to adopt proven programs.
3 Building the Research Base for Effective Programs
One of the most important requirements for evidence-based reform is the development of a substantial set of replicable programs and practices with strong evidence of effectiveness. Educators and policymakers must have confidence that if they embrace evidence-based reform, they will be able to choose among many programs that have been developed, rigorously evaluated, and found to be effective. The development process might use a series of «design competitions,» in which government sets out what it wants and then funds a variety of entities to develop and then evaluate competing alternatives (Slavin, 1997). For example, a funding agency might ask R&D organizations to develop an approach to teaching algebra to students aged 14-16 capable of increasing their performance on national or international measures. The most promising applicants could be chosen in a competition, and this number might be winnowed down over time if some designs turn out not to be effective or practical. The New American Schools Development Corporation (nasdc), funded by large corporations rather than government, did exactly this to create comprehensive school reform models in the 1990’s (Kearns & Anderson, 1996; Slavin, 2008b). Comprehensive school reform models such as Success for All (Borman et al., 2007; Slavin, Madden, Chambers, & Haxby, 2009), America’s Choice (Supovitz, Poglinco, & Snyder, 2001), Direct Instruction (Adams & Engelmann, 1996), Modern Red Schoolhouse (Kilgore et al., 1996), and the School Development Program (Comer, Haynes, Joyner, & Ben-Arie, 1996) have been extensively evaluated and found to be effective, and continue to be used in thousands of us schools, even in the absence of government support.
The design competition process that produced comprehensive school reform models could be applied to any subject and grade level. Imagine design teams working on the design and evaluation of programs capable of accelerating achievement in beginning reading, primary mathematics, algebra, and science, dropout-prevention, early childhood, and so on. In each case the goal would be to build on the best that currently exists, and to end up with numerous programs, all of which have been proven to increase achievement. These programs could be highly diverse. Some would involve technology, others not. Some would require extensive training, others less so. Some would challenge current conceptions of curriculum, others might be traditional. All that matters is that they are proven effective and are replicable in many schools.
A key issue in the design competition process is the design of the evaluation. Evaluations should use random assignment to conditions, should use national or state assessments as their outcome measures, and should be large enough (at least 10 schools) to avoid idiosyncrasies due to particular schools. The programs should be implemented under the realistic conditions that will exist in practice, without extra attention or non-replicable conditions. The evaluations may be done by developers or by third parties, but under close observation by the funding agency. This is the type of evaluation used, for example, in studies of the progresa (now Oportunidades) financial incentive approach in Mexico (Attansio, Meghir, & Santiago, 2005) and similar programs in Colombia, Ecuador, Costa Rica, and other countries see (Slavin, 2010).
4. Reviewing What Works
Educators and policy makers need to have scientifically valid, fair, and clearly written summaries of the strength of the research evidence showing the effectiveness of education programs. Educators are unlikely to take the time to try to weigh competing evidence from many evaluations. They need information they can rely on in a summative form.
There are several practical reviews of research on educational programs. My colleagues and I have created one of the largest review project in education, the Best Evidence Encyclopedia (www.bestevidence.org), or the bee. It reviews research on programs and practices for elementary and secondary schools. The bee contains brief “educator’s summaries” in each area as well as full-scale reviews. It currently has extensive reviews in mathematics (Slavin & Lake, 2008; Slavin, Lake, & Groff, 2009), elementary reading (Slavin et al., 2009;) secondary reading (Slavin et al., 2008), programs for struggling readers (Slavin et al., in press), early childhood education (Chambers et al., 2010), and comprehensive school reform (csrq, 2006a, b). Almost all of the full reports have also been published in selective journals, which provides an external check on the scientific value of the reviews.
Other us review projects include the What Works Clearinghouse (www.whatworks.gov), Social Programs that Work (www.evidencebasedprograms.org), the Comprehensive School Reform Quality Center (www.csrq.org), and the Promising Practices Network (www.promisingpractices.net). The Best Evidence Encyclopedia links to the education sections of other web sites, and is trying to facilitate access to reviews of research in education from all sources.
Although there are controversies and difficulties in program effectiveness reviews (see Slavin, 2008c), this enterprise is moving forward rapidly on many fronts, and within a few years it is likely that there will be multiple high-quality, reliable reviews available to English-speaking educators and policymakers. A similar approach could review research on programs and practices available to Spanish-speaking educators.
5. Evidence-Based Policies
Ultimately, it is not enough to have many research-proven programs and trusted reviews of research. Education lacks a tradition of looking to evidence for program decisions, and without clear support from government, other factors will remain more important than evidence.
In government policies to support the use of proven programs, it is essential to be clear about which programs have strong evidence of effectiveness. Governments should then provide incentives to schools to use programs that have been proven to be effective.
Any country could experience a transition within a moderate period of time to an evidence-based system of education. To do this, governments might do the following:
1. Establish grants programs to help educational researchers evaluate existing programs, adapt programs from other countries to their national context and evaluate them, and develop and evaluate new programs targeted to their countries.
2. Encourage schools to use programs found to be effective in rigorous experimental-control comparisons. This encouragement might include having ministry officials positively note effective use of proven programs, providing professional development in choosing and implementing proven programs, and providing grants to help schools select and implement proven programs appropriate to their needs.
3. Provide funding to help schools choose proven, effective programs and implement them with fidelity.
The policies discussed here put greater freedom and choice in the hands of local educators. They enable teachers and school leaders to make key decisions about curriculum, teaching methods, and programs. It is apparent that they will need useful and valid information about which approaches to choose. Government’s role is to provide information and resources designed to help school leaders make wise choices. Evidence-based reform allows government to devolve substantially greater authority to local educators while still maintaining standards and offering assistance to improve teaching practices through voluntary rather than mandatory means.
6. Consequences of Evidence-Based Reform
The consequences of evidence-based reform could be profound. If government policies began to favor programs with strong evidence, developers would have an incentive to engage in serious development and evaluation efforts. Seeing the immediate impact of research and development, policy makers might provide substantially greater funding for these activities.
Evidence-based reform would finally apply to education the process that led to dramatic developments in medicine, agriculture, and technology in the 20th century, where every practical solution that meets evidence standards supersedes less effective solutions, and a vast R&D enterprise works to improve on the best we have available today.
The winners in this would be children, especially those who are least well served by the current system, the teachers who yearn for more effective tools to help them do their job well, and the whole society, which would come to expect progress in education as confidently as it currently expects progress in other fields.
Recent developments in research and policy make it possible to finally put education on the road to genuine reform. With consistent support, proven, effective models of school and classroom reform can be developed, rigorously evaluated, and disseminated, benefitting hundreds of thousands of children. It remains for policymakers and our profession to take the necessary steps.
References
Adams, G. L. y Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond DISTAR. Seattle, WA: Educational Achievement Systems.
Aladjem, D. K. y Borman, K. M. (eds.) (2006). Examining comprehensive school reform. Washington, D.C: Urban Institute Press.
Attanasio, O.; Meghir, C. y Santiago, A. (2005). Education choices in Mexico: Using a structural model and randomized experiment to evaluate progresa. ifs working paper ewpo 5/01.
Borman, G.; Slavin, R.E.; Cheung, A.; Chamberlain, A.; Madden, N. A. y Chambers, B. (2007). «Final reading outcomes of the national randomized field trial of Success for All», in American Educational Research Journal, 44(3) 701-739.
Chambers, B.; Cheung, A.; Slavin, R. E.; Smith, D.y Laurenzano, M. (2010). Effective early childhood education programs: A systematic review. Manuscript submitted for publication.
Cheung, A. y Slavin, R. E. (2050). «Effective reading programs for English language learners and other language-minority students», in Bilingual Research Journal, 29 (2), 24-267.
Comer, J. P.; Haynes, N. M.; Joyner, E. T. y Ben-Avie, M. (1996). Rallying the whole village: The Comer process for reforming education. New York: Teachers College Press.
Comprehensive School Reform Quality Center (Csrq) (2006a). Csrq Center report on elementary school comprehensive school reform models (revised). Washington, Dc: American Institutes for Research.
— (2006b). csrq Center report on middle and high school comprehensive school reform models. Washington, dc: American Institutes for Research.
Kearns, D. T. y Anderson, J. L. (1996). «Sharing the vision: Creating New American Schools», in S. Stringfield, S. Ross y L. Smith (eds.). Bold plans for school restructuring: The New American Schools. Mahwah, NJ: Erlbaum.
Kilgore, S.; Doyle, D. y Linkowsky, L. (1996). «The Modern Red Schoolhouse», in S. Stringfield, S. Ross y L. Smith (Eds.), Bold plans for school restructuring: The New American Schools Development Corporation designs. Mahwah, NJ: Erlbaum.
Slavin, R. E. (1997). «Design competitions: A proposal for a new federal role in educational research and development», in Educational Researcher, 26 (1), 22-28.
— (2008a). «Evidence-based reform in education: What will it take?», in European Educational Research Journal, 7 (1), 124-128.
— (2008b). «Comprehensive school reform», in T. Good et al. (eds.), 21st century education: A reference handbook. Thousand Oaks, CA: Sage.
— (2008c). «What works? Issues in synthesizing educational programme evaluations», in Educational Researcher, 37 (1), 5-14.
—(2010). Can financial incentives enhance educational outcomes? Evidence from international experiments. Research Review, 5(1), 68-80.
Slavin, R. E. ; Cheung, A.; Groff, C. y Lake, C. (2008). «Effective reading programs for middle and high schools: A best-evidence synthesis», in Reading Research Quarterly, 43 (3), 290-322.
Slavin, R. E. y Lake, C. (2008). «Effective programs in elementary mathematics; A best-evidence synthesis», in Review of Educational Research, 78, 427-515.
Slavin, R. E.; Lake, C.; Cheung, A. y Davis, S. (2009). «Effective reading programs for the elementary grades», in Review of Educational Research, 79(4), 1391-1465.
Slavin, R. E.; Lake, C.; Davis, S. y Madden, N. A. (in press). «Effective programs for struggling readers: A best-evidence synthesis», in Educational Research Review.
Slavin, R. E.; Lake, C. y Groff, C. (2009). «Effective programs in middle and high school mathematics: A best evidence synthesis», in Review of Educational Research, 79, 839-911.
Slavin, R. E.; Madden, N. A.; Chambers, B. y Haxby, B. (2009). Two million children: Success for All. Thousand Oaks, CA: Corwin.
Stringfield, S.; Ross, S. y Smith, L. (eds.) (1996). Bold plans for school restructuring: The New American Schools designs. Mahwah, NJ: Erlbaum.
Supovitz, J. A.; Poglinco, S. M. y Snyder, B. A. (2001). Moving mountains: Successes and challenges of the America’s Choice comprehensive school reform design. Philadelphia: University of Pennsylvania, Consortium for Policy Research in Education.
1 Portions of this paper are adapted from Slavin, R. E. (2008). Evidence-based
reform in education: What will it take? European Educational Research Journal, 7 (1), 124-
128.
2 Este trabajo es una adaptación basada en el artículo de Slavin, R. E. (2008).
Evidence-based reform in education: What will it take? European Educational Research
Journal, 7 (1), 124-128.
3 Este trabalho é uma adaptação baseada no artigo de Slavin de Slavin, R. E.
(2008). Evidence-based reform in education: What will it take? European Educational
Research Journal, 7 (1), 124-128. |
