November 3, 2017
The following is the latest post from Voqal Education Program Director Vinny Badolato sharing his thoughts on the need for better accountability measures for America’s education system.
I’ve been thinking a lot about school quality and accountability lately. My oldest son entered third grade this year, which marks the major milestone of many years of having to sit for standardized assessments primarily for school and teacher accountability. I also finished two recently published books on the topic: The Testing Charade by Daniel Koretz, and Beyond Test Scores, by Jack Schneider (two really engaging reads). These books are extremely well-researched and make strong arguments against our current test-based accountability system in favor of richer measures of school effectiveness.
Daniel Koretz is a noted scholar and one of the top experts on educational assessment and testing policy. The Testing Charade summarizes his over three decades of scholarship on assessment written for the lay audience (see him talk about it here). Koretz argues that educational assessments are important and necessary, but we have tragically misused a blunt instrument in extraordinarily unrealistic ways to judge teachers, schools and the education system. The result is a system that has led to wide-scale manipulation, cheating and the general degradation of teaching and learning in the name of so-called accountability. He offers a plan for doing better that will call for extensive trade-offs regarding how we evaluate and hold schools accountable.
Jack Schneider takes this a step further by offering a powerful framework for fairly and comprehensively measuring school quality based on a multi-year project undertaken in the highly diverse urban school district of Somerville, Massachusetts. This framework calls for measuring a wide-range of Essential Inputs (Teachers and Teaching Environment, School Culture, and Resources) in addition to Key Outcomes (Academic Learning, and Character and Well-Being) to evaluate schools. By collecting relevant data, analyzing it appropriately and presenting it to teachers, administrators, policymakers, students and parents in accessible and useful ways, a richer and more accurate picture of school effectiveness develops.
What these books stress very clearly is that educating humans is extremely nuanced and complicated, and schools are complex ecosystems. Neither can be accurately or confidently evaluated through a single instrument, no matter how refined and validated that instrument is. This is important. Standardized assessments are critical to providing a snapshot of how some academic learning is progressing in a school, but no standardized test can capture more than a relatively small fraction of what is actually going on in a school. And they certainly cannot by themselves tell us how good (or bad) a school or teacher is at helping kids learn.
Education and schooling are about much more than teaching children how to read, write and do math. It is also about helping children grow socially and emotionally so they can become engaged and empowered citizens and well-rounded adults. Yet, our current test-based accountability system only examines cognitive development, and only snippets of development at that. We are missing major pieces of the learning puzzle, but we currently use this incomplete information to make extremely high-stakes decisions concerning educators, schools and students (note: the newish federal education law, the Every Student Succeeds Act (ESSA), represents a slight shift away from this, but there is a long way to go).
More importantly, as Koretz so robustly documents, the use of academic tests as the primary accountability method is not only seriously flawed, it is educational malpractice. The core of this argument comes from Campbell’s Law, which states: the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor. The current misuse of test-based accountability and resulting unintended negative consequences, unfortunately, follows this law to the letter.
These books have gotten me excited about the promise and potential for thinking bigger about measuring school quality. By shifting away from faulty test-based accountability in favor of wide-ranging measures and robust use of new data we can better understand what makes a good school, determine what schools need to be successful, and evaluate quality — while holding schools accountable — through a richer set of student outcomes. It’s not easy or cheap and will call for some significant disrupting of our current teaching, learning and accountability practice. But I am even more convinced now that it is possible. And worth it.