"Tests fail because of an effect I call the "Numerical Outcome Principle". . . .
". . .Our leaders need to look beyond the numbers,
and judge the context in which the numbers arose. . . "
- Joe Ganem
-------
Why Testing Fails: How Numbers Deceive Us All
By Joseph Ganem Ph.D.
There is broad agreement, that improving education requires assessing student outcomes, and holding those responsible accountable for the results. But, the use of standardized tests, as a tool for assessment and accountability, has resulted in more disillusionment than improvement. The heavy emphasis on testing mandated by the No Child Left Behind Law, has not lead to the sought after gains in educational outcomes.
I believe that the failure of mandated testing is a predictable result. Tests fail because of an effect I call the "Numerical Outcome Principle." Simply put: Once a number is used to judge outcomes, people will adjust their behavior to maximize that particular number. The actual outcome no longer matters. The field of education is rife with examples of the Numerical Outcome Principle in action; instances in which the attainment of lofty goals is undone by the substitution of numbers in place of sound human judgment.
Consider the Scholastic Aptitude Test (SAT), which is a widely used instrument for college admission decisions. The original purpose of the SAT was to provide a level playing field for admission to college. The intent of the test was to identify the students most likely to succeed in college, without regard to social and economic background. In theory, a student from a poor inner city school district, and a student from an expensive private prep school, who have identical SAT scores, should have equal abilities to succeed in college.
But, the result of the SAT is the appearance of an industry that profits through the sale of expensive test preparation services to students. Private companies, such as Kaplan and Princeton Review, charge hundreds of dollars for SAT prep courses, and thousands of dollars for private tutoring. Both companies boast that the combined SAT scores of their students increase more than 100 points because of their services. As a result, there is no way to know from a raw SAT score, if the number represents innate ability, or intensive prep work. A privileged background can provide an advantage on the SAT test, which defeats its purpose.
In response, many colleges have rethought the reliance on SAT scores for admission decisions. Loyola University Maryland, where I teach, no longer requires that applicants submit SAT scores. The fact is success in college depends not only on scholastic aptitude, but also on many other intangibles, such as desire, focus, motivation, and maturity. These intangibles defy quantification, but a competent admissions officer is usually able to make an accurate judgment.
Rather than learn from decades of experience with the shortcomings of the SAT, politicians and educators continue to push testing as a cure for education ills. Consider the testing mandated by the No Child Left Behind Law. The enormous pressure placed on administrators and teachers to raise test scores, has changed both teaching methods and curricula in unintended ways. Teachers must adhere to rigid schedules, and have little flexibility to accommodate natural variations in student interests and abilities. The tests no longer assess the effectiveness of curricula; instead curricula are determined by the tests.
The result is the phenomena of "teaching-to-the-test." As a teacher, I have always found that a strange expression. After all, I give tests, and I only test what I teach. To do otherwise would be unfair to the students. Few people argue against tests, as a method to assess students learning, and test scores as a basis for assigning grades. Used appropriately, tests are a useful learning tool, for both teachers and students. Tests motivate students to study, and inform teachers about the effectiveness of their instruction.
But, "teaching-to-the-test" is something different. It is an educational mindset, in which test scores are not measures of learning outcomes; the test scores are the outcomes. While that distinction might be subtle, it has real effects on how classes are taught, and in the messages we communicate to students about the goals of an education. Tests are measurement tools; they should not be the reasons that students come to class.
The aversion, that "teaching-to-the-test" evokes, arises because at its essence, it is a form of manipulation. Children are perceptive, and they can figure out when adults are manipulating them. Teachers are perceptive, and they will adjust their actions to the reward structure in place. The net result is a poisonous class atmosphere, rather than a collaborative atmosphere needed for learning. It is the Numerical Outcome Principle in action. Which is why testing will fail.
Teachers and students are perceptive. What we need are leaders who are perceptive, and willing to use judgment instead of hiding behind meaningless numbers. A school is not a factory turning out widgets with identical product specifications. But, too often, the characterizing of educational outcomes solely in terms of test scores, results in the mindset of manufacturing company, rather than an educational institution. Our leaders need to look beyond the numbers, and judge the context in which the numbers arose. Each child, each teacher, each school is unique. We should celebrate that uniqueness, instead of trying to eliminate it.
ساحة النقاش