Kill the Messenger: The War on Standardized Testing
|Description: In response to public demand, new federal legislation requires testing of most students in the United States in reading and mathematics, for grades three through eight. In much of the country, this new order promotes an Increase in the amount of standardized testing. Many educators, parents, and policymakers who have paid little attention to testing policy issues in the past will now do so. They deserve to have better information on the topic than has generally been available, and Kill the Messenger is intended to fill this gap. Kill the Messenger is perhaps the most thorough and authoritative work in defense of educational testing ever written. Phelps points out that much research conducted by education insiders on the topic is based on ideological preference or profound self-interest. It is not surprising that they arrive at emphatically anti-testing conclusions. He notes that external and high stakes testing in particular attracts a cornucopia of invective. Much, if not most, of this hostile research is passed on to the public by journalists as if it were neutral, objective, and independent. Kill the Messenger describes the current debate, the players, their interests, and their positions. It explains and refutes many of the common criticisms of testing. It describes testing opponents strategies, through case studies of Texas and the SAT. It acknowledges testings limitations, and suggests how it can be improved. It defends testing by comparing it with its alternatives. And finally, it outlines the consequences of losing the war on standardized testing.
ISBN Nos. 0765801787 (cloth) # 1412805120 (paper)
Library of Congress call number: LB3051 .P54 2003
Dewey Decimal System number: 371.26
Association of American Publishers' Summary of Kill the Messenger
Selected as a Profiler's Pick by YBP, Inc. and Academia Magazine from among their CORE 1000 recommended books for academic libraries, 2003/2004. "Reviews the debate over standardized testing requirements and argues in favor of testing."
Book News, Inc.
"While this education researcher clearly draws the battle lines over required standardized testing of US public school students in selected grades and subjects and acknowledges its limitations, he defends testing over other alternatives and chides media bias. Includes a foreword and preface by like-minded academics, anti- testing vocabulary used by opponents, and a general glossary."
TABLE OF CONTENTS:
Kill the Messenger: The War on Standardized Testing
Table of Contents
Reveille--Prelude to Battle (introduction)
Chapter 1--The Battlefield (testing systems and testing interests)
Chapter 2--Attack Strategies and Tactics (arguments and methods)
Chapter 3--Campaigns: The Big, Bad SAT
Chapter 4--Campaigns: Texas, the Early Years
Chapter 5--Campaigns: Texas, the Presidential Year 2000
Chapter 6--War Correspondence (media coverage)
Chapter 7--The Fruits of Victory (benefits of testing)
Chapter 8--The Spoils of War (valid concerns about testing)
Chapter 9--The Agony of Defeat (the consequences of losing the war/ the alternatives to testing)
Appendix--An Anti-Testing Vocabulary
"Kill the Messenger demolishes criticisms and myths associated with standardized testing that have been promulgated by many of the most ardent critiques and documents the half-truths, selective reporting of research, and unsophisticated manipulations of statistical data often used to demonize assessments. In so doing, Phelps provides documentation for the utility and validity of assessments in education today."
Wayne J. Camara, VP Research and Development, The College Board
"By the author's own admission, Kill the Messenger is as much about censorship and professional arrogance as it is about testing. And Richard Phelps makes compelling cases against the perpetrators: scholars and journalists who deliberately distort and ignore the facts so that schools and teachers avoid any standardized measurement of educational achievement.
"Furthermore, Phelps answers every anti-standardized testing argument with data, commonsense reasoning, and practical comparisons of how much we rely on standardized tests in every facet of our lives. His position is that education is an exception, and absent fair high-stakes exit examinations based upon standardized tests, students get the message that their schoolwork is unimportant. Phelps cautions that even with standardized testing, no standardized test can salvage bad curricular standards."
Charlene K. Haar, author of The Politics of the PTA, and President, Education Policy Institute
"Fighting for objectivity: Improving education is hard. It's a lot easier to attack tests that show improvement is needed. In Kill the Messenger, Phelps explains and defends testing, and attacks ideologically motivated scholars and gullible journalists." JoanneJacobs.com, June 15, 2003
"...a thoroughly researched and highly readable book..."
--Pat Naughtin, The Age (Melbourne, Australia), October 13, 2003
"If you're interested in how standardized testing affects education - and since we're doing quite a lot of it, you should be interested - you ought to read [it]."
--Linda Seebach, Rocky Mountain News, January 24, 2004
"As may be guessed from the spelling, Kill the Messenger: The War on Standardized Testing by Richard P. Phelps is for, and about, America. But the situation it describes is very similar to what is happening in the UK.
The jacket says this is "perhaps the most thorough and authoritative work in defense of educational testing ever written. Phelps points out that much research conducted by education insiders on the topic is based on ideological preference or profound self-interest. It is not surprising they arrive at emphatically anti-testing conclusions."
The preface, written by Professor J. E. Stone, whose work is much admired in the UK, explains perfectly: "To educators, knowledge and skills are important but not indispensable. So-called thinking skills, attitudes and developmental outcomes are of equal importance. For example, many educators would consider students who have merely acquired positive self-esteem and an ability to work well with others to be educational successes."
Standardized tests, this book makes clear, are essential for accountability. They may not be perfect, but they're all we've got. That is why they are feared and hated by incompetent teachers and, of course, by the NUT.
Kill the Messenger is essential reading for everyone with an interest in educational policy or state education generally.
--Campaign for Real Education Newsletter, No.51, Winter 2003, 18 Westlands Grove, Stockton Lane, York YO31 1EF.
"The faintest of all human passions," wrote A. E. Housman, "is the love
of truth." In the USA, host, hearth and home to the most advanced psychometric testing industry, war
rages between the proponents and opponents of standardised testing. All the checks and democratic
balances of the robust US Constitution are daily strained as first one side, then the other, seek or gain
some advantage over hearts and minds. Yet it seems a full and sufficient explanation of the complex
battle to say that opposition to testing is essentially a matter of vested interests protecting themselves
against demands for accountability.
This is an embattled book. Readers delighted, as I was, by the author�s earlier pamphlet, Why Testing Experts Hate Testing (Phelps, 1999), will here find the themes greatly amplified. Incensed by the partisan tactics of anti-testing groups, Phelps deliberately goes to great lengths to expound and analyse the differing points of view, helped by scrupulous and scholarly documentation and a robustly empirical approach. Fairness and impartiality, he reasons, will redound to his benefit, while the tactics of suppression, smear and distortion will do his enemies no good at all. I was particularly impressed by his own researches (Chapter 6) into media bias, exposing the capture of the liberal (illiberal) media yet again, with whose anti-IQ mindset we are already drearily familiar. Even the supposedly anarchic internet turns out to be as anarchic as the former Soviet Union, with tight ideological control of directory listings by those hostile to standardised testing (pp. 168�176).
Much of the street-level fisticuffs is inevitably murky. To "You don't fatten a pig by weighing it" we can now oppose, "If you don�t like the temperature, break the thermometer." But the former proposition can be wonderfully verified: Phelps details beneficial outcomes on students' abilities, ranging from NAEP (National Assessment of Educational Progress) scores in Texas and Canada to SATs in New York State, as the effects of testing kick in (Chapter 7).
There is the perception that the USA education system is lousy with unnecessary tests that children are forever sitting; but Phelps shows that US children are among the least tested in the world (Chapter 1). Moreover it is clear to me, as an end-user in contact with American children, that standards in written calculation skills have recovered greatly, to a level now above that in the UK, following the 1983 federal report, A Nation at Risk, and much subsequent testing.
In 1987 Whole Language policies were officially mandated across the Golden State, as in Ontario and Britain. Through a curriculum document, the English-Language Arts Framework, Whole Language policies were implemented to a greater extent in California than any other state, according to the federally-funded NAEP. Seven years later, in late 1994, the NAEP, applying national tests and criteria of reading competence, found that Californian children were the least competent readers in the nation. Nearly 60% of fourth graders (9�10 year olds) read so poorly that they could not gain even a superficial understanding of their school books. Even on Whole Language_inspired 'tests', the dimensions of the disaster were clear: on the specially designed, 'authentic' California Learning Assessment System, 77% of these same fourth graders obtained scores of 3 or below on a 6 point scale. The excuse of a relatively high enrolment of minority pupils was inadequate: white students were the least competent readers of their racial group in the US. Once again standardised testing provided the objective evidence that revealed the dereliction. Marilyn Adams and others were drafted in, phonic literacy teaching was mandated and Californian schools began to recover, though with a residue of unquantifiable cost in child misery.
Perhaps Phelps need not worry. "About things on which the public thinks long it commonly attains to think right," wrote Samuel Johnson (1779�1781). The perception of vested interest is one that seems to be unforgettable, as people congratulate themselves on seeing through the frantic attempts to bamboozle them. The guardians of democracy may have proved supine, but commonsense continues to fortify the apparently unquenchable preference of the general public for the information that tests give.
--Martin Turner, Intelligence v.32 (2004) pp.539�540.
"Standardized testing in the public schools -- the measurement of the results of public education -- has now reached federal proportions worth billions of dollars. Anyone at all involved in the debate (or war) needs more than newspaper columns and horror stories to go by, and this book's systematic account is a necessity for those who sincerely want real information. That Phelps is in favor of standardized testing is of course evident from the title, for "Kill the messenger" is plainly what he believes the opponents of testing are urging; yet Phelps's account of the battle represents a great deal more than advocacy. It contains facts, insights, and reports of research (real and phony both) that every teacher, supervisor, professor of education and legislator needs; and it is fascinating reading as well."
--Ralph A. Raimi, professor emeritus of Mathematics, University of Rochester, New York State, October 5, 2003
Straightforward information on the testing controversy. Just in time!!
Kill the Messenger is a wonderful book exposing the lies, biased research, and pedagogical blindness characterizing years of misinformation from the Education Establishment, which has a tremendous vested interest in maintaining the status quo of American education. It should be required reading for Ed School professors, school administrators, and all the National Board types who have such a disdain for standardized test results, but you can bet they won't be reading it. It will, however, become a favorite of the thousands of hardworking, real academic standards-oriented, traditional teachers--the ones who believe that students should be required to show what they claim to have learned--fighting their own lonesome war against the learner-centered, false-self esteem, constructivist theories that dominate education in this country today.
I am a history teacher who has worked for the past six years toward the establishment of a fair and meaningful high stakes standardized test in my state. I have heard all of the anti-testing arguments which Phelps presents in the book, but never knew exactly what to say to those die hard opponents. With this book I will not have to argue these points again--I can simply refer them to this comprehensive analysis and ask that they get back to me once they have had time to reflect on the honest facts. The misinformation about standardized testing and the lesser-known arguments in favor of testing are presented in a form that is easy for the parent and concerned citizen to understand. The examples, such as the SAT and the TAAS (in Texas) are presented in a fair and interesting manner.
Liberals and conservatives alike can feel comfortable here, for the issue cuts across the standard political classifications. Phelps sticks close to the issue of testing, covering it from all possible angles, without straying off to give an opinion on, say, vouchers. At several points he lists and then describes the people and organizations who are opposed to testing, and those who support it, pointing out how the mainstream media greatly prefers the opinions of those opponents. Charts and tables are used throughout the book to provide the data necessary to make the case. Extensive notes at the end of each chapter provide much additional information for those with a professional interest in testing. At the back of the book there is an interesting (and humorous) appendix explaining the curious vocabulary of the anti-testers.
Phelps has made a great contribution to real education reform in the United States by getting the argument over testing back on track. While not claiming that testing is the cure-all for education, the book clearly demonstrates the multiple benefits of using appropriate testing as both a guide to improvement and a measure of results.
--John Tuepker, history teacher, Long Beach, Mississippi
The Value and Importance of Standardized Testing
Kill the Messenger by Richard Phelps is an effective and extensively documented defense of standardized testing and the flawed and fabricated arguments of its opponents.
As a teacher of Advanced Placement U.S. History, I "teach to the test," a national test that over 100,000 students take each May. Colleges, the military and many employers find applicants' standardized test results useful, because they can usefully predict future success. Does anyone think that a college admissions committee can find no useful, predictive value between one student's SAT math score of 420 and and another's 620 out of a possible 800? In the real world of high schools, within one school system and even within one school building, the same year-long performance by one student might receive a grade of D or F with one teacher, while another might assign it a grade of A or B. This is the reality of American education that parents, students and teachers across the country know all too well.
By employing a common set of uniform measures, standardized tests allow a college admissions committee to see which sets of grades appear to be more reliable.
Phelps shows the contradictions in the arguments of testing opponents: "Most of us would argue that it is not fair to make high-stakes judgments of students based on the mastery of material to which they have not been exposed. Most testing opponents concur. They criticize vociferously when high stakes tests cover subject matter that students have not had an opportunity to learn. Then, sometimes in the same argument or speech, testing opponents will criticize just as vociferously the process of teaching material thatis covered on a test - that is wrong, too, that is 'teaching to the test.'"
Since public education is supported by tax dollars, the public has a right to know how its schools are performing. Standardized tests document the abject failure of many school systems to educate large numbers of students and simultaneously attest to real success, wherever it appears.
Phelps targets other evocative but baseless accusations against testing, including: "testing distorts instruction" (sad to say, the force of standardized tests often leads to the first effective teaching in a class or school!), "ignores each student's individuality," "penalizes the use of innovative curricula and teaching strategies" (could it be that these strategies, such as wasting huge amounts of instructional time on group projects and group activities, may prevent students from learning the material they are expected to know?), "unfair to women and minorities" (In reality, standardized tests reveal that many school systems are so dysfunctional that they fail to provide adequate instruction for minorities.), etc.
In a chapter that should interest all parents, Phelps examines the misleading criticisms of "The Big, Bad SAT," which almost two-thirds of U.S. colleges include in the mix of criteria for making admissions decisions. Colleges use the SAT [and AP scores] or the ACT because they are reliably predictive of a student's academic performance during his or her first year in college, which is when most drop-outs occur. Since grade inflation in many high schools masks lackluster performance and achievement, colleges need a more objective standard - and parents should be thankful that one exists. The SAT or ACT creates a common national measure that "college admissions counselors rate ... as a more reliable measure than .. high school grade point average, extracurricular activities, recommendations, essays and so on." If SAT tests had no future performance validity, colleges would not require them.
Phelps also looks at test preparation companies' claims that they they can raise SAT test scores and cites studies that show limited gains from "test coaching" - far short of the exaggerated claims. He cites one 1998 study of the recentered SAT I that found an "average effect (increase) from 21 to 34 points on the combined SAT I score scale" of coached students over those who received no coaching.
In other chapters, Phelps explains the testing systems and how and why other countries use standardized tests, looking specifically at the "testing systems of the 29 member countries of the Organisation for Economic Co-operation and Development (OECD), essentially the wealthier countries of the world, plus China, Russia and Singapore," the countries with whom US students are most frequently compared in media reports of international test results.
Phelps also examines the debates over testing in Texas and the tendency of the media to give more space to opponents of testing, while rarely subjecting their claims to critical examination.
In exposing the illogic of the arguments of testing opponents and the flawed use of evidence they cite, Phelps' work enables readers to understand some of the obstacles to improving student achievement. The next time one hears criticism of standardized testing by Alfie Kohn, FairTest, Gerald Bracey, Howard Gardner and many others, a quick check in "Kill the Messenger" might find that Richard Phelps has already examined and dissected it.
Phelps' readable prose makes this often mystifying component of modern education understandable to all of us who need to understand it: parents, teachers, school board members and interested members of the public whose taxes pay for our public schools.
--Erich H. Martel, Washington, DC, United States
Saying the unsayable
This is a long-overdue book, about the way power is used in government, especially education. To get reading right, we must stop letting the fox guard the henhouses. Educrats have been their own judge and jury, with the power to fend off all challenges.
Censorship and testing
Les deux themes du nouveau livre de Richard Phelps ont tout pour combler le public francais qui s'intéresse aux questions scolaires puisque la premiere phrase de l'introduction est :
"This book is as much about censorship and professional arrogance as it is about testing.".
Or l'on peut constater aussi en France que la question des tests est un sujet sensible sur lequel la censure est de rigueur lorsque les resultats des tests ne correspondent pas aux affirmations des responsables.
Ainsi , pouvait-on lire dans le numéro de Capital d'octobre 1997 :
"Capital : Peut-on dire, malgré tout, que le niveau d'instruction s'élève ? Roger Fauroux : Hélas non. Affirmer, comme on le fait au ministère, que les choses ne vont pas plus mal qu'avant n'a strictement aucun sens. & Là comme ailleurs, seules importent les comparaisons internationales. A cet égard, nous ne sommes guère brillants. Récemment, un test de mathématiques a montré que la France arrivait en treizième position, derrière Singapour et la Corée du Sud. Ce mauvais classement, nos fonctionnaires ne le clament pas sur les toits: en 1994, alors qu'ils avaient accepté de participer à une enqu�éte internationale de l'OCDE, ils se sont retirés sur la pointe des pieds au vu des premiers résultats."
Ne serions nous pas en France dans la situation décrite par R. Phelps :"As a cynic might say, self-evaluation is an oxymoron". Nonetheless, that is the type of evaluation strongly preferred by many mainstream education professors and administrators. After all, who else understands the U.S. education system as well as they do?"
Et la solution ne serait-elle pas que l'évaluation du système scolaire soit effectué par un organisme indépendant du ministère ?
Richard Phelps a notamment travaillé pour l'OCDE et a été professeur de mathématiques au Burkina-Faso.
Vous pouvez consulter sa page personnelle où vous trouverez un résumé du livre et ses autres écrits
Professeur de mathéématiques en collège
Comité directeur de la Société mathématique de France
The General Patton of the Testing Wars
by Nicholas Stix
A week doesn't go by, without a mainstream media story on the "horrors" of standardized testing, in which reporters tell of widespread testing error, of how testing is causing students to drop out of school, or of how testing is causing an epidemic of cheating.
The story behind the stories is that the relative prevalence of testing error is infinitesimal, that columnists stressing the dropout factor are mindlessly repeating a myth invented by radical Boston College teacher education professor Walter Haney, and that cheating is more easily prevented on standardized tests than with their alternatives.
For years, the American public has been force-fed a diet of test-bashing by the establishment media, the teachers' unions, professors of teacher education and well-financed anti-testing organizations, in which test-bashers have twisted existing data, ignored contrary data, and fabricated data outright. So reports Richard Phelps in his brilliant, new book, Kill the Messenger: The War on Standardized Testing.
As Phelps tells it, Kill the Messenger "is as much about censorship and professional arrogance as it is about testing." The author contends that the teachers and administrators who control the public education monopoly, and the teacher education professors who monopolize teacher credentialing, oppose standardized testing in order to shield themselves from public scrutiny and accountability. "it is disturbing, because school administrators and education professors represent a group of public servants who should serve as models to our children. We pay them high salaries and give them very secure jobs. Then, we give them our children. Is just a little bit of external, objective evaluation of what they do with our money and our children really asking so much"
Influential test-bashers include Walter Haney, Linda McNeil of Rice University, Harvard's Howard Gardner, University of California president Richard Atkinson, writers Alfie Kohn and Nicholas Lemann, the privately funded organization, Fair Test, and the taxpayer-funded organizations, CRESST at UCLA, and Boston College's CSTEEP. (CRESST stands for "National Center for Research on Evaluations, Standards, and Student Testing"; CSTEEP stands for "the Center for the Study of Testing, Evaluation, and Educational Policy.")
Phelps argues persuasively that objective, external, standardized, high-stakes testing is the best measure we have of how much students have learned, and how well teachers, curricula, and textbooks have done their respective jobs. The tests give us a tremendous amount of information on children's academic strengths and weaknesses, so that we may help them improve. "Objective" is in contrast to classroom grades, which are increasingly subjective, politicized, and inflated. "External" means that school officials with a stake in the results do not control examination grading. "Standardized" means that a test "is given in identical form and at the same time to students in more than one school, and all the results are marked in the same way." And "high stakes" means that test scores have consequences, so that the test serves as a powerful motivational tool. Alternatives such as classroom grades and "portfolios" of work lack the advantages of standardized testing, while being much more vulnerable to manipulation and cheating.
Phelps sets out test-bashers' strategies and tactics; presents case studies of campaigns against the SAT, the Texas teachers' literacy test, and the 2000 October Surprise attack on the "Texas Miracle" of educational progress under then-Gov. George W. Bush; media coverage; the "benefits of testing"; legitimate concerns about testing; and "alternatives to standardized testing." Two appended glossaries translate test-bashers' Orwellian jargon, and explain testing terms. Richard Phelps drives through the armies of test-bashers like Patton's Third Army cutting through France in the summer of '44. He catalogues and refutes the misrepresentations they have spread.
For instance, test-bashers have for years insisted that American students are tested more than students in any other country, and that high-stakes, standardized testing causes dropout rates to increase, and educators to "teach to the test." And liberal reporters eat this stuff up!
Phelps scolds the test-bashers for being too lazy to make a couple of calls abroad, to determine that their assumption is false. "Virtually every other industrialized country in the world tests its students more, and with greater consequences riding on the results, than we do." He shows how education professor Walter Haney inflates dropout figures by stealthily employing a highly irregular definition, whereby he counts anyone who fails to graduate on time with his age group as a "dropout," and then leaps to the baseless conclusion that the fictional dropouts were caused by standardized testing. Noting that it would be irresponsible not to teach to the test, Phelps responds to that charge, "So, they should instead teach material that the test will not cover? They should "teach away from the test?"
Kill the Messenger could have been called "Coloring Education News," since it does for education reporting what William McGowan's Coloring the News did for journalism in general. Phelps' analyses of media bias, including statistical breakdowns showing how the media let test-bashers dominate the testing debate, provide a model for media criticism. He also reports on the undisguised hostility some reporters and producers show scholars who fail to tow the party line. (Full disclosure: Phelps praises my education reporting.)
Phelps suggests that the most insidious test-bashers of all, are those who claim to support testing ... just not any existing test. For such people, "more research" is always required. "Given all the variety and all the experience, anyone who cannot be satisfied by any current testing program can never be satisfied with any testing program."
Ultimately, Phelps writes, "Most of the attacks on student testing, indeed, are attacks on measurement ... of any kind ... or, more specifically, any measurement made by groups "external" to the group being measured." Phelps cautions the reader, however, that any test is only as good as the curriculum and instructional theory it is tied to.
Written largely in a conversational style, notwithstanding its staggering scholarship, Kill the Messenger casts much needed light on a public policy issue that affects us all, but which those holding the public's trust have kept shrouded in darkness. As Phelps argues, "the debate on testing ... is part of a war for the control of our country's schools ... The booty is our children's futures. The stakes are enormous."
--Nicholas Stix New York-based freelancer Nicholas Stix has written for Toogood Reports, Middle American News, the New York Post, Daily News, American Enterprise, Insight, Chronicles, Newsday and many other publications. His recent work is collected at www.geocities.com/nstix and http://www.thecriticalcritic.blogspot.com.
Doing his part in the defense of tests
Richard Phelps, an acquaintance of mine who was very supportive when I began this blog, has finally brought his book, Kill the Messenger: The War on Standardized Testing, to fruition. It's been in the works for a few years, and I'm very happy to see that it's now out there for the general public.
Go forth and purchase. I've already ordered my copy. The book's already gotten 5 reviews on Amazon. Don't miss the comments from the reviewer who gave it only 1 star; she says virtually nothing about the book, choosing instead to relate her personal tale of woe because she didn't receive accommodated tests from some Texas universities...
What does this have to do with Phelps' book, again?
Update: An archived version of Linda Seebach's review of the book for the Rocky Mountain Times can be found here. She neatly summarizes the motive behind the anti-testing bias so often seen in the media:
The unspoken difficulty with the SAT is not educational, it is political: namely, Asians and whites consistently score on average a couple of hundred points higher than blacks and Hispanics. And it isn't because the tests are biased, any more than scales are biased because they consistently show that men, on average, are heavier than women. Tests predict almost equally well for all races.
Nobody wants testing to reveal these differences, but nobody has any idea how to change them, either, so the only way out is to look for other excuses to lessen the importance of the SAT and other similar tests.
--Kimberly Swygert, Number 2 Pencil, March 15, 2004
A lively and embattled defense of testing.
--World Future Society, Future Survey, 27(9), September, 2005
Bob Schaefer, FairTest, Feb, 2003
Heads Up to React to Forthcoming Pro-Testing Book!
ABOUT THE AUTHOR:
Richard P. Phelps grew up in St. Louis, Missouri, near Route 66 and the Frisco Route main line tracks. He received degrees from Washington, Indiana, and Harvard Universities, and a PhD from Penn's Wharton School. He taught secondary school mathematics in Burkina Faso (West Africa); worked at the Organisation for Economic Co-operation and Development in Paris, the U.S. General Accounting Office, Westat, and Indiana's Education Department; and has published dozens of articles in scholarly journals.