reseņas educativas (Spanish)    
resenhas educativas (Portuguese)    

This review has been accessed times since May 2, 2007

Nichols, Sharon L. & Berliner, David C. (2007). Collateral Damage: How High-Stakes Testing Corrupts America’s Schools. Cambridge, MA: Harvard Education Press. Pp. 250     $25     ISBN-13: 978-1-891792-35-9

Reviewed by Susan Ohanian

May 2, 2007

In her foreword to Collateral Damage, Nel Noddings forcefully summarizes the book’s contents: “Nichols and Berliner demonstrate that high-stakes testing is wrong—intellectually, morally, and practically. Not only will it ‘not work’ to improve education, it is already doing demonstrable harm.” Bringing together many press accounts of the negative impact of high stakes testing, Nichols and Berliner provide convincing argument that the punitive measures accompanying this testing is destroying America’s greatest invention, its public schools.

What I especially like about the book is that it lays out a moral argument snuggled up next to the intellectual one, detailing why harm to children is inevitable under a system of high stakes testing and also giving brief samples of just who the harmed are—from African-American high school pushouts at the World of Opportunity in Birmingham, Alabama, to special education students in Pine Level, North Carolina; from a Vallejo City teacher to teachers in San Antonio and West Virginia. Making good use of news clips, the book is replete with examples of children hurt and teachers professionalism undermined by high-stakes testing.

I would dispute one assertion made by the authors, who argue that the schools of the affluent are not much bothered by high stakes testing but instead profit from the fact that such testing “forces a kind of education of the children of the poor that ensures they cannot compete with the children of the wealthy. The drill and test-prep education we see in schools for the poor—and their high failure rate—does not prepare them for knowledge workers’ jobs nor for the best universities in the nation. This makes room for the children of the more privileged. Since the status of children from advantaged families is preserved through high-stakes testing, it is easy for these folks to defend their use.”

Sharon L. Nichols

People defending the use of these tests are Business Roundtable types and their political cronies who send their children to private schools, not suburban moms, many of whom are strong in the high-stakes resistance movement. They know that high stakes testing also harms their children. The teachers know it too. After I gave a fierce anti-high stakes test talk in an affluent New York suburban school, an administrator commented that their teachers didn’t have to worry about such tests. “We know our children will do well. Testing doesn’t disrupt the curriculum,” she insisted.

A third grade teacher voiced an immediate rebuttal, pointing to the fierce competition between and among affluent districts. “If our kids are two points lower, parents and real estate agents treat it like such a huge deal. The pressure is killing us.” And she burst into tears. The room exploded into stories of pain, frustration, and anger. With NCLB, all state tests have become high stakes, and although not all the costs are readily visible, everybody pays a high cost. Even the affluent. Unexamined by the authors is why so many poor people and the organizations purporting to represent them support the tests. We must remember that when Connecticut sued the Feds over NCLB, the NAACP sided with the Feds.

David C. Berliner

As they state in the preface, Nichols and Berliner set out to present a cohesive and convincing set of examples of the problems associated with high-stakes testing, hoping to “convince legislators and other supporters of high-stakes testing that the costs associated with high-stakes testing are simply not worth it.” This worthy effort is based on the premise that legislators are capable of being convinced by sound reasoning. I have my doubts and maybe David Berliner does too. He is a partner of the grassroots group Educator Roundtable, whose stated purpose is to eliminate NCLB. Not change, not fix, not pour more money into. Eliminate. (www.educatorroundtable.org) Don Perl, one of the test resisters profiled in the book, a teacher who refused to administer the Colorado Student Assessment Program tests, went on to form the Coalition for Better Education (CBE), an activist group informing parents of their right to opt their children out of state testing. CBE also partners with the Educator Roundtable.

The authors make compelling use of Campbell’s law, a well-known social-science law named for social psychologist Donald Campbell. Campbell observed that the more any quantitative social indicator is used for social decision-making, the more it will be subject to corruption pressures and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Campbell’s law, along with its application to test scores made by George Madaus and Marguerite Clarke, point to this inevitability: Uncertainty about the meaning of test scores increases as the stakes attached to them becomes more severe. With example piled on example, Nichols and Berliner document this inevitability of the distortion and outright corruption surrounding NCLB. This belies the claims of Senator Kennedy and other purported liberals that they can fix NCLB. Hundreds, perhaps thousands, of teachers and parents have sent this message to Kennedy’s office. And yet, he stands firm behind his original support of NCLB and now calls for strengthening “our academic standards and assessment methods to ensure that students have the knowledge and skills necessary for today's knowledge-based global economy.” Kennedy thinks promising to get more money into reauthorization will soothe his critics.

“Cheating” gets three chapters in Collateral Damage and provides an underpinning for much of the rest. The subtitle of chapter 3 points to why the authors see cheating as “prevalent, visible, and seemingly justifiable”: The Prevalence and Many forms of Cheating and the Problem of Absolute and Relativistic Standards for Judging Such Occurrences. (33) This subtitle points to my own sea change regarding cheating. Soon after Congress passed NCLB, I set up a website of resistance. Although not so bloodthirsty as Mme. Defarge, like her, my goal was to keep a record of names and incidents. Enter ‘cheating’ in the search function on my site and you’ll get 283 hits. You’ll end up with stories with titles like “TAAS Scheme Investigations Leads to School Officials' Resignations”; “Princeton Review Accused of Cheating and Plagiarism”; “To Catch a Cheat”; “Tampa Teen Charged with FCAT Felony”; Teacher Quits as Altered Tests Found”; Cheating or Misunderstandings? More Arizona Teachers Give Students Improper Test Aid”; “Celebrated School Accused of Cheating”; “Do Teachers Coach During WASL Testing?” Most of these articles were posted between 2003 and 2005. This doesn’t mean cheating has decreased since then but signifies that I have lost heart in keeping track. Over time I recognized what Nichols and Berliner observe, “Cheating seems to be an integral part of the high-stakes testing movement” ; I also recognized that I was becoming more and more sympathetic with the cheaters. What can a teacher do with and for a third grader who has already repeated the grade twice? I continue to be haunted by an article about a Florida child who was made to repeat kindergarten three times because his school was so determined that he be “test ready” when they allowed him to move on to first grade. Surely in the face of those rules, what that child needed was someone willing to cheat.

We must thank Nichols and Berliner for focusing on the complex moral dilemma teachers face:

The dilemma of teachers and administrators under NCLB is similar to those posed by Kohlberg in his studies of moral development many years ago. One of those dilemmas posed the question of whether it is right or wrong to steal medicine you can’t afford in order to save your own wife or husband. Obviously, this dilemma pushed respondents to deal with whether or not the biblical admonition “thou shall not steal” is inviolable. Under the conditions described, many deeply religious people forgave the theft of the medicine. Teachers are in a similar bind. Teachers are faced with the dilemma of cheating to help a struggling student or to ensure stability in their own family, or not cheating and watching a student falter or their family harmed. Under these circumstances, many teachers may come to the conclusion that cheating on the test, like stealing the medicine, can be justified. (p. 35)

The authors give powerful examples of all the ways individuals, schools, and states can—and do—cheat the accountability system but refuse to play the “Gotcha!” game so beloved by the media. Instead, the authors ask the question that the media ignore: “We should be asking why so many competent and decent professionals think the system they are in is so unfair to their students, their schools, and themselves and, as a result, feel justified in doing direct test preparation, violating standardization procedures, and cheating.” (p. 52)

Chapter 3, Excluding Students from Education By Design and By Neglect, the Crisis of Caring in Our Schools, and the Special Case of the “Bubble Kids,” describes institutional kinds of cheating. The chapter gives examples of schools and entire systems that push kids out of school to improve the test-taking pool and schools that put great effort and attention on those students deemed close to passing the test. These kids get special attention and tutoring while students doing less well get little extra help and more able students are pretty much ignored because their success is a given. British researchers call this educational triage, dividing students into three groups: non-urgent cases, suitable cases for treatment, and hopeless cases.

Chapter 4, States Cheat Too! How Statistical Trickery and Misrepresentation of Data Mislead the Public, describes cheating at the district and state level. Here we get fudging on the number of high school dropouts. Nichols and Berliner name prominent names here.

In 2000-2001, the year that Houston said it had a 1.5 percent dropout rate, about 5,500 left school and over half should have been counted as dropouts, but were not. For his leadership of the Houston school district, superintendent of schools Rod Paige was honored by McGraw-Hill and, on the basis of his record as a school leader, was elevated to the highest educational position in the land, secretary of education under President George W. Bush. For their outstanding urban education programs, Houston received $1 million from the Broad Foundation. (p. 75)

As Nichols and Berliner document, Houston was not alone.

In their discussion of cut scores, the authors point out something that cannot be repeated enough: Cut scores on tests, determining who is proficient and who is not, are political decisions. They are not scientific or psychometric decisions. (p. 87) Repeat: Cut scores are political decisions. I dream of the day one million teachers march on Washington, carrying banners: Cut scores are political decisions. Or maybe the signs should just say, We know it’s politics, Mr. Kennedy.

This chapter also includes a number of stories--ranging from students refusing to take required state tests to teachers refusing to give them to shoddy test questions, test scoring mishaps, and the dubious qualifications of people grading the tests. The stories have been told before, but because of the hard core secrecy surrounding the tests, we repeat the only stories we are able to get our hands on. Teachers have become so intimidated by threats of dismissal that they don’t tell what they know. The real scandal here is that because of test secrecy and layers of bureaucratic intimidation, even educators don’t know the depth of the problem and they are afraid to reveal what they do know. In a number of states, teachers are threatened with losing their jobs if they even look at the test. One must ask how professional it can be that teachers are forcing children to take tests that are likely to be both developmentally inappropriate and filled with error. One can wonder how the teacher unions and professional organizations can defend their silence about teachers blindly handing out secret tests with dubious validity, tests that make children vomit.

Here is a small example from a McGraw-Hill test given to New York fourth graders some years back. Children were asked to read a passage about a chance meeting between a young girl named Julie who wandered away from her class field trip and a wispy-haired man in Princeton, a man who wore no socks. An afterword informs young readers that the man was Albert Einstein, whose Special Theory of Relativity “is sure to play a big role in human expeditions to the stars.” It also explains that the story is based on a real incident involving Mary Budd Rowe, “an education professor at Stanford University in Palo Alto, California.” The passage ends with the rhetorical question, “Don’t you think she was a great person to be teaching teachers?”

Does any fourth grader in the country care who teaches teachers? For starters, fourth graders are worrying about

• Why Julie wandered away from her school group
• Why Julie talked to a stranger
• Why Julie changed her name to Mary Budd Rowe.

And then there’s the problem that two of the three test questions focus not on the story but on the Afterword, with its discussion of the Special Theory of Relativity

Here is one of the three tedious writing prompts accompanying this item: “Pretend you are either Julie OR Einstein. Write a letter to a friend describing your meeting at the fountain, and what you thought about the person you met. Use details from the story and the ‘Afterword’ in your letter.” What fourth grade boy is willing to pretend he’s a girl? This leaves him the job of writing in the persona of Einstein.

D. H. Lawrence once observed, “Without secrecy there would be no pornography.” Surely this applies as much to the testing industry as to obscenity. Without secrecy, there could be no pretense that tests marketed as measurements of reading ability aren’t actually measuring something else entirely. But the State colludes with test publishers to keep these tests secret because without secrecy, the state loses the leverage that helps it do the bidding of the corporate-politico alliance that has been calling for high standards and testing since the 1980ies, and alliance bent on keeping teachers and students scared and subservient. They call it “rigor,” a word worthy of study.

If scholars could employ methods of discourse analysis and talk with schoolchildren about why they chose the answers they did, we would all learn a lot about the pervading adult perspective of the tests, about cultural and class assumptions that dominate the tests. We would also see how convincing children's "wrong" answers are and how the reading tests aren’t about reading ability at all and how the math tests are complicated by many other factors besides facility with numerical relationships. Imagine asking Massachusetts, New York, Florida, Texas, and California fourth graders why they answered questions the way they did. The whole lid would come off state-run accountability systems. And corporate-politicos would scream that our position in the global economy would wither. Never mind that among industrialized nations, we already rank 20 out of 21 in child well-being.

Ask fourth graders why they answered questions the way they did and public schools across America could no longer be the scapegoat for a host of bad business decisions. There’s a reason the Business Roundtable is the chief cheerleader for the renewal of NCLB. So-called accountability in education is, of course, modeled on what passes for accountability in corporate America, i.e., an increase in productivity. As Kathy Emery and I point out in Why Is Corporate America Bashing Our Public Schools? (Heinemann 2004 ), from the hand-holding at the 1989 corporate-politico education summit in Charlottesville through the passage of the No Child Left Behind Act in January 2002, the Business Roundtable has been pushing tests as the chosen way to measure school productivity. The Business Roundtable doesn’t veer from its message, which is echoed in newspaper headlines and on many editorial pages ranging from the New York Times to the rural weeklies: school effectiveness (and hence teacher quality) is rated up or down according to student scores on tests. Any attempt to explain that schools are concerned, not with widgets but with students in all their variety, is met with the rejoinder “No excuses.”

In a list of ten arguments offered by supporters of the NCLB legislation, Nichols and Berliner posit number five as the theory of action behind the law, the very heart of NCLB: Teachers need to be held accountable through high-stakes tests to motivate them to teach better and to push the lazy ones to work harder. Nichols and Berliner counter that, based on their first-hand observation in hundreds of schools, they believe the percentage of lazy teachers “is considerably smaller than the percentage of lazy politicians who do not read the legislation they support.” Hoo-hah! Let’s put this up in faculty rooms across America.

I admit that, even though the authors assert that validity is the most important characteristic of a test,” for me, worrying over the distinctions between content validity, construct validity, criterion validity, and consequential validity is truly academic. Unless and until we make tests public, allowing educators to talk to children about why they responded the way they did, then I say all this is spinach and I say to hell with it.

That said, in Chapter 5, What Happens to Validity When Indicators are Distorted and Corrupted, the Many Ways That High-Stakes Testing Promotes Such Distortion and Corruption, and How Those Practices Lead to Confusion About What Test Scores Mean, Nichols and Berliner pose a question that should be on the lips of everyone who plans to spend the next couple of decade in this country:

Are we sure we want to live with the consequences of high percentage of minority students not finishing high school?

I thank the authors for caring about the students at the World of Opportunity in Birmingham, Alabama, a school with which I have longstanding connections. I thank them for showing readers that we all have a stake in the horrendous policy of pushing kids out of school. And I would extend their question to the assaults on younger children, also documented in the book. Do we want to live with the consequences of children growing up without art, music, and recess, children who have been condemned to a school life of relentless test prep? Children who have grown up hearing the constant message that they don’t measure up? That they aren’t good enough?

The school board in my village is currently discussing a policy of depriving middle schoolers entrance to school dances if they aren’t up-to-date on their homework assignments. Surely the obsession with homework would not be so prevalent if it weren’t for the obsession with making adequate yearly progress on the tests required by NCLB. Vermont changed its whole testing product and procedure in order to march behind the U. S. Department of Education drumbeat. We don’t yet have a high school exit exam, but one can wonder how long our politicos will hold out. I think of the slightly altered version of Nichols and Berliner’s question and I know I don’t want to live next door to citizens who were demeaned and diminished as primary graders, excluded from school dances as middle graders, and denied high school diplomas. Such kids will grow up to be very angry adults. Children trained in the Business Roundtable model of a dog-eat-dog world of maximized global economy profits don’t get educated for the common good. If for no other reason than our own well-being, we must all care about this.

The book ends with a bang. The authors call for an immediate moratorium on the use of high-stakes testing and they give a dozen strong reasons why NCLB cannot be defended. They do this to “stop the wreckage of our public educational system.” Throughout, they have shown that if we allow our public educational system to be wrecked, our very social fabric will crumble.

About the Reviewer

Susan Ohanian Susan Ohanian, a longtime teacher, is a Fellow at the Vermont Society for the Study of Education. She is a co-founder of Educator Roundtable. In addition, she is a free-lance writer whose articles have appeared in periodicals ranging from the Atlantic and Washington Monthly to Phi Delta Kappan and Education Week. Susan is the recipient of The George Orwell Award for Distinguished Contributions to Honest and Clarity in Public Language, National Council of Teachers of English, 2003; The Kenneth S. Goodman "In Defense of Good Teaching" Award, College of Education, University of Arizona; and The John Dewey Award for Extraordinary Contributions to the Education of Young People In America (2006). Copyright is retained by the first or sole author, who grants right of first publication to the Education Review. Editors: Gene V Glass, Kate Corby, Gustavo Fischman ~ ER home | Reseņas Educativas | Resenhas Educativas ~ ~ overview | reviews | editors | submit | guidelines | announcements | search ~