EDUCATION REVIEW

Educational reform efforts that seek to deeply and positively change classroom practices have been notoriously unsuccessful. James Spillane’s Standards Deviation: How schools misunderstand education policy adds an important perspective to study of this problem: how National and State policies are understood and implemented at the local district and classroom levels.

Reading Standards Deviation reminded me of a John Trever cartoon (Note 1) about the No Child Left Behind Act. In the first frame President Bush hands an Education Bill to the States, announcing, “No child left behind!” Next, a State policymaker exclaims, “Leave no child behind!” to a District superintendent, who passes the message, “Don’t let any child fall behind!” to a school principal. The latter then instructs a teacher, “Don’t let any child fall behind and fail!” The teacher exhorts parents to “Don’t fail to get behind your child!” Finally, the parents warn their child, “Don’t fail or it’s your behind!”

The cartoonist has captured one of Spillane’s main ideas, namely, that policy messages are distorted as they filter down through the various levels of educational administration. However, Spillane might argue, the cartoonist has mistakenly depicted local educational administrators as relatively passive conduits of National policy. Spillane claims that this simplistic approach to local policymaking is common also in the scholarly literature, which has largely overlooked the role of school districts in educational reform. Standards Deviation is an attempt to fill this gap, by demonstrating the active and critical roles played by local policy-makers in interpreting and shaping National and State initiatives. In this review I first give a general overview of the book and then critically discuss what I see to be its primary advantages and shortcomings.

Spillane explores the implementation process of standards-based reforms of Mathematics and Science teaching in Michigan between 1992 and 1996. He adopts a cognitive perspective, which foregrounds how agents in the implementation process make sense of the policies upon which they act. Spillane offers a vivid analogy for how this process can be conceptualized:

The book follows the “stories” of the Mathematics and Science standards as they are picked up by Michigan Department of Education officials from National agencies, and passed on to district and classroom level players. The organization of its eight chapters reflects this progression: chapter one provides an introduction to and overview of the book; chapter two discusses the context and content of State policy-making with regard to Mathematics and Science standards; chapters 3-5 explore how policymakers in nine school districts interpreted, responded to and incorporated the standards into their instructional policies; chapters six and seven examine how teachers in those districts understood the standards and enacted them in their classroom practice; and the final chapter discusses implications of the study for policy analysis, research and design.

In the late 1980s two loose coalitions of state officials, academics and school professionals in Michigan sought to significantly transform the teaching of Science and Mathematics in the state. Whereas previous policy – embodied in the State’s statement of “Essential Goals and Objectives” and in the Michigan Educational Assessment Program (MEAP) – had emphasized basic skills, the reformersendeavoured to join a National movement for more intellectually challenging standards. Spillane emphasizes that this program did not involve merely superficial modifications, but an ambitious and substantial shift in how Science and Mathematics education should be conceived.

Spillane summarizes this reconceptualization as a shift from an exclusive emphasis on procedural knowledge (i.e. “following predetermined steps to accurately compute correct answers”) to a curriculum that balances procedural and principled knowledge (i.e. conceptual understanding). Spillane expects this shift to be manifested in fundamental changes in the academic tasks given to students and in the way students and teachers discuss Mathematics and Science.

State policymakers faced multiple challenges, including severely limited resources, an unstable political environment and a fragmented governmental system. Some of these problems led to inconsistencies in the policy. For example, in a fascinating section, Spillane recounts how policymakers’ attempts to align the Mathematics standards and the MEAP were thwarted. State officials designed a testing instrument relying on multiple choice questions for one third of the examination, with the other two thirds consisting of free- or extended response questions, which were deemed to be better suited for assessing conceptual understanding. However, financial constraints and legal concerns ultimately led to a predominantly multiple choice examination, 90% of which measured procedural knowledge, according to one official’s estimate.

Successful implementation of the standards depended on what Michigan’s 545 school districts did with them. Spillane studied how a sample of nine diverse districts responded to the standards. He found that State policymaking was accompanied by a flurry of local policymaking in all nine districts, as district officials attended to the new standards and sought to align their own curricular guides, materials and professional development activities to the new State policy. However, in two thirds of the districts, modifications were for the most part superficial, primarily involving changes to topic coverage and sequencing. In only one third of the cases studied did district policy support the fundamental shifts intended by State policy.

What explains the differences between the high and low support districts? Spillane suggests that the differences stem from different understandings of what the standards entailed:

These policymakers construed new policies as being essentially similar to previous ideas they held, focussed on surface level features of policies and attended primarily to the familiar (at the expense of the novel). Spillane found that officials working in districts with low support of the standards were far more likely to have surface understandings of the policy than their counterparts in the high support districts. He also inquires into the factors that influence these different understandings, highlighting human, social and material resources.

The next stage in this policy implementation process is to follow the standards from the districts into the schools. Spillane surveyed teachers in order to inquire into the sources of instructional advice that influence them (e.g. district policies, State standards), their beliefs about Mathematics and its instruction and the teaching practices in which they engage. Teachers reported that the district policy was their main source of advice, often mediating and amplifying messages from State and national sources. Results from the survey suggest also that teachers’ beliefs about Mathematics were more closely aligned with the reform ideas (e.g. “To be good in mathematics [it is very important] for students to be able to provide reasons to support their solutions.”) than with conventional beliefs (e.g. “To be good in mathematics [it is very important] for students to remember formulas and procedures.”). Regarding teachers’ reports about their own instructional practices results were more mixed, with some practices promoted by the reforms being adopted more widely than others. A regression analysis suggests that district policies are an important factor in teachers’ (reported) practice. In the districts with high support for the standards, “teacher familiarity with their district’s… curricular guide was a significant predictor of standards-oriented instruction” (p. 135).

Spillane is aware that teachers’ reports of practice do not necessarily reflect what actually happens in their classrooms. Thus, he and his colleagues randomly selected a sample of teachers from among the 10% who reported practices most closely aligned with the standards. Each of these teachers were observed and interviewed twice. Of the 25 primary teachers observed, Spillane judged only four to be balancing principled and procedural understanding as intended by the reform policy. He terms this “level I” implementation. In Level II implementation (10 classrooms), students engage with principled tasks, but classroom discourse is devoted to procedural issues. Level III implementation (11 classrooms) can be summarized as “new activities, old mathematics”: “while we observed tasks that involved problem solving and applying mathematics to real-world situations, these tasks focused almost exclusively on procedural knowledge and facts… represented mathematics as being chiefly, often exclusively, about computing right answers using predetermined formulas and procedures” (p. 148).

These results are sobering for anyone hoping to enact meaningful reform of instructional practice. Recalling that these teachers were selected from a subsample of respondents reporting highest adherence to the standards, it is reasonable to assume that a very small proportion (e.g. 16% of %10 = %1.6?) of teachers are implementing the standards as intended. On the other hand, Standards Deviation does give some room for hope: it shows that policy is not irrelevant to practice, and that the system has worked for some of the teachers. In the final sections of the book Spillane looks closely at the Level I implementers and their circumstances. He describes teacher sense-making of the standards as complicated and arduous. Among the factors that positively impinge upon this process are sense-making resources, opportunities for investigation and a school culture that enables and encourages teacher collaboration.

I highly recommend Standards Deviation to academics and professionals interested in school reform. Its subject matter is timely and important, and it adds important perspective to studies of policy implementation, both in terms of its cognitive focus on individual sense-making and with regard to the way it highlights local district policymaking. It is very well-written: the prose is straightforward and clear, the argument coherent, and personal profiles and vignettes at the beginning of each chapter bring the material alive. I would also recommend Standards Deviation to research students interested in a good example of an elegant combination of multi-level analysis, methodological flexibility and a coherent framework.

Finally, without intending to diminish that recommendation in any way, I want to offer two criticisms of Spillane’s argument and what I see as problematic assumptions upon which it is based.

The first assumption I wish to question is that there is one correct understanding of the policies advanced. Spillane is very clear in distinguishing between policymakers’ true intention – i.e. instruction reflecting a balance of principled and procedural knowledge – and users’ surface or mis- understandings of the policy – i.e. aspects relating to the organization of topics covered and/or selection of activities. This distinction seems problematic in light of the book’s discussion of the inconsistencies in State policy. Specifically and most strikingly, the MEAP and the “Essential Goals and Objectives” reflected two different instructional approaches. Similarly, districts communicated to schools conflicting messages. As Spillane notes in his concluding chapter, “policy might best be thought about as plural rather than singular” (p. 172).

However, Spillane does not discuss the implications of this plurality for his argument as a whole. If “the policy” is actually plural, and thus lends itself to multiple understandings – why cast any interpretation except that favored by Spillane as a misunderstanding? Perhaps district policymakers and teachers have understood the mixed messages being transmitted by the State and have chosen to adopt the version upon which they will be tested. Or perhaps they have understood the ambitious intentions behind the standards, but see them as unworkable. While I am personally sympathetic to the educational aims and values implicit in Spillane’s interpretation, it is unfortunate that he has denigrated others’ divergent interpretations by casting them as the result of a cognitive shortcoming. This approach may limit a more complete understanding of the problem.

The second problem I wish to raise concerns the assumption implicit throughout the book that teachers’ beliefs and understandings about instruction are key determinants of their practice. Spillane writes, for example,

While this assumption seems common-sensical, there is good reason to believe that the relationship between a teacher’s cognition and practice is not so straightforward. Rather, much of what happens in the classroom is the product of unconscious habits and routine interactional norms. Spillane’s discussion of level III implementers is suggestive of this dynamic. He documents how discourse in these teachers’ classrooms focused on getting the right answer instead of providing opportunities for exploring mathematical reasoning.

If teaching activity is primarily governed by a teacher’s beliefs and understanding, why would the teacher interviewed find it “really hard not to give them the answer”? The answer, I believe, is that most actions in the classroom are a function of habit, not conscious decisions. They are a function of one’s identity, “as a teacher”. The teacher in this example experiences tension between what she thinks (i.e. the teacher shouldn’t always give students the answer) and the teaching role to which she has become accustomed (i.e. the bearer of truth and solver of problems). Moreover, teachers’ actions are not entirely their own. Rather, they are at least partially the product of interaction with students, who bring their own expectations about mathematics instruction to the classroom. Frustrating those expectations, at least in the short term, can create numerous other difficulties, which may partially explain the sketchy implementation Spillane documents.

In closing, I would like to emphasize that, notwithstanding these criticisms, Standards Deviation makes an important contribution to understanding the complicated yet critical problem of implementing meaningful and ambitious instructional change.

Note

1. AlbuquerqueJournal, January 10, 2002. (Available on-line at http://cagle.slate.msn.com/politicalcartoons/pccartoons/archives/trever.asp?Action=GetImage)

About the Reviewer

Adam Lefstein is a doctoral student in Educational Studies at King's College London. His research explores teacher enactment of the UK National Literacy Strategy. He has previously worked as a teacher, facilitator of teacher learning and director of a school reform program at the Branco Weiss Institute in Jerusalem.

Spillane, James P. (2004). Standards deviation: how schools misunderstand education policy. Cambridge, Massachusetts: Harvard University Press.

Reviewed by Adam Lefstein
King’s College, London

December 15, 2004

Note

About the Reviewer

Spillane, James P. (2004). Standards deviation: how schools misunderstand education policy. Cambridge, Massachusetts: Harvard University Press.

Reviewed by Adam Lefstein King’s College, London

December 15, 2004

Note

About the Reviewer

Reviewed by Adam Lefstein
King’s College, London