reseņas educativas (Spanish)
resenhas educativas (Portuguese)
This review has been accessed times since May 6, 2005
Wilson, Mark (Ed.). (2004). Towards Coherence between Classroom Assessment and Accountability. Chicago, IL: The University of Chicago Press
Pp. xii + 290
$39.00 ISBN 0-226-90139-4Reviewed by Bo Zhang
University of Wisconsin – MilwaukeeMay 6, 2005
People interested in or puzzled by the relationship between classroom assessment and accountability of educational systems should read the book “Towards coherence between classroom assessment and accountability”. As the 103th yearbook of the National Society for the Study of Education, this book addresses that challenging relationship by introducing five accountability systems, each solidly based on everyday classroom assessment. As stated by Mark Wilson in the introduction chapter, a coherent relationship between classroom assessment and accountability must have channels set up for two-way information exchange: to make state accountability assessment relevant and useful for classroom instruction and to make classroom assessment consistent with and supportive of the accountability system. While it is not hard to see the value of such two-way information exchange, to implement it is by no means easy. In many accountability systems, information from accountability tests is buried in a few reported standard scores, which are hard to use in classroom instruction. On the other hand, the rich information from everyday classroom assessment is not utilized to assess system accountability.
This book is divided into four parts. Part one is an introduction chapter. Part two include five source chapters, each devoted to one accountability system in reasonable detail. The focus is on how coherence is achieved between daily classroom activities and accountability system. Two of the five systems are from countries other than the United States, one from Australia and the other from the Great Britain. This seems to show that while different countries may have different educational systems, the need of such coherence is shared. Part three have ten commentary chapters. They either evaluate the five systems in Part two or address some other important issues on how to establish the coherence. Contributors are either authors of the source chapters or experts in the educational assessment field. Diverse perspectives are reflected in those commentary chapters. Part four is a summary chapter, talking about the appropriate degree of coherence.
The first accountability system described in the book is the math and science project designed by Paul Black and Dylan Wiliam from the Great Britain. This system is built upon the two authors’ strong belief that formative assessment enhances learning. Thus as Margaret Jorgensen pointed out in her commentary chapter in the book, the focus of this system is on learning rather than on accountability. Formative assessment is distinguished from the summative assessment not by when the assessment is given, but by whether there is further learning after the assessment. If assessment results are used to improve learning, it is formative assessment. To make formative assessment happen, teachers need to take a much more active role than in the traditional standardized tests. They design assessment tasks, interpret test results and monitor student progress. Both students and teachers can benefit from this formative assessment. The authors provide examples on how teachers begin to ask more meaningful questions and to provide feedback more helpful to improve learning after adopting formative assessment.
However, the link between classroom assessment and accountability is not clear for this system. As teachers are endowed with the power to develop tailored assessment tasks to fit student needs, it would be hard to compare student performance across different teachers. However, such comparability is a must for any accountability purpose. If an external test should be designed for accountability purpose, the link between the classroom assessment and that test is not clear. Besides, to reflect such a rich but diverse classroom learning environment itself is challenging.
The second system is described by Margaret Forster and Geoff Masters from Australia. This system aims to embed standardized tests into classroom assessment. A progress map links classroom assessment with curriculum. The map reports student test scores along with the description of the achievement so that student’s progress could be tracked over time. The progress map is empirically constructed by observing student performance. Examples of progress map on speaking and reading from language arts class are given. For assessment, teachers are provided with standard tasks. They can also design their own tasks to assess relevant knowledge or skills in the progress map. The assessment mode they choose is not important. What is more important is that they should ensure that their students achieve the expected progress in time. Classroom teachers are the judges of student works both for the classroom assessment and the standardized tasks. To ensure teachers’ ratings are comparable, professional training is offered on grading and interpreting students’ work. As classroom assessment tasks and accountability assessment are designed under the same progress maps, coherence could be established.
The third system suggested by John Frederiksen and Barbara White puts students at the center in establishing the coherence between classroom assessment and accountability. Like the first system by Dylan and Wiliams, this system also incorporates the concept of formative assessment. The emphasis of assessment is on how to analyze students’ work. The authors believe that students’ work should be assessed in both formative and summative manners– formative for learning and summative for accountability. Two standards are set for a coherent accountability system. Directness requires that the assessment tasks be directly linked to the cognitive process and students have opportunity to directly show their cognitive process. The transparency standard specifies that students should be able to assess themselves and their peers in the same way they are assessed by teachers. Thus reflective assessment and peer assessment are important components of this system.
In the demonstrative ThinkerTool Inquiry system, the focus is on the inquiry activities to develop understanding of science by computer-based tools. The system aims to help students make their own inquiry process explicit. Students are instructed to evaluate their own work and one another work based on the same scoring criteria their teachers use. Studies by the authors show that reflective assessment improves student scores on science projects from both classroom assessment and state assessment. In addition, low-achieving students benefit more from the reflective assessment approach.
The authors also argue that assessment of student classroom work can be used to evaluate school’s effectiveness within the accountability system. Actually only when the results from classroom assessment are taken as part of the accountability system is it fair to compare student and school performance. One limitation of this project is that this system might work better for science subjects, where a consistent evaluation of student projects is easy to achieve. It might be hard to make standard-based ratings for courses other than science.
The fourth system by John Smithson and Andrew Porter aims to develop a multidimensional language to describe the instructional content. According to these two authors, a common language is a prerequisite for possible coherence between accountability and class activities. Instead of progress map, content map is constructed in this system. By the level of specificity, two versions of the content map could be developed: the coarse grain version and the fine grain version. A content map could be constructed for a large content domain such as the fifth-grade mathematics, or an individual content area such as understanding of number sense. According to the authors, one key issue in defining a content map is the employment of a detailed language for systematically describing practice. This language gives a common understanding of the content and pedagogy of practice for teachers, administrators and policy makers.
The DEC Project (the Data on the Enacted Curriculum) is based on this content mapping method. The standards are developed by content experts. The authors also discuss the possibility of using state standards directly. Information about curriculum enactment is collected from teacher survey and students’ log. In evaluating strength of the coherence, percentages of content area covered in standard, classroom instruction and state assessment are compared. The higher those percentages are, the more coherent the accountability system is. The other way to reflect the degree of alignment among those three areas is an alignment index, whose interpretation is similar to that of a correlation coefficient. In the research on K-8 mathematics done by the authors, the alignment index between instruction and assessment is .20, indicating students in a higher alignment classes gained more in achievement than those in the low alignment classes.
This method also provides a tool for teacher to understand exactly what is taught in their class, what is assessed in state assessment and what has been learned by students. The authors are skeptical of a standard system to delivering classroom assessment information to the accountability system as variation within and between schools might be too large.
Like the second system, the fifth system described by Mark Wilson and Karen Draney also makes use of the concept of progress map to link classroom assessment and accountability system. Furthermore, the progress map is extended into a common set of progress variables. Progress variable is not equivalent to the curriculum. It needs to be summarized from the content of the curriculum. These progress variables provide a developmental perspective of student learning. By assessing students along with the progress variables, students learning over a certain period of time could be reflected. While the authors did not describe a general methodology on how to define progress variables, a detailed example is provided on progress variable were defined from a content area. Assessment is aligned to progress variables, creating a progress map. Assessment tasks themselves could link instruction and assessment as they are designed simultaneously with assessment material around the progress variables. Examples on how to develop assessment tasks are also included.
Obviously teachers need to play a very active role in this system. They need to understand how progress variables measure student learning. Also for accountability purpose, they need to have a good understanding of the expectation of the accountability system on their students. In order to make assessments from different classroom comparable, assessment tasks need to be put into a quality control process. Variables of quality control are related to the progress variables and also to test reliability and validity issues. The target of this system is to employ standardized assessment into everyday teaching. In addition, summative assessments are designed at different transitional points of instruction.
As Larry Suter pointed out in his commentary chapter, the purpose of and information provided by classroom assessment and accountability system are different. The assessment system for accountability purpose provides summative information, which could be used to evaluate the effectiveness of a school system. On the other hand, classroom assessment reflects student performance on everyday learning task. Their purpose is to track the progress of student learning. The question thus becomes whether it is possible to build a coherent relationship between them. If possible, there is also a degree issue. How strong should that coherence be? There are also risks associated with this coherence, as pointed by Pamela Moss in the commentary chapter. Each of the above five systems represents a unique perspective on how to build that coherence. Some of them are more successful than others. To build a strong coherence, components from different systems in the book could be blended. For example, the content map described by John Smithson and Andrew Porter could be combined with the progress map presented by both Margaret Forster and Geoff Masters, and Mark Wilson and Karen Draney.
There are questions unanswered by this book. With regard to the above two-way information exchange, the focus of the five systems is on how the information from classroom assessment could help accountability purpose. The other way of information exchange is not as clear. Another issue is how to evaluate the effectiveness of the coherence established. To answer that question, some criterion variables need to be defined and empirical research with control group needs to be conducted.
About the Reviewer
Bo Zhang, Ph. D., is an assistant professor at University of Wisconsin–Milwaukee. His research is mainly on topics in educational measurement, such as item response theory, large-scale assessment and international assessment.
~ ER home | Reseņas Educativas | Resenhas Educativas ~
~ overview | reviews | editors | submit | guidelines | announcements | search ~