This review has been accessed times since January 1, 2005

King, Patricia M. and Kitchener, Karen S. (1994). Developing Reflective Judgment: Understanding and Promoting Intellectual Growth and Critical Thinking in Adolescents and Adults. San Francisco: Jossey-Bass Publishers

351 pp.

$37.95           ISBN: 1-55542-629-8w99

Reviewed by Jan van Aalst, Simon Fraser University
                    Steven Katz, OISE/University of Toronto

August 11, 1999

          The intended audiences of this book are educators who work with college students and who attempt to promote critical thinking among high school and non-college students, those who teach about college students in higher education programs, faculty and administrators with responsibility for the assessment of college outcomes, and developmental psychologists who seek to understand the nature of cognitive development (pp. xviii-xx). The book discusses more than 15 years of theory building and research on what King and Kitchener call the Reflective Judgment Model (RJM)--a stage model of how assumptions about knowledge and concepts of justification develop throughout adolescence and (young) adulthood. Chapters 1 and 2 describe reflective judgment and previous work on it, distinguishing it from critical thinking, and provide a preliminary account of the model; a fuller description of the epistemic assumptions and concepts of justification for each stage of the model is given in chapter 3. In chapter 4 a set of criteria is proposed for assessing reflective judgment; the authors argue that extant practice in the assessment of critical thinking and postformal thinking does not meet these criteria. In chapter 5 the Reflective Judgment Interview (RJI) is described as an alternative assessment instrument, and validation studies are discussed. Chapter 6, which the authors view as the centerpiece of the book, discusses research on reflective judgment. This chapter was written in collaboration with Phillip K. Wood. Chapter 7 examines the relationship of reflective judgment to critical thinking, and chapter 8 to character development. The final chapter describes some possible educational strategies. The book has a number of extensive appendices which provide further details on the RJI and the studies discussed in the main text.
          King and Kitchener begin their argument (after a brief interview transcript excerpt) with a discussion of John Dewey's writing on reflective thinking:
[Dewey] observed that true reflective thinking is initiated only after there is recognition that a real problem exists. Such real problems, he argued, cannot be solved by logic alone. Rather, they are resolved when a thinking person identifies a solution to the problem that temporarily closes the situation. True reflective thinking...is uncalled for in situations in which there is no controversy or doubt, no concern about the current understanding of an issue, or in which absolute, preconceived assumptions dominate. (p. 6)
Thus the development of thinking King and Kitchener are interested in has more to do with such issues as the ability to recognize that there is a problem and that some problems may not be stated with certainty, than with logical argument. Problems that require reflective judgment--they call these ill-structured problems--go beyond such problems as "figuring out the circumference of a circle," "translating a set of instructions into a computer language, or playing a game of chess," which, the authors claim, can be solved "quite adequately" on the basis of mathematical formulae, logic, or rules of play (pp. 6-7). The choice of examples here is unfortunate. For example, any design problem involves an evaluation of competing design constraints which cannot be done on the basis of logic alone; nor can one know with certainty with which move, consistent with the current board configuration and the rules of the game, a player's opponent will respond. (Figuring out the circumference is a well-defined problem, but does not seem to require formal thought.) The authors seem to oversimplify problem solving. Nevertheless, there are many problem solving situations in which the uncertain nature of knowledge is not evident, such as items on the Cornell Critical Thinking Test (CCTT) and the Watson- Glaser Critical Thinking Appraisal (WGCTA). These tests, they claim, invoke cognitive or metacognitive processes rather than epistemic assumptions of the respondents. (p. 12).

Description of the Reflective Judgment Model

          The model is based on Perry's (1970) work on reflective thinking (Note 1), as well as works by a variety of philosophers (e.g., Popper, Lakatos, Dewey), and has undergone further development since the authors' first study of reflective judgment (Kitchener & King, 1981). According to the current model, there is a progression of seven distinct sets of epistemic assumptions about knowledge and how knowledge is acquired; each set has its own logical coherency, and is called a stage. Each successive stage is "posited to represent a more complex and effective form of justification, providing more inclusive and better integrated assumptions for evaluating and defending a point of view. ...The more advanced sets allow greater differentiation between ill-structured and well-structured problems and allow more complex and complete data to be integrated into a solution" (p. 13). The stages, they claim, are traversed in an invariant order, but "homogeneity associated with stagelike development decreases with age" (p. 22), and people at any time operate in a range of stages, depending on, for example, the problem context. The stages can be mapped onto the seven levels of Fisher's ( 1980) skill theory. In this theory, seven levels, marked by qualitative differences in the skill structures an individual can control, occur between ages 2-30. Further, a person's best performance in a familiar domain improves sharply when a new level is emerging. Optimal performance requires that performance is induced and supported by the environment (p. 34). During the plateau the new skill structure is elaborated and applied in new areas. King and Kitchener hypothesize that the assumptions associated with each RJM stage develop concurrently with the corresponding skill level in Fisher's theory.
          We summarize (the authors' account of) the seven stages of the RJM. The stages fall into three groups indicative of pre-reflective thought (Stages 1-3), quasi- reflective thought (Stages 4 and 5), and reflective thought (Stages 6 and 7).
  • Stage 1: Students use single, undifferentiated categories; no justification concept is needed because there is assumed to be an absolute correspondence between what is believed to be true and what is true. Students do not see discrepancies between two views or see that two people disagree on an issue (pp. 48-50).
  • Stage 2: There is a true reality that can be known with certainty but is not known by everyone. (p. 51). Certain knowledge is seen as the domain of authorities, and those who disagree with authorities are wrong. Defending one's point of view is not done to explain the reasons for beliefs but rather to show (by stating them) that one's own beliefs are right and those who believe otherwise are wrong" (p. 52). Underlying the Stage 2 belief system is the ability to relate two or more instances of something, but the individual "considers knowing in relationship to concrete issues and not as an abstraction" (p. 53). Knowledge is assumed to be available directly through the senses or via authority figures. Students also often try to align evidence with their own views by distorting evidence. When they do perceive differences, they assume these can be resolved simply.
  • Stage 3: Although in some areas even authorities may not have the truth, at some point in the future there will be knowledge from concrete data. In cases where authorities do have answers, beliefs continue to be justified on the basis of the word of an authority. This stage involves further differentiation of stage 2 categories into simple concrete systems. Where authorities do not know, people can believe what they want (p. 55). Assumptions do not reflect an understanding of evidence as an abstraction, a relational concept (p. 56).
  • Stage 4: This is the first of the quasi-reflective stages. One cannot know with certainty, and there is the emergence of knowledge as an abstraction, not limited to concrete instances. However, knowledge and justification remain poorly differentiated from each other. (p. 58). Although students may acknowledge that opinions do not form sufficient basis for developing an argument, "they are not consistent in their use of evidence for this purpose" (p. 58). Students are likely to be unable to differentiate a theory from evidence for the theory, thus "they cannot perform the necessary mental operations that would allow them to evaluate the theory on its own merits" (p. 60).
  • Stage 5: While people may not know with certainty or directly, what is known is always limited by the perspective of the knower--a position sometimes referred to as relativism. The underlying concept of this stage is the ability to relate two abstractions. In this stage, students can differentiate an event from the interpretation of the event. However, knowledge remains context-bound--the individual has not yet developed the ability to relate several abstractions into an abstract system. Students do recognize that there are alternate theories and that some evidence does not support one theory, but what is missing is "the ability to coordinate the two in a well-reasoned argument" (p. 64). Nevertheless, while students are unable to perform the above coordination, they are "trying on one frame of reference after another" (p. 66), and this provides the basis for synthesis.
  • Stage 6: Knowing is a process that requires action on the part of the knower, and that knowledge must be understood in relationship to context and evidence. Students are now able to coordinate the subtle similarities and differences of abstract relationships into intangible systems; they make decisions on the basis of compelling evidence rather than for idiosyncratic reasons.
  • Stage 7: While "reality is never a given, interpretations of evidence and opinion can be synthesized into epistemically justifiable conjectures about the nature of the problem under consideration" (p. 70). People in Stage 7 take on the role of inquirers; they are agents involved in constructing knowledge (p. 70); they recognize that their knowledge claims may later be superseded by others. There also is an ability to integrate several Stage 6 systems (knowledge and justifications about scientific issues, social science issues, etc.) into a general framework about knowing and justification. This allows for a generalization of assumptions and a clarity of judgment that were not present at Stage 6. This stage goes beyond Perry's model--the idea of being committed to a point-of-view does not fit in this stage (p. 71).
          One example may illustrate how the stages map onto the Fisher levels. In Level 4, two or more representational systems are related to produce a new way of thinking--single abstractions. This is the first of the abstract levels, and represents the ability to construct and operate on intangible categories; Fisher claims this occurs early in adolescence. In a single abstraction about knowledge, in the RJM, a person can coordinate concrete observations that different people draw different conclusions on the basis of their own circumstances and biases (p. 32).

Evaluating Reflective Judgment

          Chapter 4 turns to evaluating reflective judgment. The authors present a set of seven "desirable features of a measure of reflective thinking:" (a) Ill-structured tasks should be the focus of the assessment task; (b) assessment should elicit information about the rationale for a response as well as its content; (c) thinking should be sampled across a variety of issues; (d) the content should be generally familiar to a wide range of individuals; (e) the content should not constrain the instrument's use to individuals in educational settings; (f) the reading level should be such that it can be used with a wide range of potential test-takers; and (g) if the measure is based on a model that purports to describe the development of reflective thinking, the theoretical model on which it is based should be validated. As already mentioned, on the authors' view tests of critical thinking such as the Watson- Glaser test do not have the desired focus on ill-structured tasks.
          The Reflective Judgment Interview (RJI) discussed in chapter 5 typically consists of four ill-structured problems and a set of standard follow-up questions. Each question focuses on at least one major concept of the Reflective Judgment Model (i.e., the assumptions about knowledge and concepts of justification). The four standard problems cover a range of issues; each problem is defined by two contradictory points of view and is designed to focus on problems in the intellectual domain. An example is a problem on the safety of chemical additives to foods:
There have been frequent reports about the relationship between chemicals that are added to foods and the safety of these foods. Some studies indicate that such chemicals can cause cancer, making these foods unsafe to eat. Other studies, however, show that chemical additives are not harmful, and actually make the foods containing them more safe to eat. (p. 101)
          A fifth problem on nuclear waste has also been used. All five of the standard problems, as well as several discipline-based problems (in chemistry and psychology), and instructions for conducting and scoring the interviews are provided in an appendix.
          The second part of chapter 5 addresses the validity and reliability of the RJI. The authors insist that trained and certified people must be used for conducting and scoring the interviews. After a presentation of verbatim discussions of one standard problem (two interviews), interrater reliability and agreement, test-retest reliability, and internal consistency (between the standard problems) are examined. This part of the chapter provides an informative account of the kinds of issues one must attend to in test development; it should be of interest to graduate students.
          King and Kitchener consider the scores of two raters to be in agreement when they are discrepant by less than one stage (p. 110). Most of the 32 studies that used the standard RJI problems report (Pearson) correlations between ratings given by two blind raters, as well as percentage agreement; Cohen's kappa, which corrects percentage agreement for chance agreement, is not reported. Almost 40% of the studies reported an agreement of at least 87%, and 25% an agreement of at least 90%. The authors conclude that trained raters can score RJIs consistently; from four studies in which there was less than 70% agreement they conclude that "some raters may be differentially evaluating cues in the transcripts from adult students or that older students may be responding in more diverse ways to the interview questions" (p. 111). Test-retest reliability was .71 for a three-month interval, and .87 for a two-week interval (p. 112). For 14 studies that met the criteria of using all four problems and reported (Cronbach's) alpha, the median alphas were .79 and .85. A second measure of consistency across problems used was the inter-problem correlations (problem 1 with problem 3, etc.). These correlations ranged from very low to very high, typically falling around .40. Problem-Total correlations (scores for any given problem with the average of the scores for the other three problems) were typically .60 and varied little from problem to problem. Thus, the authors conclude, the problems are remarkably consistent.
          Preliminary data on three domain-specific problems are also discussed. Average scores on these were "almost identical" to averages on the standard problems done by the same respondents, leading the authors to conclude that the RJI taps people's underlying assumptions about knowledge, and not knowledge about the discipline (p. 118). A test of the hypothesis based on Fisher's theory that spurts in performance in cognitive skills occur between ages 14-15, 18-20, and between 23-25 is reported. Respondents were presented with two prototypical responses to a problem for each of Stages 2-7 and asked to summarize the statements in their own words. The study (Kitchener, Lynch, Fisher, and Wood, 1993) found evidence for spurts and for improved scores due to practice, as well as a ceiling effect-- students younger than age 23, even with support and practice, did not accurately paraphrase the statements that reflected Stage 7 reasoning (p. 122).

Research on Reflective Judgment

          We focus our review on chapter 6, describing the findings reported in chapters 7 and 8 very briefly. Chapter 6 asks six questions about reflective judgment (pp. 124-25):
  1. Does reflective judgment develop between late adolescence and middle adulthood?
  2. How do high school, college, and graduate students reason about ill-structured problems?
  3. Does student reasoning improve with additional exposure to and involvement in higher education?
  4. Do adult learners differ from traditional-age students in their reflective thinking?
  5. How does the reasoning of adults who have not participated in higher education compare to the reasoning of those who have earned a college degree?
  6. Are there gender or cross-cultural differences in the development of reflective thinking?
          The first part of the chapter deals with a 10-year longitudinal study which traced the development of reflective judgment of 80 individuals between 1977 and 1987; interviews were conducted in 1977, 1979, 1983, and 1987. At the first three testings, the four standard problems of the RJI were used; the problem on nuclear waste management was added at the fourth testing (p. 129). The authors are careful to point out limitations of the studies that arise from the fact that not all individuals were interviewed at each of the testings (p. 131). They analyze three groups of individuals: (a) 38 people who participated in all four testings, (b) 13 who missed only one testing, and (c) 80 who participated in any one testing. Consistent patterns of increasing RJI mean scores are reported for all three groups at each subsequent testing (Note 2); RJI mean scores of former high school junior students showed the largest gains, 2.9 to 5.5. Examining this gain in more detail, the authors conclude that the greatest amount of growth--3.9 to 5.3--was over a four-year period between 1979 and 1983, and that no other group came close to this "spurt." The period in question coincides with college attendance for 75% of the participants, and is predicted by Fisher's theory; thus, King and Kitchener conclude, the spurt may be due to a combination of education and development (p. 136).
          Also discussed is a study of the variability of the scores over stages, that is, of the extent to which individuals in the 10-year study were operating in a spread of stages. The study (Wood, 1983) re-analyzed data from 15 studies, calculating the proportion of time each stage was represented in an individual's rating across four standard problems, referring to this as the 'stage utilization score'. Wood performed a spline regression of these scores against the overall reflective judgment score. As King and Kitchener discuss, the analysis showed that the spread over stages increases with the dominant stage, the stage most frequently assigned at scoring. (For example, it is smaller for scores with the highest percentage utilization score at Stage 3 than for scores for which it is at Stage 6.) The authors point out that for Stages 2 and 7, the slopes of the curves may be distorted by the fact that only one adjacent stage is used (p. 140), which is another example of the care the authors take in qualifying their claims. The section concludes with a discussion of two examples of verbatim RJI transcripts, which helps to illustrate the various changes in RJI scores over time (p. 141-146).
          To examine the influence of age on RJI scores, all data in the 10-year study were pooled to calculate the modal (in some cases bimodal) frequencies for each stage over 5-year intervals. For Ages 16-20 the model stage was 3, for Ages 21-25 Stage 4, for Ages 26-30 Stage 5, for Ages 36-40 Stage 6, and above Age 40 Stages 6 and 7. There was a strong relationship between age and the stage of development, but the authors caution that the subjects were in a variety of educational pursuits at the times of the testings (p. 150).
          The second part of chapter 6 deals with data found across six other studies that use the RJI, using 12 different samples and 214 individuals. In all samples tested, scores either stayed the same or increased with time; and with two exceptions, the mean scores increased significantly for all groups tested in 1-4 year time intervals. (The exceptions were adult college freshmen and advanced doctoral students.) Another finding across the studies is that the amount of change over time is strongly related to the duration of time between testings (p. 146). The authors examine not only increases with age, but also "regressions," (decreases with time). The conclusion from the analysis of reversals is that "stability rather than development characterized the thinking of some participants at some of the longitudinal testings suggest[ing] that the rate of change across stages varies across individuals" (p. 157).
          Next, several cross-sectional studies are discussed--in different regions and across 10 educational levels. The aim of their analyses was to discover how individuals at different age and educational levels think about and resolve ill-structured problems, but as the authors point out, age and educational level are often confounded--age brings with it access to a greater range of educational opportunities. (p. 160). The mean score for undergraduate students (traditional-age and non-traditional) is 3.8; for graduate students it is 4.8. Scores increase, but so does variability. As with the 10-year study, scores are not representative of US high schools--of 11 the samples examined, 7 were from academically talented students (p. 162). An analysis of 20 cross-sectional studies of traditional-age college students that used standard RJI scoring techniques showed that only one sample of 44 averaged above 4.0. The authors comment that the prevalence of stage 3 reasoning "means that like the high school students, many of the college freshmen articulated the belief that absolute truth is only temporarily inaccessible, that knowing is limited to one's personal impressions about the topic (uninformed by evidence), and that most if not all problems are well structured (defined with a high degree of certainty and completeness)" (p. 165). Nevertheless, it is apparent that 2/3 of freshmen reasoned between stages 3 and 4, "suggesting that other freshmen were beginning to accept the concept that uncertainty may be an ongoing characteristic of the knowing process. Consequently, they are beginning to accept the idea that some problems are truly ill structured" (p. 166). In addition, although there is only a small difference during college (3.6 for Freshmen to 4.0 for seniors), the stage 4 reasoning prevalent among the senior samples is "clearly more adequate and defensible" than the reasoning it replaced.
          An analysis of reflective judgment of 196 graduate students distinguished between students in the first two years of graduate studies and those beyond that (labeled 'advanced doctoral'). The authors observe that graduate students were the first to consistently demonstrate the use of Stage 5 assumptions on the RJI; moreover, there was a 0.7 stage increase from beginning to advanced doctoral levels, and only at the latter level was there there evidence of the consistent use of Stage 6 assumptions on the RJI (p. 172). They point out that graduate admissions committees may select students who have already demonstrated advanced reflective thinking skills, and that it may be no accident that students who scored high were in programs that required a thesis (p. 173). One study showed disciplinary differences (King, Wood, & Mines 1990): Graduate students in social science disciplines scored significantly higher than did their counterparts who were enrolled in the mathematical sciences (p. 173).
          King and Kitchener discuss gender effects for the 10-year study as well for the studies in the second part of the chapter. At the first two testings (1977 and 1979) of the 10-year study, no statistically significant effects were found, although a Time by Gender effect "approached significance" (p. 147); in the fourth testing, men scored higher than women. Of the 14 studies discussed in the second part of the chapter, 6 reported higher scores for men and 6 reported no statistically significant effects. The authors state that the available research that used the RJI is inconclusive about gender effects.
          In chapter 7, two studies are discussed that examined if development of reflective judgment could be due to increases in logical reasoning skills. From the first (Brabeck, (1983a), King and Kitchener conclude that "while only those scoring high in critical thinking scored above a Stage 4 in reflective judgment, high WGCTA (Watson-Glaser Critical Thinking Appraisal )scores did not uniformly lead to high RJI scores." (p. 191). There also was more variability among RJI scores for the high critical thinking group. The second study (Brabeck & Wood, 1990) retested 25 high school students from the original Brabeck (1983a) sample on both the RJI and WGCTA. No differences were found on the WGCTA scores, although differences were found for the RJI scores. King and Kitchener suggest that "while the development of critical thinking skills may not continue into the early college years, development in reflective judgment clearly does so" (p. 192).
          The chapter continues with two longitudinal studies of the relationship between reflective judgment and verbal reasoning. In the first study, three groups of students were tested at two-, six- and ten-year intervals, using the RJI and Terman's (1973) Concept Mastery Test (CMT). There was development over time on both measures, but the amount of development differed by group, the youngest group (Age 16) showing the largest increase on both measures. Changes in RJI scores could not be accounted for statistically by changes in CMT scores. The second study led to a similar conclusion. Finally, one study (King, 1977) is discussed that examines the relationship between formal reasoning and reflective judgment. King's sample included graduate students, college juniors, and high school students who were matched on scholastic aptitude. While there were significant differences between the three groups' scores on the RJI, there were no differences in the scores on the formal operational tasks; the correlation between the two measures was also "extremely low" (p. 201).
          Chapter 8 examines the relationship between reflective judgment and character development. Here, King and Kitchener argue that moral questions are different from other questions requiring reflective judgments:
Epistemological issues that would correspond to the moral question of whether to discontinue life supports include evaluating the prognosis for recovering and estimating the long-term financial and emotional effects on the family. Part of the ill-structured nature of moral problems derives from different values or conceptions of the good rather than from epistemological issues about the nature of knowledge. (p. 206)
          But still there is a structural similarity in development of the two kinds of thinking. In a longitudinal study that was concurrent with Kitchener and King's (1981), a moderate correlation was found between RJI scores and the Defining Issues Test (DIT; Rest, 1979). The data is summarized by the claim that the development of reflective judgment is necessary but not sufficient for the development of moral thought, as measured by the DIT (p. 210). King and Kitchener suggest that the structural similarity implies that "there may be mechanisms underlying development in both domains...that may affect how the individual operates on both sets of tasks" (p. 207).
          In the final chapter, King and Kitchener make several suggestions based on the research they discuss that instructors can use to help students question their assumptions about knowledge (p. 223). They detail each of the proposals in several exhibits (pp. 231-255). The proposals range from tolerance for and awareness of the epistemelogical assumptions students bring to learning tasks, to proposals for acting on such knowledge. For example, in proposal 2 the authors observe that students differ in regard to their epistemic assumptions, and in proposal 7 that students may view learning tasks differently as a result. Naturally, the proposed teaching strategies involve practice with considering ill-structured problems from different viewpoints. These proposals repeat, at another level, similar calls for taking students preconceptions into account in teaching (e.g., Ausubel, 1968; Diver & Easley, 1978).

Discussion

          We end with some observations about this book and the research it discusses. First, it is an excellent account of a large body of research. It is well written, and the authors have clearly made much effort not over-generalize from the studies. The authors conclude each chapter with a succinct summary, which is useful for readers who need a quick reference. Moreover, readers of the full text will benefit from reading the summaries prior to immersion in the chapter specifics, a practice that the authors and we especially advocate for the detail-rich chapter 6.
          Although King and Kitchener argue in chapter 2 that the RJM is noticeably distinct from other models of epistemological development, we suggest that such differences lie in the details of the models, the most obvious of which deal with terminology. In their review of models of epistemological development in late adolescence and adulthood, Hofer and Pintrich (1997) point out the structural similarities which exist between the RJM and the work of other theorists such as Baxter Magolda (epistemological reflection), Belenky et al. (women's ways of knowing), and Perry (intellectual and ethical development). We reference this not to criticize King and Kitchener for a lack of originality, but to validate what is emerging from diverse arenas as a cogent and important domain of human development. Indeed, Kuhn (1999) has recently argued that the popular critical thinking movement has much to gain by considering the development of epistemological meta-knowing in relation to its subject matter, a lacuna that King and Kitchener's work can help to fill.
          From the standpoint of applied educational research, the RJM, like other recent models of epistemological development, makes its greatest contribution in offering a way of knowing that bridges the apparent gap between absolutism and relativism. We refer here to stages 6 and 7, to what the authors call "reflective judgment." Olson and Katz (in press) document the ways in which practicing teachers are challenged to both meet the demands of a fixed school curriculum and the individual needs of children with their varied interests, backgrounds, and understandings that make up a class. Such bipolar concerns, they argue, are problematic because the poles tend to collapse onto the orthogonal epistemological positions of absolutism and relativism respectively. Stage 7 offers a potential solution to this pedagogical and epistemological paradox because it purports that knowledge, although constructed, can be constructed from multiple viewpoints, and that insight gained from each perspective must be synthesized into a knowledge system (see also diSessa, 1996; van Aalst & Scardamalia, in preparation). Olson and Katz explain that the "known" and the "knower" can be recognized and related this way; they believe that is what Dewey (1902/66) had in mind in The Child and the Curriculum.
          King and Kitchener point out that the data they discuss are drawn from a predominantly white population in the Mid- West, with high school students matched to participants who enrolled in doctoral programs on scholastic aptitude; therefore, this population is not representative of American college students. Further, in relation to generalizability, the emphasis on knowledge revision and abstract knowledge in Stages 6 and 7 is an artifact of Western culture, and may be less prevalent in other cultures (Bidell & Fisher, 1992).
          Although the authors report that average scores for three discipline-based problems (in chemistry, business, and psychology) have been "almost identical to the scores earned by the same participants on the standard RJI problems" (p. 118), there is some suggestion from other sources, at least in science and mathematics, that assumptions about knowledge do vary with problem context and subject matter domain (Roth & Roychoudhury, 1994; Schoenfeld, 1992; van Aalst & Key, under review). We suggest that there is sufficient reason to reconsider the emphasis on epistemological issues in isolation from subject matter and problem solving contexts that runs through King and Kitchener's work.
          An important issue that King and Kitchener leave open is the relevancy of reflective judgment to solving (ill-structured) problems. Although the claim that reflective judgment requires ill-structured problems is persuasive, it must also be demonstrated that developing reflective judgment benefits problem solving ability. Thus we would like to see longitudinal studies which document people's struggles with complex issues such as those used for the standard problems over a period of time, studies that can elaborate which aspects of reflective judgment enhance problem solving ability--or the converse. It could be argued that a single measure of reflective reasoning, such as the RJI provides, confounds issues that are better examined separately in connection with problem solving. For example, knowledge as authority-based versus uncertain knowledge is a separate issue from knowledge as isolated pieces versus knowledge as a coherent system (Hammer, 1994). Further, like all stage theorists, King and Kitchener are faced with the problem of having to explain the mechanism(s) by which individuals advance through the stages. Without a clear understanding of such mechanisms, little progress can be made in designing the interventions that the authors say are necessary for "optimal performances" (p. 34). In particular, the seemingly inviting effects of formal education await empirical investigation in a way that will untangle the maturation-education interaction. (This would require studies that involve younger children). Only then will the instructional recommendations that constitute the substance of chapter 9 move beyond the realm of rhetoric.

Notes

  1. Perry observed a progression of nine points of view, which he called positions (p. 36). These positions traced out the evolution of students' thinking about knowledge, truth, and values. Initially, students used "discrete, absolute, and authority-based categories..." and "people did not perceive legitimate conflicts about knowledge" (pp. 36-37). In positions 3-4, which he called 'multiplicity', students began to acknowledge multiple answers to complex questions. He identified major changes at positions 5 and 6, which he called 'relativism', and hypothesized that students begin to see knowledge as contextual and not necessarily true, leaving students with the need to make a decision. Perry called positions 7-9 'commitment to relativism.'
  2. One exception were former doctoral students, who remained constant after 1989.

References

Ausubel, D. (1968) Educational psychology: A cognitive view. New York: Holt, Rinehart and Winston.

Bidell, T.R., & Fisher, K.W. (1992). Beyond the stage debate: Action, structure, and variability in Piagetian theory and research. In R.J. Sternberg & C.A. Berg (Eds.), Intellectual development (pp. 100-140). New York: Cambridge University Press.

Brabeck, M. (1983). Critical thinking skills and reflective judgment development: Refining the aims of higher education. Journal of Applied Developmental Psychology, 4 (1), 23-34.

Brabeck, M., & Wood, P.K. (1990). Cross-sectional and longitudinal evidence for differences between well- structured and ill-structured problem solving abilities. In M.L. Commons, C. Armon, L. Kohlberg, F.A. Richards, T.A. Grotzer, & J.D. Sinnott (Eds.), Adult development 2: Models and methods in the study of adolescent and adult thought. New York: Praeger.

Dewey, J. (1902/1966). The child and the curriculum. Chicago: University of Chicago Press.

DiSessa, A. (1996). What "just plain folk" know about physics. In D. Olson & N. Torance (Eds.), Handbook of human development and education (pp. 709-730). Cambridge: Blackwell.

Driver, R., & Easley, J. (1978). Pupils and paradigms: a review of literature related to concept development in adolescent science students. Studies in Science Education, 5, 61-84.

Fisher, K.W. (1980). A theory of cognitive development: The control and construction of hierarchies of skills. Psychological Review, 87 (6), 477-531.

Hammer, D. (1994). Epistemological beliefs in introductory physics. Cognition and Instruction, 12, 151- 183.

Hofer, B., & Pintrich, P. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67 (1), 88-140.

King, P.M. (1977). The development of reflective judgment and formal operational thinking in adolescents and young adults. Dissertation Abstracts International, 38, 7233A.

King, P.M., Wood, P.K., & Mines, R.A. (1990). Critical thinking among American college and graduate students. Review of Higher Education, 13 (3), 167-186.

Kitchener, K.S., & King, P.M. (1981). Reflective judgment: Concepts of justifications and their relationship to age and education. Journal of Applied Developmental Psychology, 2 (2), 89-116.

Kitchener, K.S., Lynch, C.L., Fisher, K.W., & Wood, P.K. (1993). Developmental range of reflective judgment: The effect of contextual support and practice on developmental stage. Developmental Psychology, 29 (5), 893-906.

Kuhn, D. (1999). A developmental model of critical thinking. Educational Researcher, 28 (2), 16-25, 46.

Olson, D., & Katz, S. (in press). The fourth folk pedagogy. In B. Torff & R. Sternberg (Eds.), Understanding and teaching the intuitive mind. Hillsdale, NJ: Erlbaum.

Perry, W.G., Jr. (1970). Forms of intellectual and ethical development in the college years:a scheme. Troy, MO: Holt, Rinehart & Winston.

Rest, J. (1979). Development in judging moral issues. Minneapolis: University of Minnesota Press.

Roth, W.-M., & Roychoudhury, A. (1994). Physics students' epistemologies and views about knowing and learning. Journal of Research in Science Teaching, 31 (1), 5-30.

Schoenfeld, A. (1992). Learning to think mathematically: Problem solving, metacognition and sense-making in mathematics. In D.A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 334-370). New York: Macmillan.

Terman, L.M. (1973). Concept mastery test manual. New York: Psychological Corporation.

Van Aalst, J., & Key, T. (under review). Physics expectations of pre-professional students. Submitted to American Journal of Physics.

Van Aalst, J., & Scardamalia, M. (in preparation). A comparison of radical constructivism and knowledge building. To be submitted to Science & Education.

Wood, P.K. (1983). Inquiring systems and problem structure: Implications for cognitive development. Human Development, 26, 249-265.

About the Reviewers

Jan van Aalst, Simon Fraser University, Burnaby, B.C., CANADA, V5A 1S6

Steven Katz, OISE/University of Toronto, Toronto, ON, CANADA, M5S 1V6

[ home | overview | reviews | editors | submit | guidelines | announcements ]