The Sinister Science of the Human Betterment Foundation and a Rhetoric of Motives by Katherine Swift, San Diego State

September 29, 2008 by Leave a Comment

This is a graduate student submission to the 7th Triennial Conference of the Kenneth Burke Society.

My research was conducted at the Institute Archives of the California Institute of Technology in Pasadena where I examined the original documents of E. S. Gosney and the Human Betterment Foundation, and the Historical Files on Biology Division. Caltech is the holder of Gosney’s effects since, as founder of the Human Betterment Foundation, he had its real estate holdings transferred to the university’s biology department in 1942, making Caltech an endowment worth more than $470,000. This money was used to create the Gosney Research Fund, which provided scholarships to students in Caltech’s biology and biochemistry departments with a focus on behavioral biology.  The Human Betterment Foundation Papers (or HBF) represent an original eugenic archive that offers a rare look into California’s sterilization program and the dialectics of the nature-versus-nurture debate in eugenic rhetoric. Burke’s idea of identification and the embodied symbolicity of the scapegoat form the basis for an exploration of a rhetoric of motives in eugenic technocracies premised on social improvement through social control.

Interestingly, the same root word “gene” used in the creation of the word “genocide” (Black, 402) was previously employed by Francis Galton to form the term for technologies of human breeding he dubbed “eugenics,” or the science of the well-born.  Scientistis such as Galton, Huxley and Davenport spawned an international forum for improving human heredity known as the “eugenics movement” at the turn of the 19th century. Premised on the biologically determined nature of human beings, and the belief that, thus, race and mental hygiene could solve social problems, eugenic scientists set an inexorable course for the death camps of Nazi Germany.

American eugenicists were behind three major policy initiatives of the early 20th century.  For example, they maintained that the “race suicide” of Anglo-Americans could be prevented through laws prohibiting inter-racial marriage. Likewise, eugenicists campaigned for immigration restriction quotas to stem the tide of foreign genes into the U.S.  Perhaps their most controversial measure called for the compulsory sterilization of all those of unsound mind or body.  Eugenicists believed that sterilization protected society from the “menace of the feebleminded” through the practice of mental hygiene.  That the targets of mental hygiene tended to be the socially marginalized only served as further evidence of their “social inadequacy.”  Sterilization was thought to relieve schizophrenia and depression, and prevent the spread of mental illness by curtailing the birth of “eugenically undesirable children.” To quell criticisms of California’s sterilization laws, the HBF undertook an ambitious study of the results of California’s experimental program in order to document the “physiological and mental effects of sexual sterilization” on mental patients (Harry H. Laughlin Papers, D-2-3:24). These technical reports concluded that sterilization was therapeutic for mental patients and prophylactic for society.

The technical papers of the Human Betterment Foundation (HBF) marks the intersection of science and public policy and the fostering of national identity through the terministic screens of science, medicine, and public health. The HBF’s collaboration with the Department of Mental Health resulted in the sexual sterilization of 20,000 Californians deemed insane and feebleminded.  Less well-known is the HBF’s collaboration with Nazi race hygienists.  When the Berlin Imperial Minister of Justice sent out copies of its updated sterilization law to German officials in January of 1934, it appended along with it a translated copy of one of the HBF’s pamphlets in order to proved additional background information (E.S. Gosney/HBF Papers, Box 1.6 and 21.7). That HBF eugenic programs were initially paralleled by Nazi eugenic programs is more understandable when considered in light of the fact that Hitler’s national hygiene legislation, whose preliminary targets were also mental patients and the physically disabled, began “after careful study of the California experiment under Mr. Gosney and Dr. Popenoe” (E. S. Gosney/HBF Papers, box 5.15).

As new discoveries in genetics dispelled older theories of biological inheritance, eugenicists found themselves at impasse in the nature vs. nuture debate. This ideological crisis was further compounded by the revelation of Nazi race hygiene atrocities at the conclusion of the war.  Eugenicists viewed with rising alarm the erosion of their scientific ethos and Gosney and others realized that it was necessary to disassociate themselves from their former Nazi colleagues if they wished to regain their standing as technocratic experts for the public good.  Therefore, eugenicists switched ideological emphasis from genetics to psychobiology, navigating out of the terrain of strict hereditarian ideology and into the terrain of sociobiology and behavioral biology.  When Gosney passed away in 1942, he willed the proceeds of the HBF to Caltech’s developing behavioral biology program stipulating that the money be used to “research the biological basis of human qualities” (E.S. Gosney/HBF Papers box 4.2).

The HBF’s successor in Caltech’s behavioral biology department would continue to conduct research into the treatment of mental illness through psychosurgery.  In 1934, Caltech hired two neurophysiologists from the Netherlands to investigate the treatment of mental illness by the novel technique of passing an electrical current through the human brain.  Known as electroconvulsive therapy (ECT), this form of experimental psychosurgery could also result in spinal fractures, heart attacks, memory loss, and a vegetative state on a par with a lobotomy.   To redress such problems, the Caltech team pioneered another form of ECT in which the injection of a paralyzing agent prior to electroshock produced milder bodily convulsions.  At the right settings, this form of ECT was eventually refined to induce anesthesia or somnolence in the patient and was dubbed “electronarcosis.”

Like their cohorts at the HBF, Caltech scientists worked in conjunction with the Department of Mental Health  to conduct experimental research on California’s mental patients (Kay 98).  Such research would have political repercussions at another institution thousands of miles away when a group of Canadian patients sued the U.S. government for non-consensual human experimentation. Canada’s infamous “sleep room” experiments conducted at Quebec’s Allan Memorial Institute in the 50s and 60s, resulted in permanent brain damage to scores of patients (Collins 1). However, survivors were not to discover they were the victims of experimental electronarcosis until investigations into intelligence abuses during the fallout from Watergate revealed the existence of a top-secret CIA program called MKULTRA  (Collins  25 – 27).

Headed up by a Caltech chemist named Sidney Gottlieb, the CIA’s Technical Service Staff (TSS) was responsible for behavioral modification through the covert administration of chemical, biological, and radiological substances on unwitting human subjects (Marks 60).  A CIA cutout called the Society for the Investigation of Human Ecology (SIHE) funded electronarcosis experiments at the Allan which attempted to wipe clean the human psyche by “depatterning” patients in order to re-program them with alternative behaviors (Collins 130 – 135). Depatterning was followed up with pre-recorded taped messages played ceaselessly under the pillows of heavily narcotized patients for weeks (and sometimes even months) at a stretch in a technique called  “psychic driving.”   Ostensibly, the purpose of the tape-recorded messages was to  “reprogram” patients sans mental disorders by implanting hypnotic suggestion (Collins 128 – 133).  That such treatments left many of them incontinent, drooling human vegetables did not deter the doctors from performing what the CIA called “terminal experiments” (Marks 35).

During the Doctor’s Trial at Nuremberg, the revelation that Nazi scientists had practiced medical experiments responsible for the death and mutilation of concentration camp victims gave rise to the Nuremberg code.   The Code decreed that involuntary human experimentation was a violation of human rights. Burke points out that the rise of a “sinister science” in political regimes is often predicated on subsuming universal principles of scientific clarity and fairness to the exigencies of national security. The rhetoric of science and technology can thus take on a divisive and conspiratorial nature which can function in the service of obscure political agendas (A Rhetoric 35).

Nationalist rhetoric makes “use of language as a symbolic means of inducing cooperation in beings that by nature respond to symbols”  (A Rhetoric 43).  The HBF’s call for a surgical solution to mental abnormality in the American populace focused the problem squarely on the indigent, the infirm, women, immigrants, and other racial minorities, and demonstrates how technical papers can be used to foster a purified national identity.   Eugenicists employed a technocratic rhetoric designed to discern, divide, and delete the “socially unfit”,  people who presented the necessary corollary in a genetic cosmology comprised of the eugenic “us” versus the dysgenic “them.”

However, one of the problems with a genetic social order is that, in negating the ethical and dialectical dimensions of social relations, science reduces the dialectic sphere of human actions to the empirical sphere of human motions (A Rhetoric 186, A Grammar 61). The human sciences too often succeed in denying the relevance of dialectical and metaphysical issues lying at the heart of social relations by blaming  personal and sexual neuroses on the individual.  This serves to mystify social relations by using psycho-sexual terms to suggest that neuroses are the fault of the individual rather than the social order which engenders them.  Burke’s comparison of rhetoric and psychology ascribes the shortcomings of the latter to the fact that what it deems a sublimation of motives is, in fact, a dialectic of motives. Dialectical motives are “implicit in the nature of language” (A Rhetoric 279) since it is a symbol-system used to facilitate human interactions through mediation of  “the parliamentary jangle” (189).  The dialectical motive is predicated upon a meta-rhetorical reality drawing on universal standards of justice as well as personal principles of “self-interference” in persuasive appeal, which hold in check and guide human cooperation (280).  The psychological sublimation of motives, on the other hand, is predicated upon a materialistic reality grounded in idiosyncratic self-interests in a quest for perpetual personal advantage (276).

Burke contends that the human sciences fail to distinguish,

[d]ialectical factors at the very center of realism. Here, implicit in our attitudes toward things, is a principle of classification. And classification in this linguistic, or formal sense is all-inclusive, “prior” to classification in the exclusively social sense. The “invidious” aspects of class arise from the nature of man not as a “class animal,” but as a “classifying animal.”   (283)

Burke’s nuanced examination of the shortcomings of science as a human meaning system notes that it employs a reductive terminology that attempts to exclude the metonymic principle of poetic realism (A Grammar 507), a fact that distracts attention from science’s precondition in poetry. The absence of poetry in science is indicative not so much that poetry is a flawed meaning system so much as science must stand in relief to poetry in order to establish its very line of inquiry.  While the behaviorist views a blush as a series of biochemical motions precipitated by an environmental stimulus, the poet views the blush as a dialectic act prefiguring the spectrum of human communicative interactions (A Grammar  507).

The possibilities for transcendence are negated in a socio-scientific perspective that perceives the meaning of human behavior in the reductive terms of stimulus/response. Burke contends that such scientific reductionism cannot account for the cognitive/linguistic function of human action, for, in trying to understand complex behavior through analysis to its constituent parts, behavioral science only ends up confronting “the paradox of substance in a terminology unsuited to the illumination of this paradox” (A Grammar 60). This error is compounded by the fact that science tends to confuse scenic terms (instinct, drives, urges, brain) for agent terms (meditation, purpose, desires, mind). Scenic terms may “seem more ‘real’” than their dialectical counterparts, but they ultimately “serve as a rhetorical deflection of social criticism” (A Grammar 49).

The rhetoric of motives in socio-scientific technocracies suggests that the deflection of social criticism lies at the heart of eugenic social control campaigns. It relegates the social function of class to the empirical sphere of the rigid and bureaucratic rather than the dialectical sphere of the hierarchic and transcendent, and calcifies human social interactions by substituting poetic realism for scientific realism (A Rhetoric 186, A Grammar 506). Thus it successfully disguises social control for social improvement and creates a dyslogistic rhetoric that must find recourse in a social scapegoat.

Filed under 2008 Tagged with , , , , , , , , , , , , , , ,

The Issue of Error Correction in ESL Student Writing by Chi-Ping Chang

March 10, 2002 by Leave a Comment

After having several English writing classes in college, I realized that, except for the college writing structure, every writing instructor has a different standard and expectation of ESL students’ writing. Errors that were recognized by one teacher may not be recognized by an- other; the paper that receives positive comments from one teacher may be criticized by an- other. The confusion has resulted in frustration. Recalling my experience in writing classes provides the point of departure for this review of the literature. Despite of my personal negative attitude towards writing, I think it might be helpful to my academic writing to find out what researchers have found on error correction in ESL writing.

I first started with the website “Google,” and found the Journal of Second Language Writ- ing. I easily obtained a few articles about grammar correction and feedback in ESL writing. One of them titled “The Case for Grammar Correction in L2 Writing Classes: A response to Truscott (1996)” written by Ferris (1999) caught my attention the most. After skimming through it, I desperately wanted to read Truscott’s article on grammar correction. Therefore, I went back to the library and searched for articles related to the subject of “grammar cor- rection in ESL writing classes.” I first tried ERIC, one of the Article Databases in the li- brary, but the result was not satisfactory. Then I searched under LLBA (Linguistics and Language Behavior Abstracts), and I luckily obtained some relevant articles. Among other articles, I found Truscott’s paper about grammar correction in L2 writing published in 1996. To my surprise, Truscott (1999) later wrote an article responding to Ferris’ criticisms. It is very interesting to see two researchers defending themselves by providing numerous evi- dence to argue against each other. Besides ERIC and LLBA, I also searched under PsycInfo. This database also has a few articles related to this topic, but compared to LLBA, the results are not as relevant to my interest.

Individual review of each source

I. The Case Against Grammar Correction in L2 Writing Classes

This is a journal article published by Language Learning in 1996. The complete reference is as follows: Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language Learning, 46, 327-369.

Truscott (1996) argues that grammar correction in second-language writing classes should be abandoned because it is ineffective and harmful. A number of studies are cited to provide evidence for the ineffectiveness and unhelpfulness of grammar correction in ESL writing classes. He concludes that there is no reason to correct grammar errors.

Though Truscott managed to find numerous negative results in many studies to support his thesis and only few positive results stemming from the grammar correction. It needs to be noted that he did not conduct any actual experiment in ESL writing classes to support his argument in the paper, and he might have overlooked some other significant findings that contradict arguments against grammar correction. For those ESL teachers who find respond- ing to students’ errors in writing very tiring and time-consuming, or those who think that students’ motivation might be negatively affected, Truscott’s article might be beneficial.

II. The Case for Grammar Correction in L2 Writing Classes: A Response to Truscott (1996)

This is a journal article published by Journal of Second Language Writing in 1999. The complete reference is as follows: Ferris, D. R. (1999). The case for grammar correction in L2 writing classes: a response to Truscott (1996). Journal of Second Language Writing 8(1), 1-11.

After the scrutiny of Truscott’s evidence, Ferris (1999) claims that, “Truscott’s conclusion that grammar correction has no place in writing courses and should be abandoned is pre- mature and overly strong” (p. 2). She accuses Truscott for rejecting findings of previous studies that favor grammar correction, and claims that the studies Truscott used to support his conclusion do not address the present issue. She points out two problems in Truscott’s argument. One is the definition for the term “error correction,” and the other is his review of previous research findings on error correction in ESL writing classes. Examining the articles that Truscott cited to support his thesis, Ferris finds that, first, the subjects in those studies are very diverse and not comparable; second, the research design and instructional method vary across studies; third, Truscott overstates the negative findings, and disregards those that contradict his argument. Ferris also uses her own previous research findings and teach- ing experiences to confront Truscott’s argument and to support error correction in L2 writ- ing classes.

Although Ferris did not cite as many research findings as Truscott, she evaluated his argu- ment carefully and systematically, by pointing out not only the flaws in Truscott’s article but also her agreement with Truscott. Ferris encourages teachers to listen to students and consider their needs before deciding what and how to provide feedback on error correction to students. Her article is a great support to writing teachers who are for error correction in L2 writing.

page3image488

III. The Case for “The case Against Grammar Correction in L2 Writing Classes”: A Response to Ferris

This is a journal article published by Journal of Second Language Writing in 1999. The complete reference is as followings: Truscott, J. (1999). The case for “The case against grammar correction in L2 writing classes”: a response to Ferris. Journal of Second Lan- guage Writing, 8(2), 111-122.

Truscott responds to Ferris’ evaluation of his previous article published in 1996. He rejects Ferris’ criticisms on his case again grammar correction, and asserts that Ferris’ arguments are unfounded and biased. He responds to each of the issues from Ferris’ article with his own counter arguments, and compares the previous evidence and statements of his thesis with Ferris’ arguments. He claims that Ferris does not provide any published sources to support her position against his argument, nor does she find any adequate and sufficient evidence to show that his conclusion is inconsistent with other research findings he cites (since Ferris criticizes his conclusion as being “premature and overly strong.”). He says, “…only the most sympathetic of readers would consider this adequate support for the accu- sation” (p.114).

Truscott’s responses to Ferris article provide readers, especially ESL writing teachers, great opportunities to reevaluate the arguments that have been made by both Truscott and Ferris. Like Ferris, I made assumptions while reading Truscott’s first article on grammar correc- tion. For instance, I doubted the credibility and reliability of the research studies that Trus- cott used in his article in 1996. The reason was that some of the experiments seemed to be only carried on for a short period of time; therefore, the results did not appear to be convinc- ing to me. Ferris calls that “one-shot experimental treatment.” Yet, Truscott denies the statement, and claims that those studies were considerably longer than a semester. It leads us to think that to be an objective reader, one should not ignore views of either side.

IV. Grammar Correction in ESL students writing: How effective is it?

This is a journal article posted on Schuylkill website which is founded and edited by gradu- ate students of Temple University. The complete reference is as follows: Loewen, S. (1998). Grammar Correction in ESL students writing: How effective is it? Retrieved October 7, 2001, from Temple University, Schuylkill web site: http://www.temple.edu/gradmag/fall98/loewen.

Loewen conducts a research project to examine Truscott’s argument, and devises a research question: “Does correcting grammatical errors in students’ essay result in improved accu- racy in the use of those structures in subsequent revision as well as in the new essays?” (p. 2). He finds no significant effect for grammar correction, which confirms Truscott’s argu- ment. However, Loewen points out some limitations of his study, which need to be consid-ered before generating any conclusion. One is the small sample size of the study, and the other is the infrequency occurrence of the structures that were under investigation.

This study provides a more objective view on grammar correction in ESL writing, although the result shows no significant effect on error correction in this experiment. Further research on the present issue may be essential in providing more objective scientific evidence.

V. ESL Learners’ Performance in Error Correction in Writing: Some Implications for Teaching

This is a journal article published by Elsevier Science Ltd in 1997. The complete reference is as follows: Lee, I. (1997). ESL learners’ performance in error correction in writing: Some implications for teaching. System 25 (4), 465-477.

The purpose of this study is to examine three common assumptions about error correction in ESL writing classes—1. overt correction is helpful; 2. students can cope with error feed- back in the form of a correction code; 3. all errors deserve equal attention (p. 465). An er- ror correction task was designed in three different conditions (direct prompting, indirect prompting, and no prompting at all) to test the assumptions, and 149 subjects participated in this study. The results show that students’ major difficulty in error correction is recognizing the existence of errors. Some pedagogical implications arise from the study. Lee concludes that, first, teacher’s error feedback is very important to facilitate error location to help error detection; second, teachers must clearly explain to students the principals of grammatical terminology to prevent them from making errors; third, it is suggested that teachers decide their priorities in error correction, based on the needs and language proficiency of the learn- ers.

This article is beneficial to both students and teachers. It provides useful insights as to what methods of instructions teachers should use in order to empower students to become better editors and writers. At the time students reading the article can learn what are the essential skills they need in order to improve their writing abilities, and useful tools for self-error detection and correction.

Conclusion

Truscott’s review paper on grammar correction published in 1996 has led to a great deal of discussion. He argues that grammar correction is both ineffective and harmful and therefore it has no place in the writing classroom. However, Ferris rejects Truscott’s thesis after scru- tinizing his sources. It was found that some of her arguments against Truscott were not valid. For example, she criticized the lack of definition for the term “error correction, ” which was denied by Truscott in his response to Ferris’ article in 1999. Before making any judgment, I went back to Truscott’s article published in 1996, but failed to find the term “error correction” in his article. Nevertheless, Ferris made good points that, “…the viewer has under- or over-stated the findings and claims of the original studies to suit his or her own generalizations or arguments” (p.4)

Loewen and Lee both reviewed Truscott’s paper on grammar correction. Instead of debating over the issue of grammar correction, they conducted experiments individually. Loewen focused on how much grammar correction can improve students’ accuracy in writing, and the study result showed no significant improvement. Loewen avoids making a conclusion on whether grammar correction should be abandoned or not. Instead he makes another hy- pothesis for further research. That is,

…analytic learners might benefit from grammar correction because it would be consistent with their method of language learning. Holistic learners, on the other hand, might not benefit from such a method (p.7).

On the other hand, Lee focused on students’ performance in error correction in written texts, and examined some common assumptions about teachers’ error correction. Based on the study findings, Lee generated some pedagogical implications by rephrasing the three as- sumptions mentioned above: 1. error feedback may be more desirable than overt correc- tion; 2. error feedback by means of a correction code must be handled with care; and 3. some errors may deserve more attention than others (p. 471).

There are some statements from Ferris’ article that I, as an ESL student, totally agree with. I think every ESL writing teacher should be aware of the fact that: “…teachers are inconsis- tent in their ability and willingness to recognize and correct errors and provide adequate grammar explanations to their students.” She goes on and states, “…often students don’t understand grammar feedback or are unmotivated to deal with it.”

There is contrary evidence as to whether or not grammar correction is beneficial to students in ESL writing classes. Due to the lack of conclusive evidence for each case, it should be left for the readers to decide whether error correction in teaching ESL writing should be abandoned or not.

Filed under 2002 Tagged with , , , , , , , , , , ,

Contexts and Criteria for Evaluating Student Writing by Jane Hindman

February 10, 2001 by Leave a Comment

Of all responsibilities you have as a composition instructor, evaluating student writing occupies most of your time and has furthest reaching material effects. Though you may spend lots of hours preparing for class, conferencing with your students, and actually teaching, chances are you’ll spend many more grading. Though we instructors often place the highest value on the content and methods of our classrooms–be they critical pedagogy and Marxist interpretations of Clinton’s impeachment trials or traditional grammar drills and a New Critical reading of Paradise Lost, thegrades that we assign our students are the only concrete, as well as the most valuable, cultural capital that our teaching creates. As Evan Watkins says, in his analysis of what transformative effect our teaching actually has in our culture, you don’t report to the registrar that [your student learned]. . .a revolutionary fusion of contradictory ethical claims. . . .You report that 60239 got a 3.8 in Engl 322, which in turn, in a couple of years, is then circulated to the personnel office at [for instance] Boeing as 60239′s prospective employer. (18)

As a general rule though, new instructors spend much less time training to be graders than training to be facilitators in the classroom. In fact, you may be wondering why you need to learn how to grade at all, for you may think you know how already. In the teacher training courses I’ve taught, most fledgling teachers have initially imagined that grading is a skill they already have, that–as former English majors and/or good writers themselves–they can “naturally” evaluate essays. After all, they reason, they’ve received enough comments on their own papers, right? They know how the process goes; and besides, good writing is obvious: we all know it when we see it, so it should be pretty easy to figure out how to evaluate it.

Would that it were that simple. In actuality, and as a visit to any norming session[2] comprised of instructors from across campus will demonstrate, few university faculty agree on what good writing looks like. In fact, it’s highly unusual when faculty from other departments do agree with the criteria for good writing that we in composition espouse. Even within our own departments (and within your group of new writing instructors perhaps), commanding debates flourish about issues like which factors should have the highest priority in determining a grade–grammar or content; about whether or not a five paragraph essay signals proficiency or a lack of critical thinking skills that necessitates developmental writing work; about how many aural/oral confusions should be “allowed” in a passing essay; about how much “credit” a student should get for taking risks in her argument. These sometimes heated discussions are the rule rather than the exception.

Why do people disagree so much about what constitutes good writing? And considering that no department yet has been able to find the definitive resolution to these debates, what are you going to do to be a consistent and fair evaluator, especially if students try to argue with you over grades? (And believe me, they will.) How can you feel confident that your grades and/or your guidance to students about how to improve the quality of their writing are not “just” subjective interpretations? If your supervisor(s) and/or department chair review your grading practices, how can you be sure that your evaluations will be sanctioned, that you are fulfilling the goals of the writing program, the department, and the institution that employs you?

My advice is to integrate the following tenet into your composition theory and practice: “good” is a rhetorical term whose application and definition depends on its context. In other words, evaluations of writing are always relative because they’re contextually determined. As a matter of fact, the power of any specific use of language depends on its context, regardless of whether the “power” is judged to be sublime(to use literary terms), “felicitous” (to borrow from speech act theory), appropriate, pornographic, persuasive, humorous, disgusting, satisfactory, bland, or “awesome, Dude.”

What does this tenet mean with respect to your efforts to learn how to evaluate student writing? The perhaps bad news is that you’ll have to disabuse yourself of the notion that there are universal standards for good writing, that if we just look hard enough and argue long enough, we’ll uncover those standards once and for all. The necessity to let go of that notion may seem commonplace to you, especially if you’re a proponent of post-modern theory and/or Foucault’s discussions of the order of discourse. On the other hand, you may be someone who thinks that not supporting the belief in inherent qualities of good writing is virtual heresy. Regardless of your predisposition, in practice understanding and internalizing the context-dependent nature of writing evaluation can be difficult.

Imagine, for instance, that the person sitting beside you in a holistic grading exam session believes that spelling errors are the mark of illiteracy and so wants to give the lowest score possible to the very same essay that you found outstanding because of its well-developed discussion of the sexist implications in the weapons imagery of Die Hard 2. It often seems “obvious” to us writing instructors that idea development is more important than spelling, that organization supercedes mechanics as a criteria for quality writing. But, like the definition and arrangement of all criteria for evaluation, the privileging of those characteristics depends on their context.

Lest you think we’re dangerously close to the slippery slope of solipsism, let me reassure you that there is some good news: mediating the context(s) within which you evaluate will ensure that your practices are fair, consistent, and authorized. In other words, if you understand and internalize the purpose(s) of each specific evaluation process you participate in, as well as the criteria developed for judging that specific writing task, then you will have sufficiently evaluated the context. As a result, your application(s) of “good” (or “mediocre” or “excellent” and so on) will be not merely haphazard nor “subjective” (to you alone); rather, your scores/grades will be systematic, consistent (with the purpose and criteria of that context), systemic (i.e. relative to the system within which you’re evaluating).

To return to our example of the “spelling = illiteracy” person sitting beside you in the norming session. If the leader of the session has provided graders with a rubric–a description of the critieria–for evaluation, then you can refer to that description to adjudge the disagreement between you about which to privilege in the Die Hard 2 essay–the development of the claims about sexism or the spelling of the essay. Chances are that the discussion that ensues will expand not just in topic (from spelling to “grammar” and from idea development to “content” and “thinking”) but also in numbers of speakers. Such discussions are an integral aspect of the process by which grading session participants come to agreement about the purposes and criteria specific to each grading context that confronts them. If the leader of an evaluation session does not provide a rubric, then the debate about spelling and development provides you and the other graders with the opportunity to decide among yourselves what criteria you should consider when you read the papers that you’re charged with scoring.

On the other hand, if and when you yourself are the person with sole responsibility for grading students’ writing (as you probably will be when you teach your own section(s) of composition), then your fairness and consistency depend in large part on your careful determination of your purpose(s) and criteria for evaluating. Many writing programs assist instructors by prescribing general purposes for writing (and therefore for the evaluation of) individual assignments and criteria for grading. These clarifications maintain consistency across different sections of the same course and supply individual teachers with the written description of the “what you want in this paper” that students often ask for. In addition, programmatic criteria for grading offer new and experienced instructors with the materials they need to best understand the institutional goals that inform their specific classroom contexts. In the context of individual assignments (or sometimes in lieu of any stated programmatic goals for composition courses), many instructors negotiate with their students the criteria for grading essays. Such a process makes explicit for students and for the instructor what expectations and standards will be adjudicating their evaluations.

But enough talk. It’s in the actual doing anyway that these complicated, contextually-dependent meanings become more clear. So not to worry if what I’ve said so far seems abstract and/or confusing. The exercises that follow are intended to illustrate in practice what I’ve just theorized. If you have a leader(s) directing your training, she should be able to supply the sample students papers you need and oversee the group discussions you have. But even if you are not a member of an organized teacher-training program, if you and at least two or three other new teachers can find some sample student essays and complete these activities, you will develop a good understanding of the following:

· Assumptions about good writing you and others currently have
· Instances of specific writing practices that demonstrate your (and others’ ) assumptions
· Revisions to your assumptions you want to make
· Assumptions about good writing explicitly or implicitly required in your institutional context(s)
· Descriptions of the criteria for good writing that facilitate students’ understanding
· Variety of purposes for evaluating student writing
· Variety of methods of writing evaluation and the purposes they best serve
· Variety of contexts within which composition instructors evaluate student writing.
Good luck, and happy grading.

ACTIVITIES

Part One – Taking placement exams, Defining criteria

Individual work

1. Diagnostic Writing — 30 minutes– Write a response to the prompt you are given. (Trainers or Groups—see Appendix I if your program can’t supply a sample essay placement exam prompt.) Be sure to save your essay as you will refer to it again after several other activities.

2. Metacognitive Writing about Diagnostic Writing — 30 minutes. Write about the process of writing the in-class diagnostic essay. What strategies did you use to be successful on the exam? For instance, what aspects of the writing process (invention/free-writing, planning, organizing, drafting, revising, proof-reading) did you most attend to? Which did you ignore? Did you make conscious decisions about how to divide your time? If so, on what basis did you make those decisions? Did you maintain those decisions or change your mind later? If you didn’t consciously make such decisions, why not? What choices would you make again during an in-class writing situation? Which choices would you change?

3. Defining Good Writing. Make a list or write a description of what you think constitutes good writing. What are the characteristics of mediocre writing? Of definitely “bad” writing? Now write about how you have formulated these opinions about writing. Whose attitudes have you adopted and whose are you rejecting? In what circumstances would you change your mind about what constitutes good writing? What characteristics of good writing are immutable?

Group work

1. First, share your criteria for good writing with each other. On what points (if any) do you agree? On which do you most forcefully disagree? Decide among group members which criteria you want to represent the group’s consensus. If applicable, repeat this process within the large group. When all groups have reached an agreement, make an official list of that criteria.

2. Now, evaluate this criteria’s usefulness:

· What type(s) of writing will it best measure? (For instance, what writing tasks does it best assess: proficiency writing exams that receive a holistic pass/fail grade, placement exams that determine the appropriate level of composition instruction for individual students, formal research papers that receive a letter grade, rough drafts that will be revised later, journal writing that demonstrates students’ engagement with their reading assignments, in-class timed writing essays, take-home essay tests? Is it intended to assess developmental writing rather than advanced composition and/or any other level(s) of writing? Should it be?)
· How specifically does it articulate the characteristics of good writing? Will students be able to understand the terminology it uses? For instance, does it rely on a COIK (clear only if known) explanation of development or organization?
· Is each level of quality uniquely defined? Are highest, middle, and lowest levels demarcated with specific descriptions? Are the middle and lower categories explained in their own right or are they defined only in opposition to the highest category?
· What other aspects of the criteria need to be considered?

3. Discuss what you have learned about the most effect methods for constructing and evaluating criteria for grading.

Part Two — Holistically Scoring Placement Exams

Individual work

1. Holistically score student placement exams

· Read several examples of essays that incoming first year students wrote—preferably in response to the same prompt that you wrote to.
· Read “A Rubric for Freshman Placement Essay Evaluations.”
· Re-read and assign a score to 6-8 different student essays. (Trainers or Groups—see Appendix II for how to choose these essays.) Make very brief notes to remind you of what characteristics of each essay evoked the score you assign so you can discuss your choices with the group.
2. Compare and contrast your group’s collaborative criteria for “good” with “A Rubric for Freshman Placement Essay Evaluations.” In what ways are your group’s views ignored or undermined? What specific purpose is this evaluation of placement exams and its rubric meant to serve? In what contexts might the exam and the criterion given in this rubric not apply? In what ways are the rubric’s descriptions vague or fuzzy? What difficulties might a new English instructor have with internalizing the criteria assigned by the rubric? Would your group’s criteria improve the instructor’s ability to internalize or not?

Group work

1. Referring to the 6-8 essays that you scored on your own, stage a norming session in which you discuss your scores with each other. (Refer to Appendix II if you don’t have a supervisor who can conduct the session.) Keep track of how much fluctuation you see between your scores and others’.

2. After the norming session, discuss these issues: Which readers are usually high or low? What seems to be the explanation for that tendency? Which persuasive points during the norming session most convince you about another evaluator’s point of view? What points of your own seem the most persuasive? What most annoys you about other people’s view of writing? Why do you suppose that particular thing annoys you?

3. In a large group, grade 10-20 more essays after the norming discussion. How consistent were you as a group? How “normalized” were you as an individual?

4. Discuss as a group what the session has taught you about evaluating placement exams.

5. Optional

· Anonymously grade two of your peers’ essays (also anonymous) that they wrote to the prompt. Ask the Trainer or your fellow students to return all essays to original owners. Review your essay and reflect on the one(s) you evaluted. Did you and/or your colleagues perform as well as you expected? What (if anything) does your performance and/or theirs tell you about the effectiveness of timed-writing assignments in assessing writing proficiency?
· Review together the metacognitive writing you each did about your process of writing to the prompt. What clues does that writing provide about the strategies most effective in timed-writing settings? How do those “clues” translate into strategies you’ll teach your students?

Part Three — Responding in Writing to Student Essays

Individual work

1. Read about various ways to evaluate student writing. For instance, look at the chapter entitled “Responding to Student Writing” in Erika Lindemann’s A Rhetoric for Writing Teachers.

2. Write about what you see as the differences between holistic grading and responding in writing to student essays. What different strategies must an instructor employ in each context? How would a wise writer’s strategies change according to the method by which she’ll be evaluated?

3. Letter-grade students’ final drafts for a first year composition class.

· Read 3-5 student essays provided.
· Assign a letter-grade and comment on each as if you were responding to the student who wrote the essay.
· Be prepared to share your comments in class.

Group Work

1. Individually record in writing the criteria you used for assigning a letter grade to the student essays you read. Then discuss the criteria you each used. Is there any agreement? Where? What criteria are most contested among you?

2. Compare your grades on the 3-5 essays you evaluated.

· Consider the agreement/disagreement in assessments and then compare that disparity with the agreement/disagreement on criteria. What if anything is significant about the relationship between the two?
· Now review the written comments that different instructors (to be) make. What different purposes inform their comments? For instance, are instructors writing to improve the student’s next essay, to justify the grade they assign, to motivate the student to keep trying, to engage the student in further thinking about the ideas discussed in the essay, and so on? Which of these reasons for commenting seem most useful to you? Why?
3. If your Trainer and/or the Writing Program offers criteria for evaluating student essays, refer now to that. If not, decide among yourselves and record what criteria you’ll be using to comment on the student essays written for the first formal assignment you’ll be grading.

4. Using the criteria you’ve been given or that your group has developed, grade and respond to 3 more samples of student essays written in a context similar to that of the first formal assignment you’ll actually be grading. Compare your grades and comments this time around to the ones you made before you discussed the criteria. Have the grades and/or the flavor of the comments changed? How? What purposes do the changes serve? Do they make grading more effective and/or more consistent? In what ways?

Part Four — Evaluating Rubrics

Individual Work
1. In addition to the one supplied by the writing program you’ll be teaching in or the one created by your group, collect at least two rubrics for grading first year composition essays. If your writing program cannot supply you with these additional examples, you could check web sites or books about writing assessment. (Some large writing programs make public their criteria for evaluations of student’s compositions; the University of Arizona, for instance, has published many editions of A Student’s Guide to First-year Composition which usually includes a rubric for evaluating students’ essays.)

2. Write about the differing ways that these rubrics operate. Do the criteria contribute to your ease of grading and/or to the students’ understanding of what and how they should write? In what ways are the rubrics’ descriptions vague or fuzzy? Which rubric most undermines and/or supports your views of what constitutes good writing? Which of the rubrics you’ve encountered would best accommodate a new English instructor’s efforts to internalize the criteria assigned by the rubric? Why? Which system which you most like to be graded under? Why?

Group Work

1. Discuss each of your individual appraisals of these various rubrics. (Refer again to the list for evaluating criteria in Part 1.) Compare them to the one you’ll actually be using when you grade students’ essays. Is yours explained as clearly as it could be? If not—and if it is a version required by your program—how will you explain or supplement it so that your students will best understand how to shape their writing? What views of writing inform the rubric you will use? Which specific features of the rubric(s) reveal its perspective on writing and the writing process?

Part Five — Writing Responses to Drafts, Evaluating Response Methods

Individual Work

1. Read Ed White’s article “Post-structural Literary Criticism and the Response to Student Writing” and Peter Elbow’s article “Ranking, Evaluating, Liking.”

2. Respond in writing to these sets of questions:

· White claims that ETS developed holistic scoring as a way to produce consistent test scores and thus to improve the unfairness inherent in previous grading situations. Do you agree with White’s belief that the holistic scoring method improves fairness and promotes a sense of community among English teachers? Why or why not?
· Summarize (or cite) one point Elbow makes that you support wholeheartedly and explain why you agree with his view OR do the same about a point that you disagree strongly with. Describe how you might successfully apply one of Elbow’s ideas or suggestions to a classroom OR describe what disaster (or mere problem) you think would probably result from your using another of Elbow’s suggestions.
· White claims that “[t]he simple fact is that the definition of textuality and the reader’s role in developing the meaning of a text that we find in recent literary theory happens to describe with uncanny accuracy our experience of responding with professional care to the writing our students produce for us.” Elbow advises us to learn to “see potential goodness underneath badness,” to “read closely and carefully enough to show the student little bits of proto-organization or sort of clarity in what they’ve already written.” Analyze and explain how this suggestion supports or contradicts (or both) Ed White’s viewpoints about the ways we read student writing.
3. Review your comments on the papers you read for Part Three–Responding in Writing to Student Essays. Imagine now that the student essays are rough drafts that you will return and from which students will develop their final, graded versions. Considering that new context and what you’ve read in Elbow’s and White’s essays, re-examine and re-new your earlier comments on those essays. How and why have you changed (or not) those earlier comments? How does responding to a student’s draft differ from responding to a final version?

Group Work

1. Discuss your responses to the two essays, in particular your perspectives on the similarities (or lack thereof) in White’s notions of teachers’ “developing the meaning of a text” and Elbow’s notions of “proto-organization or sort of clarity.”

2. Discuss the ways that each of you in the group changed your comments when you were responding to a draft rather than to final version of a student’s paper. Draft a list of the differences in strategies you find most useful for responding to rough drafts and for responding to final versions of student essays. If applicable, share your list with the large group and then revise a large list that reflects all groups’ perspectives.

Part Six – Synthesizing Possibilities

Individual Work

1. Read Horvath’s article “The Components of Written Response.” Write in response to the following:

· Describe at least two ways that reading Horvath’s article motivates you to revise (or shape for the first time) your beliefs about responding to student writing.
· List at least four different purposes for evaluating student writing and four different methods of evaluating. Now write about which methods work best in conjunction which purposes. Be sure to explain your reasons.

Group Work

Discuss and come to some consensus about interfacing methods for responding to student writing with the purposes of evaluating individual assignments. Also discuss these other important issues related to evaluating student writing:

· What other aspects (besides criteria and purpose for assessment) of the context for evaluating student writing are salient? How do those other aspects affect the criteria for grading and/or the purpose(s) for evaluating an assignment?
· What strategies can an individual instructor use to align a pre-determined and prescribed criteria with her purpose(s) for evaluation in a specific writing context? How can she align a pre-determined purpose for evaluating student writing with her own (or with a group’s negotiated) critieria for the assignment? How can she teach students to strategize in these same ways? Should she teach them such strategies?

WRITING ASSIGNMENT– Evaluating Student Writing
The purpose of this paper is to facilitate your synthesis and critique of the various methods you’ve considered for assessing and evaluating student writing. In some way or another, you should demonstrate that you’ve read, analyzed, and thought about the materials. You might use this paper to formulate and defend your philosophy about grading papers or to analyze the ramifications of using a particular system. Whatever the claim you want to make, the argument of your paper should be based on your response to the different modes of assessment and evaluation of student writing that you’ve examined.

What you’re being asked to do is construct a context, a writing assignment and purpose, as well as a method for evaluating that written product. Explain how the method you chose to evaluate the writing is the most effective for the context, task, and goal that you’ve constructed.

OR

Your essay could be a response to at least two different modes of assessment and evaluation of student writing. You could compare the two of the grading systems you’ve practiced or compare the benefits and drawbacks of holistic grading with those of other ways to respond to student writing. You could also compare two or more of the published writers’ viewpoints on assessing student writing.

Regardless of your choice for approach, you’ll need to do more than simply summarize the method(s) or view(s) of evaluation. Take a position about a preference for a particular method of evaluation. Based on what you’ve read and experienced during this unit and from your other experiences as a writer and student (and teacher), what method of evaluation do you promote? What makes that method preferable? In what context does is the particular method effective and why?

Suggested criteria for evaluation:

Content: Does your essay demonstrate that you have read and used the materials that you’ve discussed as a class? Are you contributing additional insight and reflection to the body of knowledge that you’ve built in group activities? Do you rely on overgeneralizations or personal declarations (e.g. “Students learn better if they get feedback.”) to support your points or do you use specific examples from the texts or from other research as support? Do you use enough examples from the student essays and/or the other texts you’ve read to support your argument?

Organization: Are the details (examples) of your essay arranged in the order that will most convince the reader that your claim is true and sustain her interest? Are like ideas chunked together? Have you logically connected one idea to the next AND explicitly signaled just what those logical connections are? Have you given enough signposts such that the reader can easily see the “map” of your essay?

Expression: Is the language of the essay easily accessible to the readers, concrete, appropriate to your purpose? Do you avoid un-necessary formality, mixed metaphors, stilted sentence structures and phrasing?

Mechanics: Are grammar problems infrequent and minor enough that they don’t impede the reader’s understanding of your text?

Appendix I: Prompts for 30 minute, timed-writing essays

These samples are similar to those often used to place students in writing programs or to assess their writing skills.

1. Choose a specific event or situation from your elementary school years. It might involve school, home, or some other aspect of your life that you remember. It might be a single moment or crisis or an event that happened over time. The event or situation should be one that was very important to you at the time. Discuss it, and then put it in perspective through mature reflection.

2. Certain things are not taught in the classroom, such as how to get along with others, how to rely on yourself, or how to manage money. Describe something you learned outside of school and how you learned it, and discuss its importance in your life.

3. “Don’t ever slam the door, you might want to go back.”

This quote considers the issue of “burning one’s bridges.” Have you ever left a situation unpleasantly and then later wished you had handled things differently? Discuss the result and explain how it affected you later.

OR

If, on the other hand, you have managed to keep all your “doors open,” discuss how you accomplished this and explain how it has affected your life.

4. Begin your essay with the following sentence (copy it into your essay):

The women’s rights movement has made great strides toward the goal of equal treatment for men and women.

Select one of the following sentences as the second sentence of your essay (copy the sentence of your choice into your essay immediately following the first sentence):

a) But we still have important work to do before our society can be considered non-sexist.

b) In fact, we must be careful not to infringe up on the rights of men in our attempt to compensate women.

c) Unless we make changes in language, however, our culture will remain biased in favor of men.

Complete your essay.

Appendix II: Norming Session
If you’re new to norming, the best results will come if you can get someone experienced to run the norming session for you. If your particular program doesn’t give placement exams or upper division writing exams and thus doesn’t have people practiced in running norming sessions, perhaps someone is the testing office at your university is familiar with holistic grading and norming sessions. If , however, you have no choice but to run the session yourself, try these procedures.

To prepare for the session:

1. Choose at least seven or eight student essays as samples. The samples should have been written to the same prompt or for the same assignment that your graders will be evaluating. (The best option is if the sample essay and the essays to be graded are written to the same prompt that the graders and you wrote to in the first activity “Taking Placement Exams” above.) The number of sample student essays you will need varies, depending on the system for grading that you’re using. You’ll need at least one representative essay for the highest, the lowest, and the middle scores of the rubric you’ll be using during your grading session. For instance, if you have a 4 point system, choose an essay that is without a doubt (or as close to that as you can get to certainty) a “1”, another that’s a clear “4,” and then a “3” or a “2.” (One of the ways to be sure about scoring is to get experienced graders to help you decide which essay is an “absolute” 4 and so on.) You also need to have at least one essay that stirs up controversy, an essay that evokes a wide range of scores from different readers. If you can find what’s called a “1/4 split” (meaning that from two different readers the same essay received a score of “4” from one reader and a “1” from the other), then you’ve got a great example of a “controversial” essay; a “3/1” split is the next best bet. And finally, you need a couple of essays that mark the middle range of your rubric; these I’ll refer to later as your “neutral” essays.

2. Make copies of all essays for all your graders. (They’re assigned to read these in Part 2, Individual Work, #1.)

3. Make copies of the prompt or the writing assignment for all your graders.

4. Make copies of the criteria for evaluation for all your graders.

To conduct the session:

1. Ask all graders to read the criteria for grading and the prompt or writing assignment carefully.

2. Ask graders to re-read the “high,” “middle” and “low” essays. Don’t tell them which one is which. Don’t even announce that you’re presenting a range of essays. Just ask them to read and score “these three essays.”

3. When they’ve finished, decide which of the three essays you’ll discuss first (probably the “high” one) and ask each person to announce her score to the group. Don’t discuss these scores yet; just record them on the board or make a note to yourself and ask the graders to do the same.

4. Ask the most experienced grader (and/or the person(s) whose score coincided with the one you intended to represent) to explain her reasons for assigning that particular score. Require the grader to use specific aspects of the student text and of the rubric to support her reasons for assigning a particular score.

5. Ask for discussion among the group members about their various scores. If the person whose score is most “off” the one you intended to represent is willing, ask her to explain her score. At this point, all group members can and should discuss their individual explanations for their scores.Require the graders to use specific aspects of the student texts and of the rubric to explain their reasons for assigning a particular score.

6. You—especially if you are or seem to be an authority figure to the other group members—would probably do best not to offer an opinion about which score is “right.” If, however, group members’ conversation gets overly heated or their debates cannot be resolved, you can mediate their discussion by calling on the people with the most experience and/or whose scores seem most reasonable to you. Don’t let individuals over-generalize about writing or criteria; insist that they refer to the specific rubric for this context and to some specific features of the student text(s) they’re discussing. If none of these plans work to mediate debates or if you don’t know for sure just who the experienced people are, then simply move on to another essay. The point of this session is for group members to norm themselves, not for you to get them to conform to what you or one other group member thinks.

7. Repeat this process (#3-6) with the “lowest” essay and then with the “middle” one. If necessary, offer other essays for the group to read which you think are examples of the score(s) about which the group members seem to have the most trouble agreeing.

8. Repeat this reading/discussing process with one of the “neutral” essays. If a relative consensus is reached (say, for instance, only two or three of ten people continue to disagree with a score and their disagreement is a number away), then ask the group to read and score the sample essay(s) that you chose as examples of “split” scores. Again, don’t tell graders why you’re giving them this particular example, just ask them to read and score it. Again, begin the discussion by asking for scores from all graders and then for comments by graders you deem reliable. Permit the group discuss their variances as they may.

9. When graders and you feel satisfied that you have discussed their individual points of view, give them the last “neutral” essay(s) to read and score. If, after sharing their scores, you have no “splits” in the scores, you’re now “normalized” i.e. you’re ready to begin an actual grading process and should have relatively reliable consistency among the scores graders assign. If, however, you still have drastic splits (the widest possible range of scores are assigned to the same essay), then my directions have probably been pretty worthless and you’re probably going to need the assistance of a trained professional. Sorry.

FOOTNOTES

[1] Jane Hindman is a professor in the Rhetoric and Writing Studies Department at San Diego State University. Her email is: jhindman@mail.sdsu.

[2] A norming session is a preparatory portion of a group’s process of evaluating writing . In the session, graders review the criteria for scoring, often called a rubric; they then read samples of the type of writing they’ll be evaluating, individually assign a score to each sample, and finally discuss their individual scores with each other. The purpose of the norming session is to calibrate scores among the group members so that as much consensus as possible results when these group members individually evaluate the essays they’ll be scoring. These evaluators and their trainers (if they are leading the norming session) use the group discussion about the sample scores to inform members about and persuade each other of specific interpretations of the rubric and of the characteristics of particular writing samples.

Works Cited

Elbow, Peter. “Ranking, Evaluating, Liking.” College English 55 (February 1993): 187-206.

Horvath, Brook K. “The Components of Written Response: A Practical Synthesis of Current Views.”
Reprinted in The Writing Teacher’s Sourcebook, 2nd Ed. Article originally appeared in
Rhetoric Review 2 (January 1984): 136–56.

Lindemann, Erika. A Rhetoric for Writing Teachers, 3rd Ed. New York, NY: Oxford UP, 1995.

White, Edward M. “Post-Structural Literary Criticism and the Response to Student Writing.”

The Writing Teacher’s Sourcebook, 2nd Ed. New York: Oxford UP, 1988. 285-293.
Article originally appeared in CCC 35 (May 1984): 186-95.

Watkins, Evan. Work Time: English Departments and the Circulation of Cultural Value. Stanford,
CA: Stanford UP, 1989.

Filed under 2001 Tagged with , , , , , , ,