College Composition Weekly: Summaries of research for college writing professionals

Read, Comment On, and Share News of the Latest from the Rhetoric and Composition Journals

Leave a comment

Ray et al. Rethinking Student Evaluations of Teaching. Comp Studies Spring 2018. Posted 08/25/2018.

Ray, Brian, Jacob Babb, and Courtney Adams Wooten. “Rethinking SETs: Retuning Student Evaluations of Teaching for Student Agency.” Composition Studies 46.1 (2018): 34-56. Web. 10 Aug. 2018.

Brian Ray, Jacob Babb, and Courtney Adams Wooten report a study of Student Evaluations of Teaching (SETs) across a range of institutions. The researchers collected 55 different forms, 45 of which were institutions’ generic forms, while 10 were designed specifically for writing classes. They coded 1,108 different questions from these forms in order to determine what kinds of questions were being asked (35).

The authors write that although SETs and their use, especially in personnel decisions, is of concern in rhetoric and composition, very little scholarship in the field has addressed the issue (34-35). They summarize a history of student evaluations as tools for assessment of teachers, beginning with materials from the 1920s. Early SETs focused heavily on features of personality such as “wit,” “tact,” and “popularity” (38), as well as physical appearance (39). This focus on “subjective” characteristics of teachers asked students to judge “factors that neither they nor the instructor had sole control over and that they could do little to affect” (38).

This emphasis persisted throughout twentieth century. A scholar named Herbert Marsh conducted “numerous studies” in the 1970s and 1980s and eventually created the Student Evaluation of Education Quality form (SEEQ) in 1987 (35). This instrument asked students about nine features:

[L]earning, enthusiasm, organization and clarity, group interaction, individual rapport, breadth of coverage, tests and grading, assignments, and difficulty (39)

The authors contend that these nine factors substantively guide the SETs they studied (35), and they claim that, in fact, in important ways, “current SET forms differ little from those seen in the 1920s” (40).

Some of composition’s “only published conversations about SETs” revolved around workshops conducted by the Conference on College Composition and Communication (CCCC) from 1956 through 1962 (39). The authors report that instructors participating in these discussions saw the forms as most appropriate for “formative” purposes; very few institutions used them in personnel matters (39).

Data from studies of SETs in other fields reveal some of the problems that can result from common versions of these measures (37). The authors state that studies over the last ten years have not been able to link high teacher ratings on SETs with improved student learning or performance (40). Studies point out that many of the most common categories, like “clarity and fairness,” remain subjective, and that students consistently rank instructors on personality rather than on more valid measures of effectiveness (41).

Such research documents bias related to gender and ethnicity, with female African-American teachers rated lowest on one study asking students to assess “a hypothetical curriculum vitae according to teaching qualifications and expertise” (42). Male instructors are more commonly praised for their “ability to innovate and stimulate critical thought”; women are downgraded for failing to be “compassionate and polite” (42). Studies showed that elements like class size and workload affected results (42). Physical attractiveness continues to influence student opinion, as does the presence of “any kind of reward,” like lenient grading or even supplying candy (43).

The authors emphasize their finding that a large percentage of the questions they examined asked students about either some aspect of the teacher’s behavior (e.g., “approachability,” “open-mindedness” [42]) or what the teacher did (“stimulated my critical thinking” [45]). The teacher was the subject of nearly half of the questions (45). The authors argue that “this pattern of hyper-attention” (44) to the teacher casts the teacher as “solely responsible” for the success or failure of the course (43). As a result, in the authors’ view, students receive a distorted view of agency in a learning situation. In particular, they are discouraged from seeing themselves as having an active role in their own learning (35).

The authors contend that assigning so much agency to a single individual runs counter to “posthumanist” views of how agency operates in complex social and institutional settings (36). In this view, many factors, including not only all participants and their histories and interests but also the environment and even the objects in the space, play a part in what happens in a classroom (36). When SET questions fail to address this complexity, the authors posit, issues of validity arise when students are asked to pass judgment on subjective and ambiguously defined qualities as well as on factors beyond the control of any participant (40). Students encouraged to focus on instructor agency may also misjudge teaching that opts for modern “de-center[ed]” teaching methods rather than the lecture-based instruction they expect (44).

Ray et al. note that some programs ask students about their own level of interest and willingness to participate in class activities and advocate increased use of such questions (45). But they particularly advocate replacing the emphasis on teacher agency with questions that encourage students to assess their own contributions to their learning experience as well as to examine the class experience as a whole and to recognize the “relational” aspects of a learning environment (46). For example:

Instead of asking whether instructors stimulated critical thought, it seems more reasonable to ask if students engaged in critical thinking, regardless of who or what facilitated engagement. (46; emphasis original)

Ray et al. conclude that questions that isolate instructors’ contributions should lean toward those that can be objectively defined and rated, such as punctuality and responding to emails in a set time frame (46).

The authors envision improved SETs, like those of some programs, that are based on a program’s stated outcomes and that ask students about the concepts and abilities they have developed through their coursework (48). They suggest that programs in institutions that use “generic” evaluations for broader analysis or that do not allow individual departments to eliminate the official form should develop their own parallel forms in order to gather the kind of information that enables more effective assessment of classroom activity (48-49).

A major goal, in the authors’ view, should be questions that “encourage students to identify the interconnected aspects of classroom agency through reflection on their own learning” (49).


1 Comment

Boyle, Casey. Rhetoric and/as Posthuman Practice. CE, July 2016. Posted 08/06/2016.

Boyle, Casey. “Writing and Rhetoric and/as Posthuman Practice.” College English 78.6 (2016): 532-54. Print.

Casey Boyle examines the Framework for Success in Postsecondary Writing, issued by the Council of Writing Program Administrators, the National Council of Teachers of English, and the National Writing Project, in light of its recommendation that writing instruction encourage the development of “habits of mind” that result in enhanced learning.

Boyle focuses especially on the Framework‘s attention to “metacognition,” which he finds to be largely related to “reflection” (533). In Boyle’s view, when writing studies locates reflection at the center of writing pedagogy, as he argues it does, the field endorses a set of “bad habits” that he relates to a humanist mindset (533). Boyle proposes instead a view of writing and writing pedagogy that is “ecological” and “posthuman” (538). Taking up Kristine Johnson’s claim that the Framework opens the door to a revitalization of “ancient rhetorical training.” Boyle challenges the equation of such training with a central mission of social and political critique (534).

Boyle recounts a history of writing pedagogy beginning with “current-traditional rhetoric” as described by Sharon Crowley and others as the repetitive practice of form (535). Rejection of this pedagogy resulted in a shift toward rhetorical and writing education as a means of engaging students with their social and political surroundings. Boyle terms this focus “current-critical rhetoric” (536). Its primary aim, he argues, is to increase an individual’s agency in that person’s dealings with his or her cultural milieu, enhancing the individual’s role as a citizen in a democratic polity (536).

Boyle critiques current-critical rhetoric, both in its approach to the self and in its insistence on the importance of reflection as a route to critical awareness, for its determination to value the individual’s agency over the object, which is viewed as separate from the acting self (547). Boyle cites Peter Sloterdijk’s view that the humanist sense of a writing self manifests itself in the “epistle or the letter to a friend” that demonstrates the existence of a coherent identity represented by the text (537). Boyle further locates a humanist approach in the “reflective letter assignments” that ask students to demonstrate their individual agency in choosing among many options as they engage in rhetorical situations (537).

To develop the concept of the “ecological orientation” (538) that is consistent with a posthumanist mindset, Boyle explores a range of iterations of posthumanism, which he stresses is not be understood as “after the human” (539). Rather, quoting N. Katherine Hayles, Boyle characterizes posthumanism as “the end of a certain conception of the human” (qtd. in Boyle 539). Central posthumanism is the idea of human practices as one component of a “mangled assemblage” of interactions among both human and nonhuman entities (541) in which separation of subject and object become impossible. In this view, “rhetorical training” would become “an orchestration of ecological relations” (539), in which practices within a complex of technologies and environments, some of them not consciously summoned, would emerge from the relations and shape future practices and relations.

Boyle characterizes this understanding of practice as a relation of “betweenness among what was previously considered the human and the nonhuman” (540; emphasis in original). He applies Andrew Pickering’s metaphor of practice as a “reciprocal tuning of people and things” (541). In such an orientation, “[t]heory is a practice” that “is continuous with and not separate from the mediation of material ecologies” (542). Practice becomes an “ongoing tuning” (542) that functions as a “way of becoming” (Robert Yagelski, qtd. in Boyle 538; emphasis in original).

In Boyle’s view, the Framework points toward this ecological orientation in stressing the habit of “openness” to “new ways of being” (qtd. in Boyle 541). In addition, the Framework envisions students “writing in multiple environments” (543; emphasis in Boyle). Seen in a posthuman light, such multiple exposures redirect writers from the development of critical awareness to, in Pickering’s formulation, knowledge understood as a “sensitivity” to the interactions of ecological components in which actors both human and nonhuman are reciprocally generative of new forms and understandings (542). Quoting Isabelle Stengers, Boyle argues that “an ecology of practices does not have any ambition to describe things ‘as they are’ . . . but as they may become” (qtd. in Boyle 541).

In Boyle’s formulation, agency becomes “capacity,” which is developed through repeated practice that then “accumulates prior experience” to construct a “database of experience” that establishes the habits we draw on to engage productively with future environments (545). Such an accumulation comes to encompass, in the words of Collin Brooke, “all of the ‘available means'” (qtd. in Boyle 549), not all of them visible to conscious reflection, (544) through which we can affect and be affected by ongoing relations in rhetorical situations.

Boyle embodies such practice in the figure of the archivist “whose chief task is to generate an abundance of relations” rather than that of the letter writer (550), thus expanding options for being in the world. Boyle emphasizes that the use of practice in this way is “serial” in that each reiteration is both “continuous” and “distinct,” with the components of the series “a part of, but also apart from, any linear logic that might be imposed” (547): “Practice is the repetitive production of difference” (547). Practice also becomes an ethics that does not seek to impose moral strictures (548) but rather to enlarge and enable “perception” and “sensitivities” (546) that coalesce, in the words of Rosi Braidotti, in a “pragmatic task of self-transformation through humble experimentation” (qtd. in Boyle 539).

Boyle connects these endeavors to rhetoric’s historical allegiance to repetition through sharing “common notions” (Giles Deleuze, qtd. in Boyle 550). Persuasion, he writes, “occurs . . . not as much through rational appeals to claims but through an exercise of material and discursive forms” (550), that is, through relations enlarged by habits of practice.

Related to this departure from conscious rational analysis is Boyle’s proposed posthuman recuperation of “metacognition,” which he states has generally been perceived to involve analysis from a “distance or remove from an object to which one looks” (551). In Boyle’s view, metacognition can be understood more productively through a secondary meaning that connotes “after” and “among” (551). Similarly, rhetoric operates not in the particular perception arising from a situated moments but “in between” the individual moment and the sensitivities acquired from experience in a broader context (550; emphasis original):

[R]hetoric, by attending more closely to practice and its nonconscious and nonreflective activity, reframes itself by considering its operations as exercises within a more expansive body of relations than can be reduced to any individual human. (552).

Such a sensibility, for Boyle, should refigure writing instruction, transforming it into “a practice that enacts a self” (537) in an ecological relation to that self’s world.