College Composition Weekly: Summaries of research for college writing professionals

Read, Comment On, and Share News of the Latest from the Rhetoric and Composition Journals


Leave a comment

Donahue & Foster-Johnson. Text Analysis for Evidence of Transfer. RTE, May 2018. Posted 07/13/2018.

Donahue, Christiane, and Lynn Foster-Johnson. “Liminality and Transition: Text Features in Postsecondary Student Writing.” Research in the Teaching of English 52.4 (2018): 359-381. Web. 4 July 2018.

Christiane Donahue and Lynn Foster-Johnson detail a study of student writing in the “liminal space” between a “generic” first-year-writing course and a second, “discipline-inspired” first-year seminar (365). They see their study as unusual in that it draws its data and conclusions from empirical “corpus analysis” of the texts students produce (376-77). They also present their study as different from much other research in that it considered a “considerably larger” sample that permits them to generalize about the broader population of the specific institution where the study took place (360).

The authors see liminal spaces as appropriate for the study of the issue usually referred to as “transfer,” which they see as a widely shared interest across composition studies (359). They contend that their study of “defined features” in texts produced as students move from one type of writing course to another allows them to identify “just-noticeable difference[s]” that they believe can illuminate how writing develops across contexts (361).

The literature review examines definitions of liminality as well as wide-ranging writing scholarship that attempts to articulate how knowledge created in one context changes as it is applied in new situations. They cite Linda Adler-Kassner’s 2014 contention that students may benefit from “learning strategy rather than specific writing rules or forms,” thus developing the ability to adapt to a range of new contexts (362).

One finding from studies such as that of Lucille McCarthy in 1987 and Donahue in 2010 is that while students change the way they employ knowledge as they move from first to final years of education, they do not seem fully aware of how their application of what they know has changed (361-62). Thus, for Donahue and Foster-Johnson, the actual features detectable in the texts themselves can be illuminating in ways that other research methodologies may not (362, 364).

Examining the many terms that have been used to denote “transfer,” Donahue and Foster-Johnson advocate for “models of writing knowledge reuse” and “adaptation,” which capture the recurrence of specific features and the ways these features may change to serve a new exigency (364).

The study took place in a “selective” institution (366) defined as a “doctoral university of high research activity” (365). The student population is half White, with a diverse range of other ethnicities, and 9% first-generation college students (366). Students take either one or two sections of general first-year writing, depending on needs identified by directed self-placement (366), and a first-year seminar that is “designed to teach first-year writing while also introducing students to a topic in a particular (inter)discipline and gesturing toward disciplinary writing” (365). The authors argue that this sequence provides a revealing “’bridge’ moment in students’ learning” (365).

Students were thus divided into three cohorts depending on which courses they took and in which semester. Ninety percent of the instructors provided materials, collecting “all final submitted drafts of the first and last ‘source-based’ papers” for 883 students. Fifty-two papers from each cohort were randomly chosen, resulting in 156 participants (366-67). Each participating student’s work was examined at four time points, with the intention of identifying the presence or absence of specific features (368).

The features under scrutiny were keyed to faculty-developed learning outcomes for the courses (367-68). The article discusses the analysis of seven: thesis presence, thesis type, introduction type, overall text structure, evidence types, conclusion type, and overall essay purpose (367). Each feature was further broken down into “facets,” 38 in all, that illustrated “the specific aspects of the feature” (367-68).

The authors provide detailed tables of their results and list findings in their text. They report that “the portrait is largely one of stability,” but note students’ ability to vary choices “when needed” (369). Statistically significant differences showing “change[s] across time” ranged from 13% in Cohort 1 to 29% in Cohort 2 and 16% in Cohort 3. An example of a stable strategy is the use of “one explicit thesis at the beginning” of a paper (371); a strategy “rarely” used was “a thesis statement [placed] inductively at the middle or end” (372). Donahue and Foster-Johnson argue that these results indicate that students had learned useful options that they could draw on as needed in different contexts (372).

The authors present a more detailed examination of the relationship between “thesis type” and “overall essay aim” (374). They give examples of strong correlations between, for example, “the purpose of analyzing an object” and the use of “an interpretive thesis” as well as negative correlations between, for example, “the purpose of analyzing an object” and “an evaluative thesis” (374). In their view, these data indicate that some textual features are “congruen[t]” with each other while others are “incompatible” (374). They find that their textual analysis documents these relationships and students’ reliance on them.

They note a “reset effect”: in some cases, students increased their use of a facet (e.g., “external source as authority”) over the course of the first class, but then reverted to using the facet less at the beginning of the second class, only to once again increase their reliance on such strategies as the second class progressed (374-75), becoming, “‘repeating newcomers’ in the second term” (374).

Donahue and Foster-Johnson propose as one explanation for the observed stability the possibility that “more stays consistent across contexts than we might readily acknowledge” (376), or that in general-education contexts in which exposure to disciplinary writing is preliminary, the “boundaries we imagine are fuzzy” (377). They posit that it is also possible that curricula may offer students mainly “low-road” opportunities for adaptation or transformation of learned strategies (377). The authors stress that in this study, they were limited to “what the texts tell us” and thus could not speak to students’ reasons for their decisions (376).

Questions for future research, they suggest, include whether students are aware of deliberate reuse of strategies and whether or not “students reusing features do so automatically or purposefully” (377). Research might link student work to particular students with identifiers that would enable follow-up investigation.

They argue that compared to the methods of textual analysis and “topic-modeling” their study employs, “current assessment methods . . . are crude in their construct representation and antiquated in the information they provide” (378). They call for “a new program of research” that exploits a new

capability to code through automated processes and allow large corpora of data to be uploaded and analyzed rapidly under principled categories of analysis. 378

 


3 Comments

Lancaster, Zak. Discourse Templates in They Say/I Say. CCC, Feb. 2016. Posted 03/13/2106.

 

Lancaster, Zak. “Do Academics Really Write This Way? A Corpus Investigation of Moves and Templates in They Say/I Say.College Composition and Communication 67.3 (2016): 437-64. Print.

Zak Lancaster analyzes three corpora of academic writing to assess the usefulness of “templates” provided for student use in the textbook They Say/I Say (TSIS), by Gerald Graff and Cathy Birkenstein. Lancaster ultimately concludes that the most cogent critique of TSIS is not that it encourages students to use “formulaic” constructions but rather that the book does not supply students with the templates that academics actually use and hence, in fact, is not “formulaic” in ways that would most effectively shape students’ understanding of academic discourse (450).

Lancaster focuses on the book’s provision of specific sets of word strings to help students structure their arguments, in particular, first, phrases that acknowledge counter-arguments and second, those that concede to alternative points of view while, in Graff and Birkenstein’s words, “still standing your ground” (qtd. in Lancaster 440). Lancaster recounts that the use of formulas to guide students in incorporating others’ viewpoints has provoked debate, with some analysts endorsing the effort to supply students with explicit language for “moves” in the academic conversations they are expected to enter, and others characterizing the provision of such specific language as a “decontextualized” approach guilty of “reducing argumentation down to a two-part dialogue” (438).

For Lancaster, this debate, though meaningful, begs the basic question of whether the templates provided by TSIS actually “capture the tacitly valued discursive strategies used in academic discourses” (439). Lancaster finds this question important because linguistic analysis indicates that variations in wording shape “different roles for the reader . . . and different authorial personae, or stances,” conveying different values and encouraging different approaches to argumentation (440).

Lancaster cites research showing that what some linguists call “lexical bundles” are indeed common in academic writing across disciplines. “[H]ighly functional” phrases such as “it should be noted that,” or “the extent to which” are used more often by expert writers than by students (441). Lancaster’s example of “hedging formulas” such as “in some cases” or “appears to be” introduces his claim that such formulas have an “interpersonal function” in concert with their “ideational meanings” (442), supplying the same information but creating different valences in the reader/writer relationship.

Research on student texts, Lancaster reports, shows that students often succumb to what some scholars call “myside bias,” struggling to include counterarguments (443). In Lancaster’s view, evidence that students who are able to overcome this bias produce more complex, “mature” arguments (444) justifies strategies like those in TSIS to open students to a more dialogic approach to argument, which they may tend to see as a matter of “winning” rather than negotiating meaning (444). Lancaster claims, however, that TSIS could provide “more systematic attention to the details of language” to offer more substantive guidance in the ways these details affect interpersonal meanings (444).

Lancaster examines three corpora: one of expert academic writing drawing from “almost 100 peer-reviewed journals across disciplines”; one of “829 high-graded papers” by advanced undergraduates and “early graduate students across sixteen fields”; and one of “19,456 directed self-placement (DSP) essays” from the University of Michigan and Wake Forest University (444-45). Lancaster examined each body of writing using “concordancing software” to search for the exact phrases proposed by TSIS, to find other phrases serving the same functions, and to examine the precise contexts for each formula to make sure that it functioned like those featured in TSIS (445). The tables presenting the findings are based on “the normalized frequency” of occurrences rather than the raw numbers (446).

Analysis of the ways in which the writers in the corpora “entertain objections” revealed “six recurring options” that Lancaster ranks as moving from “direct” moves such as “Naming the reader” and “Naming your naysayers” (a characterization quoted from TSIS) through less direct moves that he denotes as “Unattributed” like “One might argue” or a passive-voice construction, to indirect phrases like nominalizations (“Another explanation”) or what linguist Geoff Thompson calls the “Hypothetical-Real” formula: phrases like “At first glance” or “It may appear that” that suggest that the writer will delve beneath the surface to present unrecognized truths (447-48).

Analysis indicates that first-year writers did consider alternative views at frequencies comparable to those in the more advanced work. In general, indirect phrases were much more commonly used than direct ones in all corpora; Graff and Birkenstein’s “Naming your naysayers” was the least frequently used option (448-49). Though they did “name the readers” more than the first-year writers, advanced writers preferred indirect approaches at higher levels than less advanced writers (450).

Lancaster posits that the use of more indirect choices by more advanced writers, counter to the guidance in TSIS, suggests that writers resist claiming to know what readers think, a form of “interpersonal tact” (448). Importantly for Lancaster, the specific phrasings offered in TSIS “do not appear in any of the corpora” (450). Similar but subtly different phrasings perform these functions (450-51).

Lancaster’s discussion of concession notes that while TSIS describes this move in terms of “‘overcoming’ objections” (qtd. in Lancaster 452), for linguists, such interactions create “solidarity with interlocutors by affirming and validating their views” (452). Lancaster draws on the work of James R. Martin and Peter R. White to base his analysis on the concept of “concede + counter,” in which a concession move is signaled with “high-certainty adverbials” like “undoubtedly” or “to be sure,” while the counter follows through the use of words like “yet,” or “at the same time.” Lancaster notes that in advanced samples, the opening concession phrase may not even appear (452), with the result that the move may be inconsistently tagged by the software (453).

Findings indicate more explicit use of concession by the less experienced writers (452). Lancaster proposes that this difference may result from the placement-essay writers’ sense that they were expected to “strike an adversarial stance” requiring more “direct language”; conversely, the software may not have picked up more subtle moves by more advanced writers (453). First-year samples were much more likely to include the kinds of wordings TSIS recommends, such as “It is true that. . . .” (454). However, none of the writers at any level used “personalized and overt signals” like “I concede that” or “Proponents of X are right” (454).

In investigating the “counter,” Lancaster discovered that the direct phrases encouraged by TSIS, such as “I still VERB that,” were not favored by any group; shorter, less direct wordings predominated. In fact, “On the other hand,” recommended by TSIS, tended to indicate a contrast between two positions rather than a “counter” following a concession (454).

Lancaster extracts three conclusions: all groups opted most often for indirect means of considering objections; writers consistently chose to “eagerly” endorse shared viewpoints when conceding; and less experienced writers used more direct concessions like those suggested by TSIS (455).

Differences in genre and context, Lancaster notes, may affect the validity of his findings. However, he sees “interpersonal tact” as “an implicit guiding principle” that is “pervasive” in academic writing (456-57). He notes that TSIS formulas do use hedges, but posits that the authors may not “see” these interpersonal markers because the hedging phrases have become naturalized (457).

In Lancaster’s view, TSIS often echoes a common perception of argument as a form of combat; he argues that the best academic writing more fully resembles a conversational exchange, and suggests that attention to the specific details of academic language provided by “systematic analysis” (459) such as corpora research can refocus instruction on how academics do incorporate interpersonal meanings into their discourse and how students can best use these moves when they wish to enter academic conversations (458-59).