I want to focus this blog post on Automated Essay Scoring (AES) or any sort of computer-assisted scoring—specifically, I’m point out some interesting stuff from our readings.
The one chapter I read from Ericsson and Haswell’s book was from Ken McAllister and Ed White, “Interested Complicities: The Dialectic of Computer-Assisted Writing Assessment”. The chapter discusses the complicit agents in using computer-assisted assessment. The authors offer these stakeholders: the researchers, the entrepreneurs, the adopters, and the users. For each stakeholder, the authors point to the relationship and complicities that stakeholder has with other stakeholders, but also the consequences of those complicities.
What I find particularly interesting is that the authors are outlining how AES is the logical response by the complicit agents (entrepreneurs and researchers specifically) when the narrative from teachers, writing instruction, and students is that we need a less labor-intensive work load, cheaper labor, less-expensive tuition and to focus on “more important” issues than on grading. To say that AES is the logical response does not necessarily mean that this is the technology we’ve all been waiting for—rather, it may mean that the exigencies in composition that are perceived from entrepreneurs and researchers (people in ETS and AES researchers) are not being interpreted correctly. Or that we—as composition folk—may be projecting the wrong exigencies; or we are not focusing on the right exigencies to begin with. What I mean to say is: when entrepreneurs and researchers find that the appropriate and logical response to the concerns of comp teachers is AES, what does that say about the concerns we’re projecting or circulating across the discipline?
I think about Bill Condon’s article “Large-scale assessment, locally-developed measures, and automated scoring of essays: Fishing for red herrings?” He discusses how AES is a red herring: it is more a sign of a larger issue. The more productive discussions should be redirected toward the original questions about the construct of writing. Condon uses AES as a way to talk about large-scale assessment and holistic scoring; he claims that AES is not necessarily the problem, but it is a representation of a long standing issue in how assessment practices are constructing what writing is. In other words, when AES is being offered as a logical way to assess writing, then maybe something’s wrong—as I’ve pointed out with McAllister and White.