Assessment and "Hard Data:" Improving the Public and Administrative Perception of Your Basic Writing Program

Go to Related Resources

By Gregory R. Glau, Arizona State University

       I'm using the term assessment here in the sense of programmatic assessment, a lens through which we can examine a basic writing (BW) program by questioning what it does. I'm also suggesting that we focus on what I'm calling hard data-statistics and data, say, rather than anecdotal or narrative information.

       It seems to me that we often don't do a very effective job of finding out what actually happens in our Basic Writing programs, nor do we use that information to our advantage. I think that facts, data, and other statistical information can help us improve our pedagogy and at the same time positively influence how others-especially administrators-see and understand the work we do.

       I've been working with the "what can we find" and "what can we do" notions for a few years now, starting with a memo I sent around to the President of our university and other associated administrators, about the success of our BW program. I also presented on this topic at the 1999 CCCC, published a version of that paper on ERIC, and I'm also focusing in some detail in an essay in the forthcoming The Writing Program Administrator's Resource: A Guide to Reflective Institutional Practice (Ed. Theresa Enos and Stuart Brown. Erlbaum, 2002). What's below is shamelessly stolen . . . er, um, well, I mean, informed by those discussions.

Invention Work

       In addition to figuring out what questions to ask and how to find the answers to those questions, we also need to consider who on campus might be able to help us find useful data. Almost always, there is someone at every college who already collects statistical information on that school's student population; these offices are often called Institutional Analysis or Institutional Advancement or Data Collection, so the first step is to learn who on campus might have statistical information about students and programs.

       Poke around your own campus: Does your college have a student database that you or your office staff can access? Many have several: one database may contain "live" or current information and another might hold the information that is archived at various times. Many systems have a query program available through which you can gather the statistical data, and these days most can download that information into an Excel or other spreadsheet. Once the data is in the spreadsheet, it can be sorted, filtered, charted, and so on.

       Or perhaps your BW program now conducts surveys or offers other data-collection instruments in which you might find useful data? If not, might you begin such a survey?

       Or perhaps you'll need to do data collection the old-fashioned way: manually tabulating information from printed summaries.

Asking Questions

      Please understand that I'm a long way from being a statistician, and I expect most of you are, too, so while it might be useful to figure the standard deviation and pose other statistical tests, I want to keep our look at hard data simple. Administrators, too, understand and like to work with easily-understood pieces of information (what John Ramage calls "nuggets" of information), so that's another reason to keep things simple.

      In any case, we all can help our own situations if we actively try to collect and then distribute information-and again, here I mean "hard data"-that serves to convince others that what we do is "working." What questions might we start with? Perhaps questions such as

• Can we say that our BW writing programs produce better writers? How do we know?
• How accurate-and what do we mean by that term (see the previous discussion on this list by Bill DeGenaro and Ed White)-is our placement process and how can we tell?
• Can we say that we help our students succeed in subsequent writing classes and/or in their other classes? How might we tell?
• At what rate do our students go on to the next class in our course sequence? Is that rate increasing or decreasing and why?
• Do we know why we lose students? That is, do students fail? Do they drop out? Are those rates changing or improving? Can we somehow influence those rates?

      There are, of course, countless other questions we might ask, but I'm convinced that unless we can answer such questions-questions about student performance, questions about student retention-and that unless we become proactive in finding and publicizing that information, that data, in the right form, that others will do it for us, which in effect often means they're doing it to us . . . and not always in the way that best represents the work that we do.

Discussion Prompts

      In the Related Resources to this discussion, I've provided several tables and graphs-ways to display program information that helps people understand it, based on ASU's students and BW program. But the first place to start is with the questions we'd like to ask, and then to figure out what kind of data might help answer these queries. So I'm wondering (and again, these questions are just to get us started thinking-I certainly don't have all the answers. And, not every question will apply to every local BW situation):

• Who are our students? That is, what are their average test scores? High school GPA? What is their ethnic mix?
• Do we have any sense of whether our BW program "works" better for some groups of students than for others? Why?
• Have we ever asked our BW students about their views of what a writing class should be and do? About how well they think our program matches their expectations? While such questions may lead to narrative responses (useful in their own ways), they also can be tabulated so we can report that "More than 90 percent of our students-based on surveying 2,800 of them-says that our ENG XXX class helped their writing."
• And, if we think that we do have a successful program, how can we use that information to convince others, especially those who control our purse strings?

Works Consulted and Recommended

Adler-Kassner, Linda, and Gregory R. Glau, ed. The Bedford Bibliography for Teachers of Basic Writing. New York: Bedfords/St. Martins: In Press. A list of the essays and books that are included in this bibliography is available at <http://www.asu.edu/clas/english/composition/cbw/reading_list.html>

Anson, Chris M., and Robert L. Brown, Jr. "Subject to Interpretation." The Writing Program Administrator as Researcher. Ed. Shirley K. Rose and Irwin Weiser. Portsmouth, NH: Boynton/Cook Heinemann, 1999. 141-52.

Collins, Terence. "Basic Writing Programs and Access Allies: Finding and Maintaining Your Support Network." Paper presented at 1998 Conference on College Composition and Communication. Chicago: 1 April 1998.

DeGenaro, William, and Edward M. White. "Going Around in Circles: Methodological Issues in Basic Writing Research." Journal of Basic Writing 19 (2000): 22-35.

Enos, Theresa, and Stuart Brown, ed. The Writing Program Administrator's Resource: A Guide to Reflective Institutional Practice. Mahwah, NJ: Erlbaum, 2002.

Ferganchick-Neufang, Julia. "Research (Im)Possibilities." The Writing Program Administrator as Researcher. Ed. Shirley K. Rose and Irwin Weiser. Portsmouth, NH: Boynton/Cook Heinemann, 1999. 18-27.

Freeman, G., M. Gum, and J. M. Blackburn. "Proactive Approaches to Improving Outcomes for At-Risk Students." ERIC document ED 430948, 1999.

Glau, Gregory R. "The `Stretch Program': Arizona State University's New Model of University-Level Basic Writing Instruction." WPA: Writing Program Administration 20 (1996): 79-91.

---. "Hard Work and Hard Data: Getting Our Message Out." Paper presented at the 1999 CCCC Convention. Atlanta, March 1999. ERIC document ED 430229, 2000.

Gleason, Barbara. "Evaluating Writing Programs in Real Time: The Politics of Remediation." College Composition and Communication 51 (Feb. 2000): 560-88.

Kinkead, Joyce and Jeanne Simpson. "The Administrative Audience: A Rhetorical Problem." WPA: Writing Program Administration 23.3 (Spring 2000): 71-84.

Rhodes, Keith. "Marketing Composition for the 21st Century." WPA: Writing Program Administration 23.3 (Spring 2000): 51-69.
"Student Racial/Ethnic Diversity: Trends in Enrollment, Graduation, and Retention." ERIC document ED 430467, 1999.

White, Edward. Developing Successful Writing College Writing Programs. San Francisco: Jossey-Bass, 1989.

---. "The Rhetorical Problem of Program Evaluation and the WPA." Resituating Writing. Ed. Joseph Janangelo and Kristine Hansen. Portsmouth, NH: Boynton/Cook Heinemann, 1995. 132-50.

White, Edward M., William D. Lutz, and Sandra Kamusikiri, Ed. Assessment of Writing: Politics, Policies, Practices. New York: MLA, 1996.

Witte, Stephen, and Lester Faigley. Evaluating College Writing Programs. Carbondale: Southern Illinois UP, 1983.

Yancey, Kathleen Blake. "Outcomes Assessment and Basic Writing: What, Why, and How?" BWe: Basic Writing e-Journal 1 (1999). <http://www.asu.edu/clas/english/composition/cbw/bwe_summer1999.htm>