Although the scoring methods may have changed over the decades, the basic format and venue of the college writing placement test have remarkably remained the same at most universities and colleges for over one hundred years. During an orientation, matriculating freshmen are herded into lecture halls and asked to write one or two timed-impromptu essays using pen (or pencil) on paper. These tests reward writing fluency but prevent significant planning and revision – the two elements of the writing process most emphasized in contemporary composition classrooms. Moreover, the timed-impromptu makes it difficult to connect writing to the critical reading of substantial texts. Finally, many students now compose primarily on a computer, and, consequently, are most familiar with composing within an elastic medium, which allows for seamless deletion, insertion, and reordering, actions which are virtually impossible with pencil and paper.
This module describes the development of a now fully functioning national online essay evaluation service, the iCampus /MIT Online Assessment Tool or iMOAT, that has already changed how thousands of entering students are assessed each year at a diverse group of universities throughout the United States. iMOAT allows students to write essays at home on a computer, to base their writing on the critical reading of substantial essays, and to engage in all stages of the writing process--invention, drafting, revision, and editing. Moreover, iMOAT allows universities to automate many of the best practices of holistic writing assessment while maintaining complete autonomy in constructing assessments that reflect each institution’s particular mission, pedagogy, and student population.
The functionality of the online essay evaluation service can be best conveyed by presenting a narrative of a student taking an online placement test. Once the university loads the entering class into the system (using tab-delimited fields) the student may log on to system and register for a test. If the test involves online readings, the student may receive an email notifying him or her that the readings are available. The student logs in, accesses the readings, and optionally prints them out. A few days later, the student receives another email notifying him or her that the prompts for the two essays are available. The student then accesses the prompts and has 72 hours to plan, write, revise, and edit the two essays. Before the deadline, the student logs in and pastes each essay into a window. Several weeks later, both the student and advisor receive an email notifying them of the results of the placement along with comments by an experienced writing teacher and the texts of the two essays. Thus, the student's first academic encounter with his or her institution becomes an educational experience.
In 1998, Prof. Harold Abelson, then Chair of MIT's Committee on Undergraduate Admissions and Financial Aid, initiated the development of a home-grown online essay evaluation service that allowed students to securely access readings and writing prompts, submit essays, and receive both results and comments. Unfortunately, the original system was written for non-standard software that could not be easily modified and could be run only within the unique MIT computing environment.
In 2001, MIT and Microsoft Corporation’s iCampus Alliance funded the joint development of a robust Online Essay Evaluation Service by a consortium of diverse institutions, the California Institute of Technology, the University of Cincinnati, Louisiana State University, and DePaul University. By 2002, the system was operational.
Many of iMOAT’s benefits are obvious. It measures the whole writing process rather than just writing after, at most, five minutes of planning. It transcends space; students can access readings and write their essays almost anywhere in the world. It also allows students to use the tools they normally use in writing, such as a word processor.
There were also a few unexpected benefits. Almost all students with disabilities were able to accommodate themselves at home more effectively than they would have been accommodated on campus. Because students had sufficient time to write their essays and because they were able to view their essays, along with, on some campuses, detailed individual comments, the number of students contesting their scores dropped dramatically. In addition, because advisors often can easily access the essays online, the advising process is strengthened.
The Online Essay Evaluation Service, however, does have some limitations. When we began, we worried that many students would not have access to the system. Fortunately, over the five years of the service, lack of student access to a networked computer to take the exam has almost completely ceased to be a problem. Cybercafes are almost everywhere, and we have students taking tests from Uzbekistan and the Seychelle Islands with very little effort.
The main limitation of the online essay system is security, and, consequently, its unsuitability for high stakes testing. Because students write the essays online without supervision, there is no assurance that the student is actually writing the essays. Several schools, including MIT, try to compensate for this situation by including extensive readings and having students type out a statement that they wrote the essays themselves, without help from anyone else. We have explored high tech solutions to ensure security, but all of these "cures" are either impractical or worse than the problem.
Because the system is flexible, each institution was able to mold its use to fit its own educational philosophy and needs. Moreover, institutions began to use it in new and innovative ways. Irvin Peckham at LSU, for example, has successfully used iMOAT at the end of semesters to develop uniform grading among the many sections of First-year composition. Other schools are exploring its use as a mechanism for student writing portfolios.
iMOAT has also become a platform for the exchange of best practices and for new collaborations among schools. This summer, the Rice University Placement Test was administered through iMOAT using a prompt that had already been used at MIT. Moreover, the essays were holistically graded using MIT staff directed long-distance by Rice Faculty. Next year, Rice and MIT will collaborate in jointly administering and scoring all of their writing placement tests. We hope that other schools will join in using iMOAT to develop grass-roots collaborative assessments.
Perhaps the most exciting aspect of iMOAT is that it is collecting student essays in digital form that will be available for research. When students log on and receive their score, they are given a screen where they can, without any coercion, give informed consent for their essay to be used for academic research and for short passages to be quoted anonymously in academic publications. We already have thousands of student essays in digital form archived at MIT and we are now beginning to collect student essays on the same topics from Rice. Soon, we hope to make this resource available to composition researchers.
At this point, iMOAT is self-sustaining. Each school pays a yearly fee for unlimited use of the service and for limited programming and technical support. As more schools join, the yearly fee is decreasing. iMOAT is housed at the Center for Educational Computing Initiatives at MIT, but it will be governed by a consortium of member institutions. Early on, we thought of partnering with a for-profit entity, but we soon became convinced that any such collaboration would risk losing control. A central tenet of the service is that it is designed by writing programs for writing programs incorporating best practices and current research.
Joining the consortium is not the only ways schools can obtain the online service. With the support of MIT and Microsoft Research, we are making the source code available to any school who wants to run and administer the service on their own. In addition, CalTech has already developed their own version of the online essay evaluation, and we would be happy to provide any other institutions with the functional specifications. If there is interest, we will hold a workshop in Chicago on iMOAT after the 4C’s on Saturday, March 25, 2006.