 |
 |
|||
 |
  |
|||
SimNet XPert: An Effective and Innovative Component of an E-Learning Course for Microsoft Office Applications Dr. Kathleen A. Kegley Department of Management Clemson University Abstract This paper describes the transition from a traditional classroom course on Microsoft Office applications to an online, e-learning course. The e-learning course is based on the adoption of McGraw-Hill’s interactive software SimNet XPert. This software provides a feature-rich simulation environment that allows the learner to develop and practice application skills in a format that supports multiple learning styles with immediate performance feedback. The results of two pilot studies and one full scale adoption involving ten sections/three hundred students in the Department of Management at Clemson University are presented. A discussion of challenges and the impact on student learning concludes the paper. I. BackgroundThe Department of Management at Clemson University has offered a sophomore level course based on Microsoft Applications for several years. The course is required for a major in Management and is intended to provide a foundation for advanced courses that depend on skills with spreadsheets and databases. Consequently, the course has focused primarily on developing skills in Microsoft Excel and Access. Approximately ten sections are offered per semester, with about thirty students enrolled in each section. Because of the high variation in computer skills of entering freshmen, the course is offered on a pass/fail basis. This structure maintains the focus on developing a solid skill set without a grade bias that is highly dependent on a student’s previous experience. Since the creation of the course, the department has explored various ways to teach the material in an effective way. Due to the skills-based nature of the material, a traditional lecture style presentation was dismissed as ineffective. While it was clear that the students needed to develop skills in a hands-on fashion, it was not obvious how to best provide personal assistance to and assessment of students in a self-paced environment. Tutorial style textbooks were used so that students could proceed at their own pace. The role of the instructor became that of trouble-shooting individual problems as they appeared. Since the instructors’ availability was restricted to class time and office hours, the students would often have problems that halted their progress until the instructor became available. Alternately, the students would proceed with their work oblivious to a problem, only to find out much later that their efforts were incorrect. The development of fair and effective assessments presented yet another challenge. Various kinds of assessments were used by various instructors, ranging from hands-on use of the application to complete an assignment to multiple-choice questions. In the case of timed, hands-on application assignments, a student might have problems at one step that would cascade to future steps. This style of assessment resulted in a high degree of frustration for many students, and was difficult to grade in a uniform and fair manner. The use of multiple choice or fill-in-the-blank assessments might provide a reasonable assessment of a student’s vocabulary and basic concept awareness as it related to an application, but provided no indication of a student’s ability to actually apply the skills. II. SimNet XPert Preliminary ReviewAs a result of the frustrations with both the teaching and assessment of material that was considered to be essential for success in advanced courses, alternatives were sought and investigated. After a preliminary review, McGraw-Hill’s SimNet XPert software appeared to offer solutions to many of the challenges described above. Unlike tutorials that tend to be project based or textbooks that focus on general concepts, SimNet XPert teaches the individual skills that comprise each application. Since each skill is well-defined and can be presented in a relatively short module, a student can focus on learning one skill at a time in a structured format. SimNet XPert offers lessons for each skill in a learning environment that supports multiple learning styles in that it provides:
|
|
|||
Figure 1: Typical “Teach Me” in |
Figure 2: Typical “Show Me” in |
|||
| The practice session, called “Let Me Try”, occurs within an environment that simulates the application software. Hints are offered as needed and immediate feedback indicating if the student has performed the skill correctly is provided. The student may choose to take a lesson, attempt the “Let Me Try” exercise and then return to any portion of the lesson as needed to modify or reinforce the understanding of the skill. Alternately, a student may elect to attempt the “Let Me Try” exercises prior to studying the corresponding lesson as a “pretest” and use the outcome to determine if further study is needed. As a result of the flexible and engaging learning environment, students can actively and effectively navigate the material in a manner that best suits their personal knowledge level and learning style preferences. | ||||
Figure 3: A Typical “Let Me Try” from SimNet XPert |
||||
| The assessments demonstrate a strong correlation with the lessons and the “Let Me Try” exercises in that each is based on the same skill set (Figure 1). For a given skill, there is a lesson describing the skill, a “Let Me Try” exercise that allows a student to practice the skill, and a question in an assessment to evaluate the student’s capability to perform the skill. From a student’s perspective, the assessments are very similar to a collection of “Let Me Try” exercises with a slight change in scenario. Therefore, the students can practice the “Let Me Try” exercises as many times as needed in order to feel comfortable during the assessment. Since each question on the assessment refers to an individual skill, the problem of cascading errors during a project-based assessment is avoided. Furthermore, since the student is required to perform a hands-on skill within a simulation environment, a measure of capabilities with the application can be obtained beyond that provided by multiple-choice style assessments. The ability to offer the assessments online at any time during the semester provides yet another attractive feature since students with different backgrounds and learning speeds are able to proceed not only with learning but with assessments at their own pace. Finally, the realization that Microsoft Office certification exams are based on a similar skills-based performance structure added a real-world value that is effective in motivating students. In addition to earning academic credit and obtaining useful skills for other classes, the course may be viewed as preparation for passing a certification exam that could substantially enhance their position in future interviews. | ||||
Figure 4. The SimNet software is skills based. For each skill selected by the instructor, a lesson is provided to describe the skill, a “let me try” exercise allows the student to practice the skill followed by immediate feedback, and a question on an assessment evaluates the student’s ability to perform the skill. |
||||
 |
||||
III. Pilot StudiesPrior to the full scale adoption of the SimNet XPert software, two pilot studies were conducted during the summer of 2003. During the first summer session, SimNet XPert was adopted as the required course material. While Microsoft books on Excel and Access were included in the bundle, they were used for reference rather than as an integral component of the course. The Office Specialist Lessons in the SimNet XPert software were used as the basis for assignments and assessments since they provided coverage of the basic skill set covered on the corresponding Microsoft certification exam. The relatively small number of students during these sessions allowed frequent observation and monitoring of their experience. Attendance was required during the pilot studies to evaluate students’ progress with software installation and interaction. Students attended class each day during the semester and worked individually with the software. In keeping with past semesters, only Microsoft Excel and Access were covered in the course. One somewhat surprising early observation was that after a brief learning curve with the software interface, there were remarkably few questions from the students regarding the content material. Furthermore, they remained actively engaged with the software throughout the hour of the class period. Early within the first pilot study it became clear that the role of the instructor was to become more of a coach than a teacher. One of the first practical challenges that required resolution was the development of assessments. While the SimNet XPert software features made the actual creation a simple matter, it was not clear how many tasks should be included and what time period should be allowed. The assessments for the applications that were based on the Office Specialist Lessons contained over 100 tasks, which seemed like an enormous number of problems in the traditional sense for a one hour class period. However, two teaching assistants attempted and passed the assessments with very little preparation. While their experience with the applications was greater than that expected from most students, the decision was made to use the full set of tasks for each application. In order to promote success, students were given three chances to pass each application assessment. This organization allowed them to use the assessments as learning tools and reduced the stress associated with assessments while still supporting the goal of skill set development. The thought was that very few students would actually need the third attempt; later data in the full scale adoption would support this as a reality. Having established a structure for the assessments, the requirements for taking lessons was considered. Based on the differing backgrounds of the students, the decision was made to impose no requirements regarding the lessons. The relevant assignments were noted on the syllabus, but the student assumed control of their own learning experience in this regard. Given three opportunities to pass an assessment, a student might prefer to take one of the assessments prior to taking any lessons. As later observed, achieving a low score on the first assessment seemed to inspire a dedication to the material beyond that seen in traditional study programs. Students who did not pass the first exam seemed to view the assessment much as a computer game and demonstrated a strong desire to “win”. The strong correlation between the practice exercises and the assessments appeared to give them the confidence to believe that winning was achievable. The high success rate seen on second assessments during the later full scale adoption supports this observation. The assessments during the first pilot study were conducted within the classroom on specified dates. For example, the first Excel assessment was offered during the first week, the second Excel assessment during the second week, etc. However, the dynamic and diverse nature of even a small number of students soon made this structured approach seem constrictive and of no real value to the course. It also led to some potential embarrassment for students who might need to take the exam on a third try (especially if he/she was the only one in the room). Furthermore, it forced an order of study that had no particular relevance to the learning experience; the choice of having Excel exams prior to Access exams was completely arbitrary. In the second pilot study, the constraints imposed earlier were relaxed significantly. Each assessment was open and available to the students at any time during the semester, and attendance was required only for the purpose of taking assessments. This latter constraint was largely due to the fact that the lab computers had the software installed to support testing, and the requirements for installation on personal computers had not been considered yet. There were no apparent problems with this course structure consisting of a pass/fail grading system, assessments based on the complete Office Specialist Lessons, three chances to pass each assessment, and all assessments open during the semester. Therefore, this structure formed the basis for a full scale adoption involving ten sections with approximately three hundred students in the fall. Although there were no formal student evaluations, students in the pilot studies were overwhelmingly positive about their experience with the software, and generally confirmed that they believed they had learned more than they would have in a traditional class. Their comments about course difficulty did influence the decision to increase the applications covered in the fall to include Microsoft Word and PowerPoint. While most students have a passing familiarity with both of these applications, the decision was made to formally include them in the course in order to strengthen and validate skills in these applications. |
||||
IV. Full Scale AdoptionBased on the results of the pilot studies, the course development team concluded that the SimNet software was an innovative product that could add great value to the MGT218 Business Applications course. The final evaluation of the SimNet software included the following key observations:
These observations led to unanimous support for the adoption of the SimNet software. However, the adoption of the innovative software required innovation in course structure as well. The general philosophy of the course is to prepare the students for advanced classes that use Microsoft applications in projects and other types of course work. A goal for the course model is to provide a high success rate for any student willing to make a reasonable effort. While the SimNet software offers many valuable features, the online testing is based on a technology that is vulnerable to failures beyond a student’s control. Therefore, the development team decided that no technological issue would be the cause for a student failing the class, and that any doubt in a questionable situation would be resolved in favor of the student. The team perceived that online testing might be the cause of stress for the students, and repeatedly sought ways to address this issue.
The following course model was established to support the adoption of the SimNet software:
This model was implemented for 10 sections of the course which involved approximately 300 students. Support for the course included two graduate students responsible for 5 sections each, one faculty member to serve as technical administrator, and one faculty to serve as academic administrator. The graduate students interacted by email with the students to help resolve problems as they arose as well as kept track of the grades submitted by the students. One faculty member served as technical administrator, with duties including setting up the online exams on the PageOut server, and interacting with the graduate students, Clemson technical support (to maintain the SimNet software in the student labs), SimNet technical support, and PageOut technical support to resolve problems as needed. This faculty member also served as a backup for student communications in the event the graduate students had difficulty with their email. Another faculty member served as academic administrator and was responsible for managing the details of student enrollment (e.g., course balancing, drops, adds), contributing to the academic integrity of the course structure and resolving major academic issues as needed. |
||||
V. ResultsIn order to evaluate the trends of students’ success and failure on the exams, the number of students attempting an exam was compared to the number of students passing the exam with a score of 75% or higher for each application covered in the course. As Figures 5 – 8 show, the majority of the students passed the exams on the first attempt, very few if any students needed the third chance, and only 2 people of 240 failed the course due to failing an exam on the third and final chance. |
||||
Figure 5. The results of 240 students for the Microsoft Access exams. On the first attempt, 173 of 240 students passed the exam with a score of 75% or higher. Of the 67 students who needed a second chance to pass the exam, 60 successfully passed. In the third and final chance, 5 of the 7 taking the exam passed. Therefore only 2 students out of 240 failed to pass the Access exam on all three chances. |
||||
Figure 6. The results of 240 students for the Microsoft Excel exams. On the first attempt, 178 or 240 students passed the exam with a score of 75% or higher. Of the 62 students who needed a second chance to pass the exam, 55 successfully passed. In the third and final chance, all 10 students taking the exam passed. |
||||
Figure 7. The results of 240 students for the Microsoft PowerPoint exams. On the first attempt, 203 of 240 students passed the exam with a score of 75% or higher. On the second attempt, each of the 37 students who took the exam passed. Therefore, no students needed the third chance to take the exam. |
||||
Figure 8. The results of 240 students for the Microsoft Word exams. On the first attempt, 172 of 240 students passed the exam with a score of 75% or higher. Of the 68 students who needed a second chance to pass the exam, 63 successfully passed. In the third and final chance, all 5 of the students taking the exams successfully passed. |
||||
VI. ConclusionsThe full scale adoption of SimNet was deemed highly successful at the end of Fall Semester 2003 and was continued in the Spring Semester 2004. Criteria for success included the exam results and student comments. Most students indicated that they preferred the online version of the class, although several commented that meeting face to face on one day might be helpful. Three students indicated that they preferred to have deadlines to keep them on track, while the remaining 237 students indicated that they liked the flexibility of the class structure. A remarkably small percentage (approximately 3%) of the students indicated a negative evaluation of the class. Sample comments from the anonymous evaluations include the following:
In response to a question asking about any teaching methods found to be particularly helpful, sample responses include the following:
One of the most surprising responses in the student evaluations was the high level of confidence shown by students in their computer skills. Almost half (45%) of the respondents indicated they were strongly considering taking the Microsoft Certification exam. The high success rate on rigorous skills-based exams, the overwhelmingly positive student comments and a high level of student confidence in computing skills provide strong evidence that SimNet XPert is an effective and innovative component of a successful e-learning course for Microsoft Office Applications. |
||||
 |
 |
|||
