© Copyright 2000 by Jerry Drake. The right to make additional exact copies, including this notice, for personal and classroom use, is hereby granted. All other forms of distribution and copying require permission of the author.

Innovation and Risk

The student evaluations can also be analyzed to obtain some information on the risks of making teaching innovations such as the modified instructional cycle. Since student evaluations are used for tenure, promotion, and salary decisions, the impact of innovation on student evaluations may pose a risk to the instructor. This issue of risk was assessed with two types of analyses.

First, the overall rating of this course was compared to the department, college, and university norms as well as the instructor's past history of course evaluations for this course. The overage rating for this course was 4.06 (N = 28) on a 5-point scale ranging from poor (1) to excellent (5). This mean overall evaluation was compared to the means for same question for the department (4.36), the college (4.25) and the university (4.26). The overall class rating is not significantly lower than any of these reference values (t (28) = -1.56, -0.97, and -1.02, respectively, which are non-significant differences).

Since this instructor typically had course evaluations at or above the department and college means, the overall class evaluation is also slightly lower than his typical evaluation. Given the variability in the class evaluations across students and the limited sample size, the safest conclusion for the overall ratings is that the innovation did not appear to make a strong negative impact on course evaluations although it may have had a slight negative impact.

The second analysis to evaluate the risk of innovation was to compare the qualitative student evaluations of the self-assessments and on-line quizzes with their final course evaluations. If student positive or negative reactions to innovation would strongly color their evaluations of the course, this potentially poses a risk to the instructor. To examine this issue, the student positive, neutral, or negative reactions to self-assessments and on-line quizzes were used to predict each of six class evaluation items. None of the six analyses approached statistical significance.

One final analysis was to condense the six class evaluation items into one factor score representing overall class evaluation. However, the student reactions to instructional innovations also did not predict that overall composite (F (2,25) = .071, non-significant). Therefore, within the limits of sample size and precision of measurement of all variables, there is no evidence of any carry-over of student reactions to innovation to the course evaluations. Based on this evidence, there is no general negative impact of innovation on student evaluations. That is, the risk to the instructor for innovation will depend very much on precisely how the innovation is constructed and implemented.

Poorly developed or implemented innovations could certainly impact adversely on student reactions and class evaluations. Conversely, well-developed and implemented innovations could positively affect reactions and evaluations. The potential negative risk in innovation may approximate the known negative effects of teaching a course for the first time (McKeachie and Chism, 1994) as compared to subsequent iterations of the class.

The Impact of WebCT Innovations on Learning Outcomes

Advantages and Disadvantages of Technology-Enriched Innovations