For students, it's difficult to recognize the impact of the end-of-semester evaluations they're asked to provide for courses they've taken and the faculty who taught them. But their ratings and comments are vital to ensuring and improving teaching effectiveness.
The value of student evaluations of teaching -- data which isn't collected or analyzed at a university-wide level -- depends on who is using it and how. To offer insight on helpful ways to consider the data, the Center for Excellence in Learning and Teaching convened a faculty member, a former department chair and an associate dean to talk about their approaches. Here are some highlights from the April 5 panel:
'Teaching really matters'
As of fall 2016, about 6,300 Iowa State courses used the online Class Climate system for student evaluations of teaching, with students filling out more than 130,000 surveys that semester, said Amy Slagell, associate dean for academic programs in the College of Liberal Arts and Sciences. For LAS administrators, that mass of data has three main uses, she said.
The most frequent usage is in promotion and tenure decisions, where evaluation data combines with other measures such as peer observation to show a professor is succeeding in the classroom, Slagell said. Similar information is used in the college's in-depth reviews of tenured faculty every seven years.
Task force seeks input
A Faculty Senate task force is studying ways to make student evaluations of teaching more effective and fair. If you have comments or thoughts on the issue, contact task force chair Jonathan Sturm, jsturm@iastate.edu.
"The teaching really matters here," Slagell said. "It has to be solid to move forward."
However, administrators are aware of research into student evaluation bias and recognize that "not everybody can be above average," she said.
Evaluation comments and data also are useful when students raise concerns about a course or an instructor, helping to determine if the criticism is widely held, Slagell said.
"I know chairs look at it, but sometimes they're looking at a lot of course evaluations at the same time, so something might not have popped out," she said.
The third common use of student feedback is for recognizing excellent teaching. While materials supporting a faculty teaching award include numerous aspects, strong evaluation data is always part of the narrative, Slagell said.
"We want to catch people being good," she said.
Slicing and dicing
Kristen Constant has been interim vice president and chief information officer since Dec. 1, after serving five years as chair of the materials science and engineering department. As chair, she took a highly analytical approach to student evaluations.
Constant plotted course grades versus evaluations to ensure higher marks for students didn't mean higher marks for faculty. She analyzed evaluations based on year in school and course level. She shared anonymous data within the department so faculty could see how they compare. To root out bias, she compared instructors' summary scores -- a single rating meant to gauge overall effectiveness -- to an average of their scores for individual characteristics, ratings on specific aspects of teaching such as feedback quality, organization and clarity.
"I like data, so I slice it and dice it a lot," she said.
For more information
To learn more about creating, administering and analyzing student evaluations of teaching, see CELT's website devoted to assessment and evaluation.
For faculty performance evaluations, students' written comments were crucial, but she also looked at the high and low individual characteristic scores and discussed with faculty ways to address patterns of low performance. Faculty who scored two standard deviations below their colleagues were required to use an early-term evaluation and have peers observe them in the classroom, as was anyone preparing for a third-year or tenure review.
Constant is a big fan of early-term evaluations, which are conducted just a few weeks into a semester. Evaluation at that stage is meant to be formative, helping students take ownership of their learning and giving faculty feedback they can use immediately as the course progresses.
Early-term evaluations also were used by any instructor teaching a course for the first time, no matter their experience. Constant said many MSE faculty use them for every class they teach.
"They're extremely useful. You buy a lot of goodwill when you go back and respond to students' concerns," she said.
Continuous adjustment
When Elena Karpova, professor of apparel, merchandising and design, looks at her student evaluations, she's most interested in what her students are unhappy about. Seeing that she was ranked lower in explanation, for instance, helped her realize she'd incorrectly assumed her students grasped the purpose of her teaching methods.
"I was like, 'Wait, maybe they don't understand why I'm doing things,'" she said.
To be more transparent, she began explaining up front, either in class or on the syllabus, the reasoning behind different course activities. Often, she uses comments from past student evaluations to help make her points.
Like Constant, Karpova sees great value in early-term evaluations. She uses a Plus/Delta assessment early on in every course, a tool that asks students to consider their own efforts and the teaching of the course. Results are shared with the class. Though the feedback is remarkably consistent, it's still useful as another way to explain the "why" behind class assignments and activities, she said.
In addition to the standard voluntary course evaluation she encourages students to fill out, Karpova administers an evaluation of her own design during class time at the end of the semester. She lists each class activity and asks students to select which were most helpful for their learning. She also asks what could be changed to be more helpful, what they'd like to know more about and three takeaways from the course.
"It's very eye-opening for me," said Karpova, who adjusts her course materials based on the feedback.
That's how she knows that while students may not like her before-class online quizzes covering the assigned reading, they end up recognizing it forces them to keep pace.
"Even though they complain, at the end they said it was helpful for their learning," she said.
