Case Study with Monica Linden: Computerized exams with enhanced feedback

Instructor: Dr. Monica Linden, 2014 Teaching with Technology Award recipient

Course: NEUR1030 – Neural Systems

Course overview: NEUR1030 is a large lecture course serving about 150 students and required for Neuroscience and Cognitive Neuroscience concentrators. In the course, students are pushed to work on their higher-order cognitive skills including applying concepts to new situations and interpreting and analyzing data and research findings.
View course
 syllabus

Goal for project/technology use

  • Helping students to really “think like neuroscientists”.
  • Improve their higher-order cognitive skills.

Technologies and teaching methods used

  • Canvas site: Allows students to be organized. Readings for each lecture are added to the calendar. Assessments and grades are in Canvas.  
  • Electronic course reader: provides students with a free option over the printed course reader and includes hyperlinks for easy referencing through the book.
  • Lecture using multimedia PowerPoints: Allows Monica to spark interest in certain topics, and better illustrate complicated concepts.
  • iClickers: Assessing students in real-time with iClickers gives an idea of where students are in terms of the lower order knowledge which allows Monica to move onto higher order ideas more efficiently.
  • Lecture capture: Allows students to re-watch complicated parts of the lecture or to catch-up if they miss class. It was also used when Monica could not attend class due to a conference.   
  • Computerized exams with enhanced feedback: Students took their exams (which include multiple choice and short answer/brief essay questions) on the computer so that Monica and her TAs could easily generate detailed, personalized exam feedback forms for the students. The individualized feedback included a breakdown of exam performance based on subject material, learning objectives, and levels of Bloom’s Taxonomy, in an effort to help students refine their study habits to improve performance in the course.
    See a sample student report

Outcomes

To assess the effects of the detailed exam feedback, exam results from the semester with the intervention (2014) were compared to exam results from the previous year’s course without the intervention (2013). In both years, student performance improved on all types of exam questions throughout the semester. Students’ performance on higher-order questions improved more throughout the semester in 2014 as compared to 2013. However, there was no difference between years in improvement on questions using lower-order cognitive thinking. A student attitudes survey suggests that while students understood the feedback, only around half of the students felt the reports supported their success in the course. Overall, these results suggest that assessment feedback detailing the level of thinking skills may improve student performance on assessment questions requiring higher-order cognitive skills. However, more effort may be needed to demonstrate the importance of this feedback to the students.

Looking back, moving forward
Future work planned:

  • Would like to add interactive elements to the electronic course reader
  • Put more effort to help students understand how to use the feedback to refine their study habits

Tips for colleagues:

  • Do not  try to do it alone. Ask for help from Academic Technology team.
  • Take it one step at a time and follow iterative process rather than doing everything at once.
  • When using exam proctoring software, conduct a trial quiz before the actual exam.

Check back to see more case studies.

2012 TWTA Winner: Ira Wilson

Instructor: Ira Wilson

Course: PHP0310 Spring13 Health Care in the United States

Objectives: “My main teaching objectives revolve around how to make the subject matter – Health Care in the United States – accessible, interesting, and relevant to an undergraduate audience. The 3 main segments of the course are health care financing, health care providers, and health care regulation. These are all topics that have the potential to be stultifyingly boring. I use several strategies to try to bring the subject ‘to life.'”

Outcomes: “Outcomes are harder to assess. The class has grown from about 145 students, to 200 students, to 266 students in the 3 years I have taught the course, which probably speaks in part to the interrelated approaches to teaching/learning described above. It is my subjective view that the kinds of questions that people ask from one year to the next are increasingly sophisticated, which is exciting to me.”

Methods:

  1. “Habits of inquiry and analysis. I try to use this subject matter as an example of how to learn about any complex subject that is by its very nature multidisciplinary (i.e., it has elements of history, sociology, economics, politics, anthropology, and others). Insight requires that you think about the topic from a variety of different directions using tools and perspectives from many disciplines, and the habit of doing so is fully generalizable to virtually every other type of inquiry in the social sciences that one might engage in.

  2. Never dumb things down. I try to do is expose students to the kinds of concepts and ideas that graduate students and faculty write grants and papers about, but to do so in a way that is appropriate for their level of experience and sophistication. This can be challenging when some are freshman and other are senior health econ majors. This general approach comes directly from Jerome Bruner’s concept of the “spiral curriculum.” I find it very interesting, and very challenging, to present complex and sophisticated concepts like “risk adjustment” or “adverse selection” that health economists argue about in their subspecialty journals to a very diverse undergraduate audience.

  3. Make learning relevant and personal. I try to use narrative to its fullest extent as a teaching tool. In this class that means using cases and stories about real people getting real care. This is just one method to make it clear, again and again, that we may be talking about reimbursement methods, bureaucracies, and approaches to regulation and quality control – but at the bottom if it all are sick people who need timely, high quality care. It is really easy to lose this connection and make learning about structural aspects of care delivery dry and uninteresting. Sections are also designed to get students to think about how class materials relate to them and their families.

  4. Finally, using technology. The technologies that I use include trying to use Canvas to its fullest extent, using iClickers, and also videos. First, regarding Canvas (which I am sure I do not use to its fullest extent), I not only use the home page for the course as a message board, I also have a section called “In the News” that I used to post articles, papers, videos, and other links that are relevant to current class discussion. These are usually things that can be reviewed in 5-10 minutes and help bring home the point that relevant things are happening today all around the country that are relevant to what we are studying.”

Recommendations:

The hardest thing about using technology is that it is a tool, a means to an end, and not an end in and of itself. This is fundamental and obvious, but easy to forget when the “cool” factor of some types of technologies is so high. The mistakes I have made (and I experiment a lot, so there are lots of mistakes!) generally come when I haven’t carefully thought through the teaching point that I want to make, and use a technology for the sake of the technology rather than to make a specific teaching point in a specific way.

Case Study with Wendy Chun – Making the most of Multimedia

Wendy Hui Kyong Chun

Prof. Wendy Chun

Professor of Modern Culture and Media

MCM 0230
Digital Media
Spring 2010

Objectives

This course aimed to get students to think critically about new media in part by using new media, so technology was central to the course.  Rather than simply being tools, the technology became objects that we also analyzed.

Outcomes

The students got a richer sense of why new media mattered. I believe it is crucial to start with where the students are—to work with their preconceptions and their remarkable insights—rather than to flood them with information or critical theory. Using their blog posts in lecture also gave them a sense that their questions mattered so they really did use the blog space to think through ideas and to respond to their peers’ questions. The student work in general was fantastic. For a wonderful example of a response to assignment #1, please see Alice Hines’ assignment (http://foucodes.weebly.com/) — this was for a different version of the class (2008), but I used it as an example for the students in 2010. Also, for the group project, Fiona Condon, Paul Kernfeld, and Alp Ozcelik produced a tool for students to post comments / questions during lecture, which I used during a few lectures.

Methods

Video Clips/Websites, Prezi, Second Life, Blog