9 Cognitive Effects in Multimedia Learning

Tola Sylvan ’17

As a math and literacy tutor for a 10 year old boy, I often try to incorporate as many different modes of instruction as possible – from using tangible manipulatives for division, playing a lively game of ‘math soccer,’ where each goal only counts if you can answer an arithmetic question, to drawing imaginary planets, watching immersive videos, singing, dancing, and reading aloud.  I’ve found that the lessons where we do hands-on activities are more engaging, and that employing multimedia resources helps me teach the ‘big picture’ of a concept.  When multiple senses are called upon during an activity, my tutee seems more excited and engaged with the material.  I feel that he comes away from these lessons with a better overall grasp of the topic – in any case, we wind up talking on and on about the experience.  It’s imprinted in our memory.

An appreciation for multimedia instruction has grown in the past few decades, especially with the rise of digital audiovisual tools and the internet, but historically, education has placed the biggest emphasis on verbal instruction.  Is it inherently better to create lessons that draw on many different media and modes of thought?  Mayer’s theory of multimedia learning addresses the cognitive science behind the intuition that combining multiple modalities optimizes education.

In his paper, Mayer refers to a series of his experiments that play with different combinations of visual, animation, text, and narration lessons on topics such as the cause of lightning.  Mayer measured the effectiveness of different lessons through a ‘transfer score,’ or how well participants were able to respond to comprehension questions following the lesson.

The experiments serve as evidence for Mayer’s theory of multimedia learning, which draws on basic concepts in human cognition (for example, the fact that we have separate modality processing systems, which each have limited capacity, and the fact that we must actively process information in order to effectively encode it).  He argues that multimedia approaches to education are more likely to promote ‘meaningful learning.’  Meaningful learning is true comprehension – the formation of a mental model through active cognition.

9 effects Mayer observes in multimedia learning:

Multimedia effect – information presented in more than one medium (e.g. image and text, animation and verbal narration) leads to better learning outcomes than information presented in a single medium

Spatial contiguity effect – information presented together spatially (e.g. text overlaid on an image instead of presented on a separate screen) best impacts learning

Temporal contiguity effect – information presented together in time (e.g. narration during an animation, instead of before or afterwards) is better for learning than temporally disjointed information

Coherence effect – only relevant information should be included in a lesson, as extraneous information hinders the development of an accurate mental model

Modality effect – learners benefit from lessons that employ multiple modalities, like visual perception and auditory perception, because these systems can run in parallel and together increase capacity for cognitive load

Redundancy effect – it is not helpful to repeat the same information within a modality, as cognitive load is optimized and learning enhanced when information is delivered in separate modalities (e.g. an animation with an audio narration will be easier to process than an animation with written text included – the former leads to better learning outcomes)

Pretraining effect – learners benefit from getting to know the basic components and definitions relevant to a lesson before the actual lesson begins

Signaling effect – learners benefit from a signal before lesson material begins that encourages them to look for main points and ideas.  For example, asking someone a comprehension question before the lesson will help the person develop a basic concept schema, which facilitates better encoding during the lesson

Personalization effect – delivering information in a conversational instead of an official manner best promotes learning (e.g. a narrator speaks casually, as if to a peer)

Mayer’s 2002 paper is a bit old – his overview of media includes VCR, speaker systems, and cartoon animations.  With the advance of technologies like 3D modeling and VR platforms, the possibilities for multimodal, multimedia education expand exponentially.  For example, haptic feedback allows information to be reinforced through the touch sensory modality.  As VR systems grow more sophisticated, the number of modalities expressed should match the number of modalities experienced in real life (sight, sound, touch, taste, smell, proprioception, the vestibular system).  With VR, Mayer’s theory can be fully taken advantage of to optimize meaningful learning.

Mayer advocates for a ‘learner centered approach’ to designing multimedia education.  As the Gaspee team explores all the opportunities of cool new tech, we’re also staying true to a vision of education that’s built to best fit human cognitive needs.  The Gaspee experience will draw on many different modalities and media in order to help students form a deep appreciation and mental model for the Gaspee story.

Our hub and spoke model helps to consolidate information – the hub may serve as a centralized space where information is spatially grouped.  The hub may also be thought of as a sort of ‘pretraining’ or signaling ground: in this environment, the student is able to see an overview of all the experiences to come (since each spoke is represented by some visual element), and may also be able to review key information or questions about the Gaspee.   Each spoke experience may provide the student with a different look into the Gaspee affair, through visual, auditory, and physically interactive activities.  Our historical scenes will present characters that speak conversationally, which promotes engagement and social learning.  The ultimate goal is to leave the student with a meaningful, personal representation of the Gaspee Affair – the effect of a well-designed multimedia environment.

Source: http://www.sciencedirect.com/science/article/pii/S0079742102800056