Media in Cognitive Neuroscience: Using Film and TV to Assess Human Memory

Written by: Jacinda Taggett

Cognition (better known as the overall thought encompassing attention, language, decision-making, and memory) is an important aspect of daily life, yet it still has many questions unanswered. Psychologists now often lean more towards a neuroscientific approach, utilizing images and quantifications of the brain in order to answer “how” and “why” these processes happen. As psychological science starts to move towards explaining cognition through this lens, we also see an increase in wanting to study these processes in naturalistic contexts.

For example, many memory studies have been done using word pairs or trivia questions to assess “long-term memory”. However, most information in which we remember is more complex than word-pair associations. Therefore, cognitive neuroscientists over recent years have started using visual media to answer scientific questions regarding human memory. Throughout this article, we will highlight a few studies in recent years that provide insights to memory, utilizing media from Oscar-nominated films to sports plays.

What are our natural thought processes during media-watching? – Movie Clips

Our initial study of interest incorporated three movie clips across three genres, each ten minutes long. Participants in the recent Wallace et al. (2025) study watched Citizenfour (a documentary), Little Miss Sunshine (a comedy), and 500 Days of Summer (a romance) before answering questions to test their comprehension and memory of the material. Half of these participants watched inside an fMRI scanner, to create images of the brain during this task, and the other half were at a desk, to have a more naturalistic movie-watching experience.

Wallace and colleagues had found that information in these movies was more memorable if multiple senses (e.g., the visuals and audio) were highly engaging, and this effect was even more prominent with high neural activity in the parts of the brain responsible for visual and auditory processing. Likely, when events are utilizing multiple senses, and therefore leading to a more comprehensive episodic experience, they are more memorable.

How does our brain separate memories into events? – NCAA Basketball

When given a continuous stream of information on a daily basis, humans have adapted to segment their memories into events. In a neuroimaging study by Antony et al. (2021), the researchers aimed to answer how this event segmentation happens directly (when a person claims a new event has occurred) and indirectly (when their brain indicates a new event has occurred). Participants viewed and recalled 9 NCAA basketball plays while collecting fMRI and eye tracking data, in order to measure surprise both in brain areas and in pupil dilation. Critically, the experimenters measured surprise in the clips by calculating the change in win probability from one possession to the next. They found that surprise predicted event boundaries, both discreetly (e.g., when participants claimed a new boundary had occurred) and autonomically (e.g., the increased neural patterns of brain areas associated with vision and memory; V1, precuneus, and medial prefrontal cortex). Surprise, along with its corresponding fMRI and eye tracking measures, was associated with better long-term memory of the events.

How do people remember and recall events? – Memento (2000)

Life is not always composed of linear narrative events. If someone were to describe a piece of gossip they had received from someone else, they can either describe it temporally (how they were told the story) or chronologically (how the gossip’s series of events occurred).

Memento (2000), if you have not already viewed it, implements a unique storytelling technique that directly distinguishes these two storytelling strategies. The original film is segmented into two storylines, the backwards plot-driven storyline (as seen in color; red in the figure) and the standard plot-driven storyline (as seen in black and white; blue in the figure). The film makes note to jump between the end of the story (in color, first scene) and the beginning of the storyline (black and white, second scene) until it meets at the direct climax at the end of the movie.

Antony et al. (2024) took this temporal versus chronological storytelling in order to answer the question “how are humans recalling events?” Participants either watched Memento in film-production order (bouncing between timelines) or chronological order (snipped and pieced together to be one continuous storyline). They then quantitatively assessed the various ways that we can remember a film’s sequence (causal, temporal, chronological, semantic) and then measured how these corresponded with participants’ recall.

Causal: describing the events in order of which cause-and-effect occurred

E.g., scene A caused scene C, therefore we will describe scene A and scene C subsequently

Temporal: describing the events in which they were shown in our visual timeline

E.g., scene A was shown before scene B and scene C, so we will describe scenes A, B, and C in that order

Chronological: describing the events in order of which they happen in the story’s timeline

E.g., scene A happens before scene C which happens before scene B, so we will describe the events in order of scene A, then C, then B

Semantic: describing the events based on how similar their storylines were to each other

E.g., scene B and C are showcasing the same idea of the main character describing his impairment, therefore we will describe them together

Through their paradigm, they found that causal and chronological strategies of recollecting the events better predicted the overall memory for the film compared to the other strategies. Even when participants were asked to describe the film in temporal or chronological order, the causal strategy persisted. These results indicate that when humans recall a series of events, they are oftentimes organizing their thoughts in a cause-and-effect manner.

What aspects of event memories are impacted with aging? – Curb Your Enthusiasm

Movies aren’t the only form of media that have been used in cognitive neuroscience studies. In a similar manner, sitcoms have been used to measure memories for events. The hit TV series Curb Your Enthusiasm was viewed in Delarazan et al. (2023) by young adults and older adults in order to assess human memory in aging. These two groups were then tested on two forms of memory, one that was narrative (using words) and one that was perceptual (using images). Older adults were worse with the perceptual memory task compared to younger adults. This was especially true when it came to recognizing events that were similar to the visuals they were presented with in the episode, but not actually included. Since there was no difference in the narrative form of recall, this study provides evidence for perceptual aspects of memories being more vulnerable to aging in human subjects.

The media-watching paradigm: future directions and implications

An interesting new direction neuroscientists can take, that seemingly intersects all of these prior findings, is further analyzing how emotions are playing into these effects. Wallace et al. (2025) and Antony et al. (2021) both indirectly highlight this idea. Through the multisensory experience enhancing memory in the former and surprise enhancing memory in the latter, we see that the viewer’s internal experience seems to influence whether they correctly recall events, even if they are not fully aware of it. Specifically looking deeper into how dopamine levels fluctuate during a media-watching experience, particularly throughout various types of media genres, can highlight how memories for events are enhanced or impaired automatically.

This new and upcoming paradigm has made for more enjoyable research visits for participants while also holding integrity for how individuals would naturally recall events. Therefore, through all of these various media-watching studies, we are closer to answering scientific questions surrounding human memory in a more effective and enjoyable way than ever before.

References

Antony, J., Lozano, A., Dhoat, P., Chen, J., & Bennion, K. (2024). Causal and Chronological Relationships Predict Memory Organization for Nonlinear Narratives. Journal of Cognitive Neuroscience, 36(11), 2368–2385. https://doi.org/10.1162/jocn_a_02216

Antony, J. W., Hartshorne, T. H., Pomeroy, K., Gureckis, T. M., Hasson, U., McDougle, S. D., & Norman, K. A. (2021). Behavioral, Physiological, and Neural Signatures of Surprise during Naturalistic Sports Viewing. Neuron, 109(2), 377-390.e7. https://doi.org/10.1016/j.neuron.2020.10.029

Delarazan, A. I., Ranganath, C., & Reagh, Z. M. (2023). Aging impacts memory for perceptual, but not narrative, event details. Learning & Memory, 30(2), 48–54. https://doi.org/10.1101/lm.053740.122

Wallace, R. S., Mckeown, B., Goodall-Halliwell, I., Chitiz, L., Forest, P., Karapanagiotidis, T., Mulholland, B., Turnbull, A., Vanderwal, T., Hardikar, S., Gonzalez Alam, T. R., Bernhardt, B. C., Wang, H.-T., Strawson, W., Milham, M., Xu, T., Margulies, D. S., Poerio, G. L., Jefferies, E., … Smallwood, J. (2025). Mapping patterns of thought onto brain activity during movie-watching. eLife, 13, RP97731. https://doi.org/10.7554/eLife.97731