"How do we make sense of the complex visual world around us?" That opening question served as the launching pad for an unprecedented two-night Academy event, "Movies in Your Brain: The Science of Cinematic Perception," which brought together filmmakers and cognitive scientists to explore the way viewers process images, events and stories experienced on the silver screen. 

Bill Kroyer, an Academy Governor representing the Short Films and Feature Animation Branch, welcomed the audience at both evenings of the Sci-Tech Council event, held on July 29 and 30 at the Academy's Linwood Dunn Theater. The host throughout was Tim J. Smith, a senior lecturer in psychological sciences at Birkbeck, University of London, who specializes in the study of visual cognition. 

First up on Tuesday night was Uri Hasson, an associate professor of psychology and neurosciences at Princeton University, who explained how brain responses to films could be measured dynamically using functional magnetic resonance imaging (fMRI). The neurons in the brain require energy to fire up, and different areas of the brain process different kinds of information. Hasson presented a map of neural responses to clips from The Good, the Bad and the Ugly and Dog Day Afternoon, with the latter engaging 63 to 73 percent of the brain as well as provoking outward physical reactions like laughter.

Joining Hasson and Smith for a panel discussion were Jeffrey M. Zacks, a professor of psychology and radiology at Washington University in St. Louis, and three-time Oscar winner Walter Murch, a film editor and sound designer whose credits include such films as Apocalypse Now, The English Patient and The Godfather Part II. Murch explained how his edit of a tricky scene involving Gene Hackman in The Conversation (his first feature as a film editor) was inspired by a comment that legendary filmmaker John Huston had made during an interview: Huston described blinking as a physical manifestation of a psychological “cut.” Murch’s observation motivated Smith to run a live experiment during the event and capture audience responses to the clip using an infrared camera. When the infrared footage was played back alongside the clip the audience was able to monitor their own reactions and see whether their blinks coincided with Gene Hackman’s.

The trick of getting people’s brains to “tick together” while watching a film has been mastered and dissected over the decades, a history explored by James Cutting, chair of the department of psychology at Cornell University. Cutting has become an expert in perceptual and cognitive processing and their relationship to film editing, frame rates, projection, and scene and narrative structure. Here he examined the average shot duration in films, which has become consistently shorter since the art form’s inception. Although popular opinion holds music videos, the Internet and ADHD as drivers of this trend, Cutting showed that the downward trajectory has been consistent and uninterrupted since the end of the silent era.

A clip from the Monaco racing scene in Iron Man 2 followed, and Jon Favreau, the film’s director, and Talma Hendler, founder and director of the Functional Brain Center at Tel Aviv Sourasky Medical Center, joined the other guests onstage. Smith introduced eye-tracking footage captured from ten audience members earlier in the Dunn lobby, which demonstrated the remarkable consistency of where the ten focused their attention as their eyes took in the action.

The restaurant confrontation scene in Favreau’s Chef was screened next as an example of how to make an audience “squirm,” with even simple story elements like an awkward conversation and the handling of food drawing visceral reactions from viewers. Hendler added that our own empathy system is designed to respond to information being processed in the brain, while Murch and Favreau examined the tension that mounts when a scene continues without cutting away for an extended period of time.

If the first night was about how to present a story and achieve a desired response, the second evening was about brain experiments and their results, which can be applied to both filmmaking and psychology. Director, producer and writer Darren Aronofsky and his frequent collaborator Ari Handel (who co-wrote Noah and worked on The Fountain, The Wrestler and Black Swan) joined the scientists for a broad exploration of how different parts of the brain respond to such factors as frame rate, editing and emotional content.

Zacks illustrated how the brain’s ability to generate short- and long-term memories carries over into films. A clip from Mon Oncle was shown, and afterwards the audience was asked which of two objects had appeared on the screen. Only a small minority could answer correctly (it was a chair). Hendler then focused on how movies can move us by activating different parts of the brain. When scenes from the films Stepmom, Sophie’s Choice and The Ring Two were shown to subjects during an MRI scan, they produced complex responses deep within the brain and generated activity beyond normal cognitive levels. Hendler described how such studies can help with new therapeutic approaches in psychology, where an emotional reaction (which she defined as an oscillation between an individual’s socialization and survival modes) can be very tricky to pin down.

Clips from Aronofsky’s films were presented to illustrate these principles as well, including the opening and nightclub sequences from Black Swan and the creation-story sequence from Noah. Cutting amused the filmmakers by showing how some of their more intense, aggressively edited scenes could push viewers “to the edge” of what was psychologically tolerable, comparing them to other films like Cloverfield and The Bourne Ultimatum, which often drew verbal reactions from audiences who found them too much to handle. Aronofsky was especially delighted to hear that certain Black Swan sequences produced results that Hendler compared to a schizophrenia-like state, with the cognitive and emotional centers of the brain operating dramatically in and out of synch.

The two-night event closed with Andy Maltz, the Managing Director of the Academy’s Science and Technology Council, taking on a hotly-debated topic in the film community at the moment: frame rates. The visually denser 48-frames-per-second versions of Peter Jackson’s two recent The Hobbit films and the push for higher frame rates from directors like James Cameron have produced a wide variety of opinions in recent years, and a new project from the Tech Council offered a clear, dramatic demonstration of the multiple possibilities. A single-shot, silent short entitled “The Affair” was filmed with identical camera moves and identical performances more than 30 times at different frame rates: 24fps (the cinematic standard), 48fps, 60fps, 120fps and several speeds in between. Four frame rates (24, 48, 60 and 120fps) were presented. The leap between each was both fascinating and divisive, with a show of hands revealing how much further we all have to go before deciding what the experience of watching and absorbing a movie will mean in the future.