بازآفرینی یک عشق ۷۰ ساله فریم به فریم با کمک هوش مصنوعی

Recreating a 70-year love story frame by frame

“Love, Rendered,” a new documentary short, explores memory loss and the power of storytelling — and it showcases how technology can help rekindle memories that have begun to fade. The film follows Burt and Ethelle Shatz, a couple married for over 70 years, as they navigate Burt’s cognitive decline. Among his fading memories is one of their most precious: the day they met at a student co-op in Cleveland. Because it wasn’t photographed or filmed, the day existed only in their minds.

When I stepped in as the film’s technical lead, I knew the project would be as emotionally demanding as it was technically complex. Memory loss is personal to me. My grandfather suffered a stroke and memory loss before he passed. The last time I visited him, I was in my thirties. He was convinced I was still in college and was so happy I was graduating. That bittersweet memory never left me.

The experience made me want to see if this technology could help me connect more deeply with my family through the same medium. When this project began, I asked my father for old family photos. I tested our image restoration on photographs from when my parents met and used our video models to animate them. Watching my parents move as twenty-somethings really drove home how these tools could help keep what matters most from slipping away.

“Love, Rendered” was directed by Academy Award–nominated filmmaker Liz Garbus, who produced it alongside Dan Cogan and Darren Aronofsky. It was created in collaboration between Google DeepMind and Primordial Soup, Aronofsky’s creative venture.

Exploring the mystery of memories

Liz and Darren both came to the film curious about the resilience of memory. While directing the documentary “Coma,” Liz saw fMRI scans light up when patients in minimally conscious states heard familiar voices or were shown images of loved ones. Years later, Darren encountered footage of a former ballerina with Alzheimer’s who, upon hearing “Swan Lake,” instinctively danced the choreography from her wheelchair.

These touchstones led the creative team to reminiscence therapy, the clinical practice of using sensory cues like songs, family stories, and old photographs to stimulate memories, spark conversations, and rekindle emotional connections. But what happens when a memory isn’t attached to such cues?

To recreate Burt and Ethelle's unrecorded past, the filmmakers partnered with my team. Sitting alongside us, Ethelle became an active co-creator, correcting the curve of a staircase or the shape of a shoe heel.

Capturing human essence with AI

We worked alongside the team at Primordial Soup, leveraging emerging AI models as an art medium and toolkit, guiding the technology with human direction every step of the way.

Bridging the gaps between the missing details required a two-part technical approach designed to preserve emotional truth.

- Image restoration: The team used generative models to restore black-and-white photos of Burt and Ethelle from their youth. The restored photographs helped to ensure that subsequent recreations remained faithful to their subjects.

- Pose and performance control: Engineers used our performance capture models to map Burt and Ethelle’s present-day micro-mannerisms — the specific tilt of Burt’s head, a brief hesitation in his speech pattern, the subtle crinkle around his eyes. Mapping those traits onto their younger likenesses brought the recreation to life.

Combining these two mediums allowed us to intertwine Burt and Ethelle’s past and their present. In doing so, we generated a “memory” that they told us felt authentic.

See my colleague Jess Gallegos explain the workflow in more detail.

As Darren noted during production, a tool — like a paintbrush or a hammer — does nothing until it’s guided by human hands. In “Love, Rendered,” machine learning was the tool we used to guide Burt and Ethelle back through time, helping them hold onto a vanishing moment.

You can watch the full film here to experience their love story, frame by frame.

Preserving your family’s story

Whether you’re looking to connect with a grandparent or preserve your family’s core memories, you can restore old photos in the Gemini app. Upload an image of the photo and ask Gemini: “Can you restore and colorize this photo? Preserve the appearance, expression, and pose of the people.”