Blog

May 17, 2023

The people behind the technology

Artificial Intelligence (AI) is the process of synthesising, perceiving and inferring information through computer software, as opposed to via an organic brain. Chances are that you’re probably already interacting with it, if you use digital assistant tools like Siri, Google, Alexa, or ChatGPT. For the last few years, the Museum has been engaged in a groundbreaking project in collaboration with USC Shoah Foundation: using AI and sophisticated language processing technologies to preserve the humanity and life experiences of Holocaust survivors.

Photograph by Katharine Griffifths.

For an initiative based on artificial intelligence, humans have been at the core of every phase of the project right from the beginning: from the survivors who sat down to answer 1,000 questions, to the museum staff who work with this technology every day, and the visitors who have personal interactions with it.

First, over the summer of 2020-21, six of our Sydney-based Holocaust survivors: Olga Horak OAM, Yvonne Engelman OAM, Eddie Jaku OAM, Francine Lazarus, Paul Drexler, and Kuba Enoch OAM, came to the Museum to be interviewed for a full week each, about their experiences in the Holocaust, their lives, interests and hopes for the future, while being filmed within a specialised rig with 23 cameras.

The many hours of responses from those interviews have been transformed into an interactive biography – or digitised version – of each of the six survivors who took part. Now, each time you ask one of these digital survivors a question, the technology works behind-the-scenes to sort through around 1000 of the real survivor’s recorded answers, to find the right response. It will then play this footage back in real-time, in a way that feels a lot like you’re talking with a real person.

Of course, the interactive biographies didn’t immediately start answering questions perfectly. Each of them had to be trained, and in a process that took several months, each digital survivor needed to be asked close to 10,000 questions by different people with a range of voices, ways of phrasing and accents.

Special Projects Lead, Kaitlin Sim is an “AI trainer” of sorts. Kaitlin has been working behind the scenes to make sure that the interactive biographies give the right answers to each question they’re asked.

“I’ve been training the system to hear and recognise different accents and the different ways people ask questions,” Kaitlin says.

“For example, I might phrase the question as: When is your birthday? and you might ask: On what day were you born? These are the same questions, they just sound different. I make sure that each question asked to the survivors is matched with an appropriate response.”

What if a survivor doesn’t have an answer to your question? Kaitlin tells us, the digitised survivor will simply tell you that they don’t have an answer.

“Every answer given comes directly from the Holocaust survivor’s own words. There is absolutely no manipulation of their voice or ‘deep fake’ technology used in the survivor’s responses. What you see is real footage, spoken by a real person”.

“If a question is asked that we didn’t record during the filming process, such as, Who is the Prime Minister of Australia?, I match that question to a general response from the survivor, for example: ‘That’s a great question, I was never asked it during this interview’.”

Photograph by Katharine Griffifths.

With three of these interactive biographies on display in our Reverberations exhibition, and the other three nearly ready to be released for public exhibition, we are already seeing the impact of this project on visitors.

Senior Curator, Shannon Biederman, who leads this project at the Museum has found it incredibly rewarding to watch visitors interacting with the digital survivors, both in the testing focus groups, and now in the exhibition.

“The public is connecting more deeply with their testimony than we could have anticipated. For me, it’s been the little things, like how people ‘thank’ the digital survivors for sharing their stories and say ‘hi’ and ‘bye’. People treat the technology almost like it’s a real person,” she says.

While there’s no way to replace the powerful experience of meeting and engaging with a Holocaust survivor, the hope is that these interactive biographies will live on in perpetuity in digital form, so visitors of the future can continue to have meaningful interactions with survivors and their messages – even when the sad time comes that these incredible people are no longer able to share their stories in person.