During the Aotearoa AI Summit, we made world history with Sophie, our digital human collaboratively developed with UneeQ. Our vision for Sophie was groundbreaking: she was designed to introduce unscripted insights into panel discussions, marking a significant departure from the scripted conversations typically associated with digital humans in this use-case. Her unscripted contributions provided us regular humans with the opportunity to interact with AI in a completely natural manner - not only did this novel approach allow for a seamless interaction, but it also brought the invaluable insights of a broadly trained LLM into the conversation. This breakthrough in AI-human interaction set a new standard, combining the best of both worlds to create an engaging and informative experience for all involved.
We'll walk you through the step-by-step process of bringing Sophie to life and the technologies that make it all possible.
β
1. Audio Integration: Gearing Up for Immersive Conversations
The process begins with audio integration. Sophie is connected to the AV system, allowing her to receive audio input directly from the microphones the panelists used and play audio back into the room speakers. This integration ensures that they can listen and respond just like a real person would.
β
2. Real-time Transcription: Decoding the Language of Sound
One of the primary challenges in creating a digital human is accurately transcribing the real-time audio input. Imagine multiple speakers talking simultaneously in a single audio stream β deciphering this linguistic puzzle is no easy feat. Clever algorithms work tirelessly to separate the voices and create coherent transcriptions.
β
3. EPIC Integration: Guided by Human Expertise
No digital human creation process is complete without human involvement. The transcribed audio data is sent to EPIC (ElementX Panel Intelligence Controller), which acts as an interface for human-in-the-loop review. Here, the operator meticulously reviews the transcribed content in real-time, ensuring its accuracy and relevance before it is sent to Sophie for a thoughtful and contextually appropriate response.
β
4. Large Language Models: The Power of Artificial Intelligence
Modern large language models are the backbone of this process. These AI models which also power ChatGPT, enable the digital human to understand and interpret the transcribed content, forming the basis for their responses. The model has been tuned specifically for the panelist use-case to ensure appropriate and insightful responses.Β
β
5. Speech Synthesis: Breathing Life into Words
Once Sophie has processed the transcribed content and generated a suitable response, the next step is to vocalize that answer back to the audience. This is where speech synthesis comes into play. This phase is crucial for the project as it brings together all previous steps, making Sophie truly interactive and human-like.
β
6. Live Demo: Witnessing the Future in Real-Time
Once the digital human interface is up and running smoothly, it's time to showcase its incredible capabilities. A panel discussion βLarge Language Models in Aotearoaβ was the perfect venue to share the integration of AI in human-computer interactions. The panel she presented in brought together experts from various sectors, academic research, education, and public consultations.Β During the panel, Sophie interjected her unscripted thoughts, adding a fresh perspective to the dialogue and igniting fascinating discussions that challenged conventional thinking.
β
Sophie's introduction into the world of AI has already begun to change the way we interact with LLMs and AI systems. Her unscripted interjections during the panel brought new insights to the discussion and challenged conventional thinking. This revolutionary approach to human-computer interaction demonstrates the value that AI and LLMs can add in synthesising information and contributing to more effective decision-making. As we continue to evolve our understanding of AI, embracing new technologies like Sophie will pave the way for more natural and productive interactions between humans and machines.
This incredible fusion of technology and humanity opens up limitless possibilities, and while Sophie's role as a panelist is certainly impressive, it's just the beginning of what Digital Humans can do. With their empathetic faces, they can provide training, serve as information sources in complex knowledge bases, and help users of all backgrounds navigate intricate online tasks. At ElementX, we've partnered closely with UneeQ Digital Humans to deliver a wide range of innovative solutions. Whether you're looking to invite Sophie the Panelist to your next conference or embark on your very own world-first digital human project, our team is here to assist. Let's revolutionise the way we interact with technology together!
Want to stay in the loop? Subscribe to get updates from us.
