Waipapa Taumata Rau, University of Auckland is New Zealand’s largest university, with about 40,000 students passing through one of five campuses each year. With such a high volume of students navigating the complex world of tertiary education, alongside a longstanding dedication to student experience, managing student enquiries and responding quickly with the most helpful and accurate information is a critical priority.
At the start of the COVID-19 pandemic, another priority was recognised as well: as the university quickly pivoted to comply with lockdown restrictions and courses shifted to virtual learning, the Student Contact Centre team realised that the answers to student queries were changing daily - sometimes hourly - and providing the most up-to-date information was critical.
Conversational AI lays the groundwork for a powerful solution that addresses all of these concerns. There are a number of unique ways to implement a conversational AI solution - from digital humans and voice bots to agent-assist systems and chatbots. All sounded appealing to the University, so we worked alongside them to identify key priorities and determine how to focus our efforts for maximum impact, establishing a system that would allow us to build a strong foundation moving forward.
The University partnered with ElementX to help identify, scope, and implement a solution that best fit their requirements, taking into account the complex nature of student enquiries, integrating with a variety of existing systems, and providing the best possible experience for both students and Student Contact Centre staff.
Scoping is one of the most important elements in a successful project - as ElementX is technology-agnostic, we begin with scoping sessions to determine what the best option is for a project, making no assumptions about how it might be built. We worked with the University - including staff members who were actively involved in answering student queries - to workshop the desired student experience and identify high-value improvements. We then created proof of concepts to test whether or not students would use these new features. This is a critical step for any successful AI implementation, as a complete understanding of how it will be used ensures there are no surprises.
While the university is ultimately planning to roll out a multichannel system that can interact with students in any way they choose, this agile process narrowed the focus to a list of critical priorities, from which we decided to move forward with a chatbot accessible through the University of Auckland website.
Watch the Chatbot in action:
The following is a list of criteria that we identified as being critical for the success of the project:
ElementX recommended implementing IBM Watson as the platform for the University’s chatbot. Watson is a question-answering system that uses Natural Language Understanding that learns over time as it is trained on more information. In a lot of ways, natural language understanding platforms can be quite similar and therefore difficult to choose between, but we recommended Watson in this instance because once it’s been set up, the system doesn’t require a developer to update information - when updates to the system need to be made due to changing COVID requirements or university policies, a team member from the Student Contact Centre can make these changes easily.
This feature was critical for the success of the program: staff members who, prior to the program’s implementation, had been answering phone calls and emails in the contact centre. These staff members were re-trained so that they could update the Watson system with answers to frequent questions, while still answering more complex queries. Now called content authors, they’re the perfect people to do this work as they’re already familiar with the questions being asked and the way these conversations usually flow.
Our conversational AI designers work alongside these team members to ensure they’re following best practice and to provide advice when needed, but since Watson doesn’t require a developer to code answers to questions, it’s a perfect opportunity to ensure that students get the best experience, and that these team members aren’t spending their days answering the same question again and again.
ElementX put together a team of experts to integrate with university systems and ensure that questions were answered succinctly and with the most applicable responses, and also to ensure a seamless integration with existing systems.
The University assistant currently helps answer a variety of universal queries such as “what are the requirements to attend the University of Auckland” and student-specific queries such as “When is my next exam?” so that Student Centre team members can resolve complex student queries more quickly. We use two metrics to monitor the success of the chatbot - Coverage (the percentage of queries that the chatbot has the answers to) and Containment (the percentage of people whose questions can be answered without being passed over to a human agent). Currently, both Coverage and Containment are averaging well above 90%. It also efficiently handles spikes in activity around expected times like exams and enrolment, and has been well received by the student population.
The University chose to stage the rollout of their chatbot with a minimum viable product (MVP) approach, which we commend as this makes it easier to measure the success of the program so that determinations can be made on which features to prioritise next. The project includes usage analytics so the team can tell what questions are being asked, what volume of queries the system receives, and how many of these it is able to manage on its own. These reports are shared with stakeholders regularly and used to plan improvements.
In the future, the team plans to roll out additional capabilities, expand to more channels, and continue to add more functionality to the tool. We look forward to a continued partnership with the University of Auckland team, supporting them in providing cutting-edge student experience using AI. You can visit the University of Auckland chat assistant live here.
Ming Cheuk: Chief Technology Officer: Ming led the scoping portion of this project, ensuring that all options were considered and that the critical priorities were captured and satisfied.
Dmitrii Goriunov: Senior Full Stack Developer: Dmitrii managed the integrations with a wide variety of existing systems for the University, making sure they worked seamlessly with the Watson chatbot.
George Qiao: Full Stack AI Engineer: George led the development process from concept to execution, translating Ming’s scope to reality. He worked with Dmitrii on the required integrations and also on the conversation design aspects of the chatbot alongside Kevin. His contributions on both sides are invaluable.
Kevin Adams: Conversational AI Designer: Kevin ensured that the chatbot was created to answer questions in language students could understand, offering feedback and assistance to the content authors.
Daniel Jimenez: Engineering Manager: Daniel managed scheduling and timing and was the main point of contact between the University of Auckland team and ElementX. He ensured that the project was delivered as expected within planned deadlines.
Tessa Phillips: Senior AI Engineer: Tessa worked on the analytics systems to help the UoA team understand how students are using the chatbot and how it is performing so that improvements can be made and new features can be prioritised.
