From Beta Testing to Success: How SIA Takes Responsible AI Implementation to the Next Level with AI Verify

Singapore Airlines (SIA), the national carrier of Singapore, is one of the world’s leading airlines dedicated to providing air transportation services of the highest quality. Innovation is at the heart of SIA’s brand promise, enabling them to offer world-leading products and services while creating new business opportunities. To enhance customer experience and operational efficiency, the airline developed AI-powered solutions for customer-facing and operational applications. One such solution is Joey.

SIA piloted AI Verify testing framework on Joey:

  • AI model: Question-Answering model
  • Use Case: Joey - HR Chatbot

Introducing Joey, SIA’s first chatbot developed in-house for HR-related queries. Launched in April 2021, Joey uses Smart Search technology to provide fast and accurate responses, improving efficiency, accuracy, and accessibility in HR processes by:

  • Providing staff with access to HR-related information to self-serve their queries anytime, anywhere in the world. This improves employee satisfaction
  • Freeing up time for HR staff to focus on more value-added tasks.

Early testers in beta testing

SIA was among the early users of AI Verify, participating in its beta testing even before the Minimum Viable Product was launched. Recognising the importance of responsible AI practices, SIA saw AI Verify as a strong foundation to kickstart its AI testing and aimed to:

  • Foster the development and use of AI systems that are fair, transparent, and accountable, while ensuring their practices adhere to AI ethical guidelines
  • Strengthen digital and data trust in AI by ensuring that AI systems are developed and deployed to maximise long-term value for stakeholders, considering the social, environmental, and economic impact of AI. This is particularly important due to potential risks such as algorithmic bias, personal data breaches, and safety issues that may undermine SIA’s customers’ confidence in AI systems
  • Demonstrate how trustworthy AI can mitigate business risks, while ensuring compliance with data protection regulations to protect sensitive information of SIA’s staff and customers

When SIA first started its journey in testing AI Verify, it was not without challenges as the organisation had to spend a substantial amount of time to:

At the beta testing stage, AI Verify only supported limited AI model types. SIA used a state-of-the-art Natural Language Processing model to retrieve and rank the relevant HR documents and FAQs to answer user queries. However, this was not part of AI Verify’s supported algorithms.

  • Familiarise itself the way AI Verify organised the AI ethics principles and its respective testing criteria
  • Understand how these principles could be applied to technical tests and process checks
  • Set up an adequate environment and document all findings and results from the testing process

Despite the limitations in technical testing tools, it did not stop SIA from pursuing further tests. SIA reviewed and tested its AI initiatives using the testing framework and process checks where possible. While the testing process was rigorous and intensive, SIA felt it was critical to streamline the implementation of responsible AI with its AI development process. SIA made several improvements, including: 

  • Recalibrating its AI development process to introduce relevant principle testing at different stages
  • Enhancing the model card and introducing a visual aid to quantify how well an AI model could be explained responsibly
  • Continuing to drive awareness and education on responsible AI to all staff

SIA believes that the AI Verify toolkit is a great starting point for organisations as it provides a robust testing framework checklist to conduct self-assessments on AI systems. This is key in contributing to the development of international standards and industry benchmarks.

Back to User Stories

Share your experience using AI Verify today

Submit