The Foundation’s first event of 2025 was co-hosted with H2O.ai, featuring Agus Sudjianto, a renowned expert in Model Risk Management (MRM) for AI models.
Over 40 industry practitioners and experts gathered at this closed-door roundtable, bringing together an impressive collective experience of over 500 years in model building and testing.
Key takeaways from the discussion:
- Embracing Acceptable Risk: The focus of MRM and AI Governance should be on helping organisations “take” acceptable AI risks rather than eliminating risk entirely.
- Use Case Level Testing: For companies implementing Generative AI, validation is the most useful and practical when applied to the individual use case with a defined purpose and scope, rather than the underlying foundation models.
- Evaluating the Evaluators: Trusting LLM evaluations blindly is unwise. Evaluations need as much careful design and calibration, as the application itself.
- Human-AI Alignment in Testing: Automated testing frameworks should incorporate human calibration, explainable assessments, and probabilistic methods to align machine-generated evaluations with human judgments.
Throughout the year, the Foundation will continue to bring together the best of Singapore and the world on AI testing across various industries. Look forward to our upcoming efforts in incorporating emerging best practices into our open-source Project Moonshot library, and advancing the field of AI testing and validation.
