0

29.10.2024

Article Hero Image

AI is taking root in many industries, with the market projected to reach a staggering $407 billion by 2027. One thing remains certain: only trustworthy AI systems should be brought to market.

The rise of responsible AI (RAI) frameworks and toolkits in recent years is a positive step in this direction. However, despite these building blocks and recommendations, the overall landscape remains fragmented.

Enterprises are now more than ever seeking direction and clarity on which frameworks and guidelines to adopt while ensuring they remain compliant with evolving regulations and standards. For many, the challenge extends beyond selecting the right tools; it also lies in implementing them effectively amidst a landscape still finding its footing in AI governance.

To address this need, Singapore’s Infocomm Media Development Authority (IMDA) has established the AI Verify Foundation. The Foundation galvanises global efforts around AI governance testing and evaluation, serving as a neutral space for sharing of ideas, and to encourage the use of cooperative development of open-source testing toolkits like AI Verify and Project Moonshot.

Open-source testing toolkits

  • AI Verify is an AI governance testing framework and software toolkit that assesses the performance of AI systems against a set of internationally recognised principles, using standardised tests. It aligns with global AI governance frameworks such as those from the European Union, OECD, and Singapore.

    This integrated toolkit operates within enterprise environments and is capable of performing technical tests on common supervised learning models, including classification and regression, across most tabular and image datasets.

  • An extension of the Foundation’s efforts, Project Moonshot was released as an open beta in May 2024 as one of the world’s first large language model (LLM) evaluation toolkits. This open-source tool combines benchmarking, red teaming, and baseline testing — marking a major milestone in establishing consensus around safety benchmarks and addressing the “last mile” confidence conundrum in AI deployment.

    The Foundation’s partnership with MLCommons unites two leading AI safety communities to strengthen safety benchmarks, ensuring they are relevant and reflective of Asia’s unique culture, heritage, and values.

As a founding and premier member, Resaro actively supports the Foundation as a third-party tester, with whom we share the conviction that a global open-source community can be a meaningful enabler of trustworthy AI adoption.

“We see open-source testing tools as a key part of levelling the playing field for AI companies to innovate responsibly and compliantly," says April Chin, Managing Partner and CEO (Singapore) of Resaro. "As a premier member of the AI Verify Foundation, we are committed to enhancing open-source tools that are globally interoperable, building public trust in AI and laying the groundwork for an AI-led future.”

At Resaro, we are dedicated to supporting enterprises in making sure their AI systems are performing as intended, compliant with emerging regulations, and trusted by customers and citizens at large.

We are actively supporting the Foundation through technical efforts in updating the AI Verify Toolkit 2.0 by contributing new testing algorithms, simplifying the installation process, and enhancing the overall user experience — ensuring the toolkit remains adaptable to developers’ needs.

We are also deeply honoured to support Project Moonshot as a design partner by conducting alpha testing and driving development in line with industry needs. The modular design approach also allows us, as third-party testers, to introduce use-case specific benchmarks and custom metrics to evaluate performance and safety in real-world applications.

“As a premier member of the AI Verify Foundation, Resaro has been a fantastic source of insights around AI assurance and testing," says Shameek Kundu, Executive Director of the AI Verify Foundation. "Resaro has also played an important role in advancing open-source AI testing frameworks, ensuring developers and enterprises have access to robust tools for responsible AI governance. We look forward to continuing our partnership in driving the global AI community towards building safer and more trustworthy AI systems.”

Assurance of AI systems will be a critical part of enterprises’ AI adoption journey. Just as tandem skydivers rely on thorough safety checks, harnesses, and parachutes to ensure their jumps are executed well with maximum safety, Resaro likewise serves as the trusted ‘safety harness’ for AI systems, ensuring they achieve peak performance while remaining safe, secure, and compliant.

Together with the Foundation, we are committed to growing a RAI community that spans jurisdictions, languages, and cultures — making AI safer for everyone.


Article Image 1

Image courtesy of the AI Verify Foundation, in celebration of the Foundation’s first anniversary at ATxSG (2024). For more information about the Foundation, visit https://aiverifyfoundation.sg/