Computação@UFCG Leads Brazil's Contributions to the HELM-Stanford Framework in Partnership with IBM

Collaboration between UFCG’s Computer Science department and IBM makes the university the top brazilian contributor to the HELM-Stanford evaluation framework in 2025.

HELM-Stanford is one of the world’s leading frameworks for evaluating language models, measuring accuracy, robustness, and fairness. Being the top Brazilian contributor — through the partnership between Computação@UFCG and IBM — highlights the national protagonism in developing fairer, safer, and more representative metrics for LLMs, especially in multilingual and culturally diverse contexts.

The partnership between Computação@UFCG and IBM resulted in 15 significant contributions to HELM-Stanford in 2025. These contributions include adding Portuguese-language benchmarks, fixing bugs, improving source code, and including new evaluation sets, expanding the framework’s linguistic diversity and robustness.

The project, coordinated by Professor João Brunet with participation from Professors Fábio Morais and Leandro Balby, features a multidisciplinary team dedicated to LLM evaluation. The team also includes one professor from IFPB, three graduate students, three undergraduate students, and a professional with software development experience. IBM, as a project partner, has also assigned professionals to work directly on the collaboration. Together, the group has made meaningful contributions to advancing HELM-Stanford, with a focus on including the Portuguese language and continuously improving the framework.