HAA Lab Human-Agent Ally A HAA Lab Project

EMNLP 2026

MM-Compliance

A SASB-aligned multilingual multimodal benchmark for sustainability compliance auditing, with page-level evidence and disclosure-completeness annotations.

  • 6 Languages
  • 7,174 PDF Pages
  • 366 Metric Types
  • SASB-aligned
Access the dataset View the code
Six languages. Page-level evidence. Completeness labels.

Why MM-Compliance

Compliance evidence does not live in one language—or one modality.

MM-Compliance aligns industry-specific SASB metrics with real disclosure pages from corporate ESG and sustainability reports. Its completeness annotations distinguish fully compliant evidence from partial or superficial mentions.

01 / LANGUAGE 6

Languages

English, French, Japanese, Chinese, Korean, and Thai—covering linguistic and regional reporting variation.

02 / DOCUMENTS 7,174

PDF pages

Original page layouts preserve text, tables, charts, infographics, and other visual structures important to ESG auditing.

03 / FRAMEWORK 366

SASB metric types

Page-level evidence is aligned with industry-specific metrics and labeled for complete versus incomplete disclosure.

Evaluation benchmark

From finding the page to verifying the disclosure.

The benchmark separates two stages of a practical auditing workflow: locating relevant evidence in a long report, then deciding whether the evidence fully satisfies a reporting requirement.

Task 01

Page-level Compliance Judgment

Given a SASB metric and one report page, predict yes, yes but not complete, or no.

Task 02

Metric-to-Page Evidence Retrieval

Given a metric description and a multi-page ESG report, rank the pages that contain the supporting evidence.

Key finding

Current models can often shortlist relevant evidence pages—reaching an average Hit@20 of 0.882—but still struggle to distinguish partial disclosures from complete compliance. Requirement-aware completeness verification remains the central bottleneck.

Open research resources

Use the dataset. Follow the research.

The dataset and official code repository are available now. The final paper link will be added here when it is released.

DataAvailable

Dataset

Access the public MM-Compliance files from Google Drive.

Open dataset
PublicationEMNLP ’26

Paper

Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing.

Paper link coming soon

From dataset to shared task

Powering RegCom at NTCIR-19

MM-Compliance provides the dataset for RegCom, our NTCIR-19 shared task on regulatory compliance. RegCom brings this benchmark to the research community for shared evaluation.

Visit the RegCom website
NTCIR-19RegComShared task

Publication

MM-Compliance: A Multilingual Multimodal Dataset for Sustainability Compliance Auditing

Rui He, Jianhua Yang, Tomoki Kera, Atsushi Keyaki, Anaïs Lhuissier, Juyeon Kang, Min-Yuh Day, Guan-Bo Yang, Hsin-Hsi Chen, Dittaya Wanvarie, Yongjae Lee, Suhwan Park, Yuriko Nakao, Chung-Chi Chen, and Yohei Seki.

Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP ’26).