The 1st International Workshop on AI-for-Science Benchmarking (AI4S-Bench 2026)

Introduction

AI-for-Science (AI4S) has emerged as a rapidly growing research area, driven by the increasing availability of large-scale scientific datasets and rapid advances in AI technologies. Recent advances in scientific foundation models, large language models, and AI agents are increasingly supporting scientific discovery across diverse domains through knowledge representation, data analysis, hypothesis generation, and workflow automation. AI methods are now being widely applied to a broad range of scientific tasks, including protein structure prediction, single-cell genomics, materials discovery, drug discovery, climate modeling, and Earth system science. Despite these advances, the effectiveness of AI4S methods critically depends on the quality and AI-readiness of scientific datasets, as well as the performance, robustness, reliability, and trustworthiness of AI models and agent-based systems. Therefore, systematic evaluation of data, models, agents, and applications has become essential for ensuring reproducible, transparent, and trustworthy scientific discoveries. To this end, AI4S-Bench aims to bring together researchers and practitioners from data mining, machine learning, and diverse scientific domains to discuss critical challenges and emerging opportunities in AI4S benchmarking. The workshop will focus on the development of standardized datasets, evaluation protocols, and metrics for assessing AI4S data, models, agents, and applications. It also seeks to encourage reproducibility, transparency, and FAIR principles across different scientific domains. By providing a focused forum for discussion and collaboration, AI4S-Bench aims to foster the development of reliable and broadly applicable benchmarking practices for AI-driven scientific research.

Topics of Interest

AI4S-Bench aims to bring together leading researchers and practitioners to exchange and share their latest research, experiences, and application results on benchmarking AI for Science. The workshop will provide an interdisciplinary forum to discuss recent advances, emerging trends, innovative applications, and real-world challenges in the systematic evaluation of scientific datasets, AI models, AI agents, and AI4S applications, as well as corresponding benchmarking methodologies and best practices.

Topics of interest include, but are not limited to:

Scientific Data Evaluation

Methods for assessing data quality and AI-readiness
Standardized metrics and protocols for scientific datasets
Design, curation, and application of benchmark datasets

Evaluation of AI Models and Agents for Science

Task-specific evaluation of AI models for scientific problems
Assessment of the performance, reliability, and robustness of large language models and AI agents on scientific tasks
Evaluation of the interpretability, safety, and transparency of AI models and agent-based systems in scientific research
Development of standardized benchmarks and evaluation frameworks for comparing AI approaches across scientific domains

Benchmarking AI4S Applications for Scientific Discovery

Evaluation of AI-assisted workflows, hypothesis generation, and knowledge discovery
Assessment of AI-driven decision support and experimental design in scientific research

Standards and Best Practices

Community-wide guidelines for reproducible benchmarking of datasets, models, and AI4S methods
Promotion of transparency, FAIR principles, and consistent reporting of AI performance in science

Call for Papers

We invite submissions of regular research papers, vision papers, and extended abstracts. Regular research papers should be no more than 8 pages, while vision papers and extended abstracts should be 4–6 pages, including all content and references. All submissions must be in PDF format and formatted according to the latest IEEE Conference Proceedings Template. Submissions will be evaluated based on relevance, technical quality, potential impact, clarity, reproducibility, and contribution to AI-for-Science benchmarking. This year, we will also establish Best Paper Awards to recognize outstanding submissions. By ICDM tradition, all accepted workshop papers will be published in the dedicated ICDMW proceedings by the IEEE Computer Society Press. The submission link will be announced soon.