

OlympicArena
β Text source-checked Nov 21, 2026Source-checked against the vendor's site Β· rebuilt from the vendor's own pagesProfile updated Link checked How we rate
In short
OlympicArena is a free, open-source benchmark with 11,163 problems from 62 competitions to evaluate AI models on advanced scientific and logical reasoning tasks.
The vendor says that OlympicArena is a comprehensive and rigorously curated benchmark featuring a detailed, fine-grained evaluation mechanism designed to assess advanced AI capabilities across a broad spectrum of Olympic-level challenges. The vendor says that the benchmark includes a set of 11,163 problems from 62 distinct Olympic competitions, structured with 13 answer types. The vendor says that the benchmark spans seven core disciplines: mathematics, physics, chemistry, biology, geography, astronomy, and computer science, encompassing a total of 34 specialized branches.
The vendor says that the benchmark focuses on Olympic-level problems and covers 8 types of logical reasoning abilities and 5 types of visual reasoning abilities. The vendor says that the benchmark employs an instance-level leakage detection metric to validate its effectiveness. The vendor says that the benchmark conducts comprehensive evaluations from both the answer-level and process-level perspectives.
The leaderboard evaluates various models, including LLMs and LMMs, considering both closed- and open-source versions. The vendor says that the experiments utilize zero-shot prompts specifically tailored to each answer type and specifying output formats to facilitate answer extraction and rule-based matching.
The data for the seven disciplines is released on Hugging Face. The validation split includes answers for small-scale testing, while the answers for the test split are not publicly available. Users can submit their results to the platform for evaluation.
According to OlympicArena: The leaderboard evaluates various models, Including LLMs and LMMs, considering both closed- and open-source versions. Its pages list this strength: The dataset is released on Hugging Face. They also state this limitation: Users can only run evaluation locally on the validation set.
β The AI Tool Bible editorial team
Pros
- β The vendor says that the benchmark includes 11,163 problems from 62 distinct Olympic competitions with 13 answer types.
- β The vendor says that the benchmark spans seven core disciplines encompassing 34 specialized branches.
- β The vendor says that the benchmark provides evaluations from both answer-level and process-level perspectives.
- β The vendor says that the benchmark employs an instance-level leakage detection metric.
- β The dataset is released on Hugging Face.
Cons
- β οΈ Users can only run evaluation locally on the validation set.
Facts
- Pricing
- free - Free, open-source research benchmark (not source-checked)
- Open source
- Yes (not source-checked)
- GitHub stars
- 106 (checked Nov 13, 2026) (not source-checked)
- Last GitHub push
- Mar 6, 2025 (not source-checked)
- First commit
- Mar 2024 (not source-checked)
- Model used
- GPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, Qwen2-72B-Instruct (not source-checked)
- Hacker News
- 1 story Β· top 3 points Β· last Jun 15, 2025 Β· checked Nov 17, 2026
- Hugging Face
- 57 models Β· 8,590 downloads (30 d) Β· 748 likes for author GAIR Β· checked Nov 17, 2026
Where this tool sits
Use cases
Frequently asked
- How much does OlympicArena cost to use?
- OlympicArena is free and open-source. It is a research benchmark where users can access data on Hugging Face and submit results to the platform for evaluation without any documented fees.
- What types of problems are included in the benchmark?
- The benchmark includes 11,163 problems from 62 distinct Olympic competitions. It spans seven core disciplines: mathematics, physics, chemistry, biology, geography, astronomy, and computer science, covering 34 specialized branches.
- Which AI models are evaluated on the leaderboard?
- The leaderboard evaluates various Large Language Models (LLMs) and Large Multimodal Models (LMMs), including both closed- and open-source versions. Specific models mentioned include GPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, and Qwen2-72B-Instruct.
- How does the benchmark prevent data leakage?
- The benchmark employs an instance-level leakage detection metric to validate its effectiveness. This mechanism helps ensure the integrity of the evaluation by detecting potential data contamination during the testing process.
- Is the full dataset publicly available?
- Data for the seven disciplines is released on Hugging Face. The validation split includes answers for small-scale testing, but answers for the test split are not publicly available. Users must submit results to the platform for evaluation.