Skip to main content
πŸ“– The AI Tool Bible
OlympicArena preview image
OlympicArena logo

OlympicArena

βœ“ Text source-checked Nov 21, 2026

Source-checked against the vendor's site Β· rebuilt from the vendor's own pagesProfile updated Link checked How we rate

FreeΒ· Free, open-source research benchmark(not source-checked)EvaluationGPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, Qwen2-72B-Instruct (not source-checked)7.0 / 10

In short

OlympicArena is a free, open-source benchmark with 11,163 problems from 62 competitions to evaluate AI models on advanced scientific and logical reasoning tasks.

The vendor says that OlympicArena is a comprehensive and rigorously curated benchmark featuring a detailed, fine-grained evaluation mechanism designed to assess advanced AI capabilities across a broad spectrum of Olympic-level challenges. The vendor says that the benchmark includes a set of 11,163 problems from 62 distinct Olympic competitions, structured with 13 answer types. The vendor says that the benchmark spans seven core disciplines: mathematics, physics, chemistry, biology, geography, astronomy, and computer science, encompassing a total of 34 specialized branches.

The vendor says that the benchmark focuses on Olympic-level problems and covers 8 types of logical reasoning abilities and 5 types of visual reasoning abilities. The vendor says that the benchmark employs an instance-level leakage detection metric to validate its effectiveness. The vendor says that the benchmark conducts comprehensive evaluations from both the answer-level and process-level perspectives.

The leaderboard evaluates various models, including LLMs and LMMs, considering both closed- and open-source versions. The vendor says that the experiments utilize zero-shot prompts specifically tailored to each answer type and specifying output formats to facilitate answer extraction and rule-based matching.

The data for the seven disciplines is released on Hugging Face. The validation split includes answers for small-scale testing, while the answers for the test split are not publicly available. Users can submit their results to the platform for evaluation.

From the vendor's pages

According to OlympicArena: The leaderboard evaluates various models, Including LLMs and LMMs, considering both closed- and open-source versions. Its pages list this strength: The dataset is released on Hugging Face. They also state this limitation: Users can only run evaluation locally on the validation set.

β€” The AI Tool Bible editorial team

Pros

  • βœ… The vendor says that the benchmark includes 11,163 problems from 62 distinct Olympic competitions with 13 answer types.
  • βœ… The vendor says that the benchmark spans seven core disciplines encompassing 34 specialized branches.
  • βœ… The vendor says that the benchmark provides evaluations from both answer-level and process-level perspectives.
  • βœ… The vendor says that the benchmark employs an instance-level leakage detection metric.
  • βœ… The dataset is released on Hugging Face.

Cons

  • ⚠️ Users can only run evaluation locally on the validation set.

Facts

Pricing
free - Free, open-source research benchmark (not source-checked)
Open source
Yes (not source-checked)
GitHub stars
106 (checked Nov 13, 2026) (not source-checked)
Last GitHub push
Mar 6, 2025 (not source-checked)
First commit
Mar 2024 (not source-checked)
Model used
GPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, Qwen2-72B-Instruct (not source-checked)
Hacker News
1 story Β· top 3 points Β· last Jun 15, 2025 Β· checked Nov 17, 2026
Hugging Face
57 models Β· 8,590 downloads (30 d) Β· 748 likes for author GAIR Β· checked Nov 17, 2026

Where this tool sits

Use cases

process-level evaluationleakage detection

Frequently asked

How much does OlympicArena cost to use?
OlympicArena is free and open-source. It is a research benchmark where users can access data on Hugging Face and submit results to the platform for evaluation without any documented fees.
What types of problems are included in the benchmark?
The benchmark includes 11,163 problems from 62 distinct Olympic competitions. It spans seven core disciplines: mathematics, physics, chemistry, biology, geography, astronomy, and computer science, covering 34 specialized branches.
Which AI models are evaluated on the leaderboard?
The leaderboard evaluates various Large Language Models (LLMs) and Large Multimodal Models (LMMs), including both closed- and open-source versions. Specific models mentioned include GPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, and Qwen2-72B-Instruct.
How does the benchmark prevent data leakage?
The benchmark employs an instance-level leakage detection metric to validate its effectiveness. This mechanism helps ensure the integrity of the evaluation by detecting potential data contamination during the testing process.
Is the full dataset publicly available?
Data for the seven disciplines is released on Hugging Face. The validation split includes answers for small-scale testing, but answers for the test split are not publicly available. Users must submit results to the platform for evaluation.

Compare OlympicArena with similar tools

All alternatives β†’

Explore related

Reviews