ModelBias

ModelBias.ai is an open research experiment by Tormund Gerhardsen that surfaces the default preferences of large language models when given minimal context. The project ran 100 simple, opinion-shaped prompts (superpower of choice, favourite holiday destination, capitalism vs. socialism, privacy vs. safety, best programming language, and dozens more) against 100 models from 17 major providers — Anthropic, OpenAI, Google, DeepSeek, Meta, xAI, Mistral, Qwen and others — asking each question three times per model through OpenRouter with the provider's default sampling parameters. The result is a browsable corpus of roughly 30,000 answers with a filterable UI: pick a prompt to see the aggregate distribution across the fleet, or pick a model to inspect its personal leanings.

A key methodological choice is provider-balancing: aggregates weight each provider equally rather than counting individual model variants, so a company that ships many checkpoints doesn't dominate the headline percentages. The methodology and about pages document the run parameters, and the complete raw output — including original text responses, not just the extracted category — is published on GitHub under a permissive licence so researchers can re-classify, re-analyse, or extend the dataset.

Typical use is exploratory rather than production: alignment researchers use it to spot suspicious cross-model consensus (e.g. ~80% of models pick 'teleportation' as their preferred superpower), prompt engineers use it to sanity-check that a supposedly neutral prompt is nudging the model, and journalists or educators use it as a concrete illustration of how homogenised RLHF outputs can be. It is explicitly positioned as an entertainment-first experiment, not a peer-reviewed benchmark, but the underlying data is real and reusable.