LIVE

Social Simulation Arena

Social Simulation vs. The Real Future

A live benchmark for social simulation: models forecast the next public-opinion release before it is published. Locked, hashed, scored in public. No one picks the test set, because the test set does not exist yet.

social-simulation-arena.comSeason 0 · live since August 11, 2026

Why

Social simulation is easier to build than to trust

The work is real

Simile, Aaru and others sell simulated publics to enterprises and campaigns.

Gallup is testing simulated respondents against its own national panel.

New agents, personas, and synthetic publics ship every month.

The proof is missing

Every accuracy claim is graded by the team that makes it, in industry and in research alike.

So skeptics reject the whole field at once: Silver Bulletin bans AI polls outright. AAPOR and ESOMAR are writing rules.

Published benchmarks grade the past, where the answers already sit in training data.

Today a good simulator cannot prove it is good. Trust needs a source no one controls, not even us: a public test against the real future.

The consortium

A neutral social simulation consortium

An interdisciplinary team from 14 institutions: computer scientists, AI engineers, cognitive scientists, social scientists, political scientists, and economists. No single person decides the rules.

MIT

Stanford

Harvard

CMU

UC Berkeley

UChicago

UWashington

Johns Hopkins

Northeastern

UC San Diego

Brown

UBC

NUS

Microsoft

The idea

A live benchmark, graded by the real future

Left of the green line, every number is already published. Entries are hashed and locked 48 hours before each release; past the line the answer does not exist yet, and the release itself grades everyone in public.

How

A live bench built on four recurring data sources

Each round: forecast the tracker’s next release. Entries lock 48 hours before the number comes out; the release itself grades everyone; then the next round starts.

Entering a round

Push. One JSON file per round, by pull request or the site’s one-click template, any time before lock (release − 48 h). CI checks the schema and the deadline and prints your sha256. Late means void, enforced in code.

Or API. Approve the question template and hand us a key; before each lock we call your endpoint and file the answer. Point-only APIs are fine: we sample a few times and build the distribution.

The format. A distribution, not a point: mean and sd per value, or quantiles. Confidence is scored, so bluffing costs. One file covers the topline and all 16 cells.

forecasts/<round>/<you>.json
{ "topline": { "mean": 39.0, "sd": 2.0 },
  "cells": {
    "Democrat":   { "mean": 5,  "sd": 1.5 },
    "Republican": { "mean": 87, "sd": 2.0 },
    ...16 cells: party, age, race,
       gender, education } }

Entrants

Built for the teams that simulate society

Season 0 already grades the mainstream models under every harness on the left. The teams on the right are invited through a sealed track that keeps their product private.