AIALMI

Model intelligence

Explore published coding benchmark runs.

Aider Polyglot results with original test dates, configurations and source evidence. Historical results are not a promise of current workload performance.

69 published runs · tested 2024-12-21 to 2025-10-03 · 225 coding exercises per run.

These are historical runs, not a ranking of current model quality. Settings and Aider versions differ. Single-model runs are shown by default; multi-model setups are available separately. A missing catalog link means the tested identity could not be matched unambiguously.

Aider methodology · How this explorer orders results · Apache-2.0 license

66 of 69 published runs

Scroll the results sideways for test dates and run details.

Published Aider Polyglot runs. Different configurations are separate observations, not equivalent model rankings.
Tested model / setupCorrectTest dateConfiguration and evidence
gpt-5 (high)

Single-model run

Catalog: GPT-5		88.0%2025-08-23

diff edits · 225/225 reported test cases

Run details: gpt-5 (high)

Aider 0.86.2.dev · harness commit 32faf82

aider --model openai/gpt-5

Published settings: {"reasoning_effort":"high"}

Run ID: 2025-08-23-15-47-21--gpt-5-high. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-5 (medium)

Single-model run

Catalog: GPT-5		86.7%2025-08-25

diff edits · 225/225 reported test cases

Run details: gpt-5 (medium)

Aider 0.86.2.dev · harness commit 32faf82

aider --model openai/gpt-5

Published settings: {"reasoning_effort":"medium"}

Run ID: 2025-08-25-13-23-27--gpt-5-medium. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o3-pro (high)

Single-model run

Catalog: o3-pro		84.9%2025-06-28

diff edits · 225/225 reported test cases

Run details: o3-pro (high)

Aider 0.85.1.dev · harness commit 5318380

aider --model o3-pro

Published settings: {"reasoning_effort":"high"}

Run ID: 2025-06-28-00-38-18--o3-pro-high. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.5-pro-preview-06-05 (32k think)

Single-model run

No unambiguous catalog match

83.1%2025-06-06

diff-fenced edits · 225/225 reported test cases

Run details: gemini-2.5-pro-preview-06-05 (32k think)

Aider 0.84.1.dev · harness commit f827f22

aider --model gemini/gemini-2.5-pro-preview-06-05 --thinking-tokens 32k

Published settings: {"thinking_tokens":32768}

Run ID: 2025-06-06-16-36-21--gemini0605-32k-think-diff-fenced. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o3 (high)

Single-model run

Catalog: o3		81.3%2025-06-25

diff edits · 225/225 reported test cases

Run details: o3 (high)

Aider 0.84.1.dev · harness commit c48fea6

aider --model o3 --reasoning-effort high

Published settings: {"reasoning_effort":"high"}

Run ID: 2025-06-25-21-04-24--o3-price-reduction-high. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-5 (low)

Single-model run

Catalog: GPT-5		81.3%2025-08-25

diff edits · 225/225 reported test cases

Run details: gpt-5 (low)

Aider 0.86.2.dev · harness commit 32faf82

aider --model openai/gpt-5

Published settings: {"reasoning_effort":"low"}

Run ID: 2025-08-25-14-16-37--gpt-5-low. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

grok-4 (high)

Single-model run

No unambiguous catalog match

79.6%2025-07-11

diff edits · 225/225 reported test cases

Run details: grok-4 (high)

Aider 0.85.2.dev · harness commit f7870b6-dirty

aider --model openrouter/x-ai/grok-4

Published settings: {"reasoning_effort":"high"}

Run ID: 2025-07-11-19-37-40--xai-or-grok4-high. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.5-pro-preview-06-05 (default think)

Single-model run

No unambiguous catalog match

79.1%2025-06-06

diff-fenced edits · 225/225 reported test cases

Run details: gemini-2.5-pro-preview-06-05 (default think)

Aider 0.84.1.dev · harness commit 4c161f9-dirty

aider --model gemini/gemini-2.5-pro-preview-06-05

Published settings: No additional settings reported

Run ID: 2025-06-06-18-38-56--gemini0605-diff-fenced. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Gemini 2.5 Pro Preview 05-06

Single-model run

No unambiguous catalog match

76.9%2025-05-07

diff-fenced edits · 225/225 reported test cases

Run details: Gemini 2.5 Pro Preview 05-06

Aider 0.82.4.dev · harness commit 3b08327-dirty

aider --model gemini/gemini-2.5-pro-preview-05-06

Published settings: No additional settings reported

Run ID: 2025-05-07-19-32-40--gemini0506-diff-fenced-completion_cost. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o3

Single-model run

Catalog: o3		76.9%2025-06-25

diff edits · 225/225 reported test cases

Run details: o3

Aider 0.84.1.dev · harness commit c48fea6

aider --model o3

Published settings: No additional settings reported

Run ID: 2025-06-25-20-30-16--o3-price-reduction. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek-V3.2-Exp (Reasoner)

Single-model run

No unambiguous catalog match

74.2%2025-10-03

diff edits · 225/225 reported test cases

Run details: DeepSeek-V3.2-Exp (Reasoner)

Aider 0.86.2.dev · harness commit cbb5376

aider --model deepseek/deepseek-reasoner

Published settings: No additional settings reported

Run ID: 2025-10-03-09-45-34--deepseek-v3.2-reasoner. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Gemini 2.5 Pro Preview 03-25

Single-model run

No unambiguous catalog match

72.9%2025-04-12

diff-fenced edits · 225/225 reported test cases

Run details: Gemini 2.5 Pro Preview 03-25

Aider 0.81.3.dev · harness commit 0282574

aider --model gemini/gemini-2.5-pro-preview-03-25

Published settings: No additional settings reported

Run ID: 2025-04-12-04-55-50--gemini-25-pro-diff-fenced. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o4-mini (high)

Single-model run

Catalog: o4-mini		72.0%2025-04-16

diff edits · 225/225 reported test cases

Run details: o4-mini (high)

Aider 0.82.1.dev · harness commit b66901f-dirty

aider --model o4-mini

Published settings: No additional settings reported

Run ID: 2025-04-16-22-01-58--o4-mini-high-diff-exsys. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-opus-4-20250514 (32k thinking)

Single-model run

No unambiguous catalog match

72.0%2025-05-25

diff edits · 225/225 reported test cases

Run details: claude-opus-4-20250514 (32k thinking)

Aider 0.83.3.dev · harness commit 9ef3211

aider --model claude-opus-4-20250514

Published settings: {"thinking_tokens":32000}

Run ID: 2025-05-25-20-40-51--opus4-diff-exuser. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek R1 (0528)

Single-model run

No unambiguous catalog match

71.4%2025-06-06

diff edits · 224/225 reported test cases

Run details: DeepSeek R1 (0528)

Aider 0.84.1.dev · harness commit 4c161f9-dirty

aider --model deepseek/deepseek-reasoner

Published settings: No additional settings reported

Run ID: 2025-06-06-16-47-07--r1-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-opus-4-20250514 (no think)

Single-model run

No unambiguous catalog match

70.7%2025-05-25

diff edits · 225/225 reported test cases

Run details: claude-opus-4-20250514 (no think)

Aider 0.83.3.dev · harness commit 9ef3211

aider --model claude-opus-4-20250514

Published settings: No additional settings reported

Run ID: 2025-05-25-19-57-20--opus4-diff-exuser. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek-V3.2-Exp (Chat)

Single-model run

No unambiguous catalog match

70.2%2025-10-03

diff edits · 225/225 reported test cases

Run details: DeepSeek-V3.2-Exp (Chat)

Aider 0.86.2.dev · harness commit cbb5376

aider --model deepseek/deepseek-chat

Published settings: No additional settings reported

Run ID: 2025-10-03-09-21-36--deepseek-v3.2-chat. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-3-7-sonnet-20250219 (32k thinking tokens)

Single-model run

No unambiguous catalog match

64.9%2025-02-24

diff edits · 225/225 reported test cases

Run details: claude-3-7-sonnet-20250219 (32k thinking tokens)

Aider 0.75.1.dev · harness commit 60d11a6, 93edbda

aider --model anthropic/claude-3-7-sonnet-20250219 --thinking-tokens 32k

Published settings: No additional settings reported

Run ID: 2025-02-24-21-47-23--sonnet37-diff-think-32k-64k. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o1-2024-12-17 (high)

Single-model run

No unambiguous catalog match

61.7%2024-12-21

diff edits · 224/225 reported test cases

Run details: o1-2024-12-17 (high)

Aider 0.69.2.dev · harness commit a755079-dirty

aider --model openrouter/openai/o1

Published settings: No additional settings reported

Run ID: 2024-12-21-19-23-03--polyglot-o1-hard-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-sonnet-4-20250514 (32k thinking)

Single-model run

No unambiguous catalog match

61.3%2025-05-24

diff edits · 225/225 reported test cases

Run details: claude-sonnet-4-20250514 (32k thinking)

Aider 0.83.3.dev · harness commit e3cb907

aider --model claude-sonnet-4-20250514

Published settings: {"thinking_tokens":32000}

Run ID: 2025-05-24-22-10-36--sonnet4-diff-exuser-think32k. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o3-mini (high)

Single-model run

Catalog: o3-mini		60.4%2025-01-31

diff edits · 224/225 reported test cases

Run details: o3-mini (high)

Aider 0.72.4.dev · harness commit b0d58d1-dirty

aider --model o3-mini --reasoning-effort high

Published settings: No additional settings reported

Run ID: 2025-01-31-20-42-47--o3-mini-diff-high. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-3-7-sonnet-20250219 (no thinking)

Single-model run

No unambiguous catalog match

60.4%2025-02-24

diff edits · 225/225 reported test cases

Run details: claude-3-7-sonnet-20250219 (no thinking)

Aider 0.74.4.dev · harness commit 75e9ee6

aider --model sonnet

Published settings: No additional settings reported

Run ID: 2025-02-24-19-54-07--sonnet37-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Qwen3 235B A22B diff, no think, Alibaba API

Single-model run

No unambiguous catalog match

59.6%2025-05-09

diff edits · 225/225 reported test cases

Run details: Qwen3 235B A22B diff, no think, Alibaba API

Aider 0.82.4.dev · harness commit 91d7fbd-dirty

aider --model openai/qwen3-235b-a22b

Published settings: No additional settings reported

Run ID: 2025-05-09-17-02-02--qwen3-235b-a22b.unthink_16k_diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Kimi K2

Single-model run

No unambiguous catalog match

59.1%2025-07-17

diff edits · 225/225 reported test cases

Run details: Kimi K2

Aider 0.85.3.dev · harness commit 915ebff-dirty

aider --model openrouter/moonshotai/kimi-k2

Published settings: No additional settings reported

Run ID: 2025-07-17-17-41-54--kimi-k2-diff-or-pricing. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek R1

Single-model run

No unambiguous catalog match

56.9%2025-01-20

diff edits · 225/225 reported test cases

Run details: DeepSeek R1

Aider 0.71.2.dev · harness commit 5650697-dirty

aider --model deepseek/deepseek-reasoner

Published settings: No additional settings reported

Run ID: 2025-01-20-19-11-38--ds-turns-upd-cur-msgs-fix-with-summarizer. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-sonnet-4-20250514 (no thinking)

Single-model run

No unambiguous catalog match

56.4%2025-05-24

diff edits · 225/225 reported test cases

Run details: claude-sonnet-4-20250514 (no thinking)

Aider 0.83.3.dev · harness commit ef3f8bb-dirty

aider --model claude-sonnet-4-20250514

Published settings: No additional settings reported

Run ID: 2025-05-24-21-17-54--sonnet4-diff-exuser. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek V3 (0324)

Single-model run

No unambiguous catalog match

55.1%2025-03-24

diff edits · 225/225 reported test cases

Run details: DeepSeek V3 (0324)

Aider 0.78.1.dev · harness commit 502b863

aider --model deepseek/deepseek-chat

Published settings: No additional settings reported

Run ID: 2025-03-24-15-41-33--deepseek-v3-0324-polyglot-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.5-flash-preview-05-20 (24k think)

Single-model run

No unambiguous catalog match

55.1%2025-05-25

diff edits · 225/225 reported test cases

Run details: gemini-2.5-flash-preview-05-20 (24k think)

Aider 0.83.3.dev · harness commit a8568c3-dirty

aider --model gemini/gemini-2.5-flash-preview-05-20

Published settings: {"thinking_tokens":24576}

Run ID: 2025-05-25-22-58-44--flash25-05-20-24k-think. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Quasar Alpha

Single-model run

No unambiguous catalog match

54.7%2025-04-04

diff edits · 225/225 reported test cases

Run details: Quasar Alpha

Aider 0.80.5.dev · harness commit 8a34a6c-dirty

aider --model openrouter/openrouter/quasar-alpha

Published settings: No additional settings reported

Run ID: 2025-04-04-02-57-25--qalpha-diff-exsys. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o3-mini (medium)

Single-model run

Catalog: o3-mini		53.8%2025-01-31

diff edits · 225/225 reported test cases

Run details: o3-mini (medium)

Aider 0.72.4.dev · harness commit 2fb517b-dirty

aider --model o3-mini

Published settings: No additional settings reported

Run ID: 2025-01-31-20-27-46--o3-mini-diff2. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Grok 3 Beta

Single-model run

No unambiguous catalog match

53.3%2025-04-10

diff edits · 225/225 reported test cases

Run details: Grok 3 Beta

Aider 0.81.2.dev · harness commit 2dd40fc-dirty

aider --model openrouter/x-ai/grok-3-beta

Published settings: No additional settings reported

Run ID: 2025-04-10-04-21-31--grok3-diff-exuser. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Optimus Alpha

Single-model run

No unambiguous catalog match

52.9%2025-04-10

diff edits · 225/225 reported test cases

Run details: Optimus Alpha

Aider 0.81.2.dev · harness commit 532bc45-dirty

aider --model openrouter/openrouter/optimus-alpha

Published settings: No additional settings reported

Run ID: 2025-04-10-19-02-44--oalpha-diff-exsys. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4.1

Single-model run

Catalog: GPT-4.1		52.4%2025-04-14

diff edits · 225/225 reported test cases

Run details: gpt-4.1

Aider 0.81.4.dev · harness commit 7a87db5-dirty

aider --model gpt-4.1

Published settings: No additional settings reported

Run ID: 2025-04-14-21-05-54--gpt41-diff-exuser. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-3-5-sonnet-20241022

Single-model run

No unambiguous catalog match

51.6%2025-01-17

diff edits · 225/225 reported test cases

Run details: claude-3-5-sonnet-20241022

Aider 0.71.2.dev · harness commit 6451d59

aider --model claude-3-5-sonnet-20241022

Published settings: No additional settings reported

Run ID: 2025-01-17-19-44-33--sonnet-baseline-jan-17. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Grok 3 Mini Beta (high)

Single-model run

No unambiguous catalog match

49.3%2025-04-10

whole edits · 225/225 reported test cases

Run details: Grok 3 Mini Beta (high)

Aider 0.81.3.dev · harness commit 8ee33da-dirty

aider --model xai/grok-3-mini-beta --reasoning-effort high

Published settings: No additional settings reported

Run ID: 2025-04-10-23-59-02--xai-grok3-mini-whole-high. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek Chat V3 (prev)

Single-model run

No unambiguous catalog match

48.4%2024-12-25

diff edits · 225/225 reported test cases

Run details: DeepSeek Chat V3 (prev)

Aider 0.69.2.dev · harness commit 0a23c4a-dirty

aider --model deepseek/deepseek-chat

Published settings: No additional settings reported

Run ID: 2024-12-25-13-31-51--deepseekv3preview-diff2. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.5-flash-preview-04-17 (default)

Single-model run

No unambiguous catalog match

47.1%2025-04-20

diff edits · 225/225 reported test cases

Run details: gemini-2.5-flash-preview-04-17 (default)

Aider 0.82.3.dev · harness commit 7fcce5d-dirty

aider --model gemini/gemini-2.5-flash-preview-04-17

Published settings: No additional settings reported

Run ID: 2025-04-20-19-54-31--flash25-diff-no-think. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

chatgpt-4o-latest (2025-03-29)

Single-model run

No unambiguous catalog match

45.3%2025-03-29

diff edits · 225/225 reported test cases

Run details: chatgpt-4o-latest (2025-03-29)

Aider 0.79.3.dev · harness commit 0decbad

aider --model chatgpt-4o-latest

Published settings: No additional settings reported

Run ID: 2025-03-29-05-24-55--chatgpt4o-mar28-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4.5-preview

Single-model run

No unambiguous catalog match

44.9%2025-02-27

diff edits · 224/225 reported test cases

Run details: gpt-4.5-preview

Aider 0.75.2.dev · harness commit b462e55-dirty

aider --model openai/gpt-4.5-preview

Published settings: No additional settings reported

Run ID: 2025-02-27-20-26-15--gpt45-diff3. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.5-flash-preview-05-20 (no think)

Single-model run

No unambiguous catalog match

44.0%2025-05-26

diff edits · 225/225 reported test cases

Run details: gemini-2.5-flash-preview-05-20 (no think)

Aider 0.83.3.dev · harness commit 214b811-dirty

aider --model gemini/gemini-2.5-flash-preview-05-20

Published settings: {"thinking_tokens":0}

Run ID: 2025-05-26-15-56-31--flash25-05-20-24k-think. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-oss-120b (high)

Single-model run

No unambiguous catalog match

41.8%2025-08-06

diff edits · 225/225 reported test cases

Run details: gpt-oss-120b (high)

Aider 0.85.3.dev · harness commit 1af0e59

aider --model openrouter/openai/gpt-oss-120b --reasoning-effort high

Published settings: No additional settings reported

Run ID: 2025-08-06-04-54-48--gpt-oss-120b-high-polyglot. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Qwen3 32B

Single-model run

No unambiguous catalog match

40.0%2025-05-08

diff edits · 225/225 reported test cases

Run details: Qwen3 32B

Aider 0.82.4.dev · harness commit aaacee5-dirty, aeaf259

aider --model openrouter/qwen/qwen3-32b

Published settings: No additional settings reported

Run ID: 2025-05-08-03-20-24--qwen3-32b-default. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-exp-1206

Single-model run

No unambiguous catalog match

38.2%2024-12-22

whole edits · 225/225 reported test cases

Run details: gemini-exp-1206

Aider 0.69.2.dev · harness commit b1bc2f8

aider --model gemini/gemini-exp-1206

Published settings: No additional settings reported

Run ID: 2024-12-22-18-43-25--gemini-exp-1206-polyglot-whole-2. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Gemini 2.0 Pro exp-02-05

Single-model run

No unambiguous catalog match

35.6%2025-02-25

whole edits · 225/225 reported test cases

Run details: Gemini 2.0 Pro exp-02-05

Aider 0.75.2.dev · harness commit 2fccd47

aider --model gemini/gemini-2.0-pro-exp-02-05

Published settings: No additional settings reported

Run ID: 2025-02-25-20-23-07--gemini-pro. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Grok 3 Mini Beta (low)

Single-model run

No unambiguous catalog match

34.7%2025-04-10

whole edits · 225/225 reported test cases

Run details: Grok 3 Mini Beta (low)

Aider 0.81.2.dev · harness commit 14ffe77-dirty

aider --model openrouter/x-ai/grok-3-mini-beta

Published settings: No additional settings reported

Run ID: 2025-04-10-18-47-24--grok3-mini-whole-exuser. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

o1-mini-2024-09-12

Single-model run

No unambiguous catalog match

32.9%2024-12-22

whole edits · 225/225 reported test cases

Run details: o1-mini-2024-09-12

Aider 0.69.2.dev · harness commit 37df899

aider --model o1-mini

Published settings: No additional settings reported

Run ID: 2024-12-22-21-26-35--polyglot-o1mini-whole. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4.1-mini

Single-model run

Catalog: GPT-4.1 mini		32.4%2025-04-14

diff edits · 225/225 reported test cases

Run details: gpt-4.1-mini

Aider 0.81.4.dev · harness commit ffb743e-dirty

aider --model gpt-4.1-mini

Published settings: No additional settings reported

Run ID: 2025-04-14-21-27-53--gpt41mini-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

claude-3-5-haiku-20241022

Single-model run

No unambiguous catalog match

28.0%2024-12-21

diff edits · 225/225 reported test cases

Run details: claude-3-5-haiku-20241022

Aider 0.69.2.dev · harness commit a755079-dirty

aider --model claude-3-5-haiku-20241022

Published settings: No additional settings reported

Run ID: 2024-12-21-21-46-27--polyglot-haiku-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

chatgpt-4o-latest (2025-02-15)

Single-model run

No unambiguous catalog match

27.1%2025-02-15

diff edits · 223/225 reported test cases

Run details: chatgpt-4o-latest (2025-02-15)

Aider 0.74.3.dev · harness commit 108ce18-dirty

aider --model chatgpt-4o-latest

Published settings: No additional settings reported

Run ID: 2025-02-15-19-51-22--chatgpt4o-feb15-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4o-2024-08-06

Single-model run

Catalog: GPT-4o (2024-08-06)		23.1%2024-12-30

diff edits · 225/225 reported test cases

Run details: gpt-4o-2024-08-06

Aider 0.70.1.dev · harness commit 09ee197-dirty

aider --model gpt-4o-2024-08-06

Published settings: No additional settings reported

Run ID: 2024-12-30-20-44-54--gpt4o-ex-as-sys-clean-prompt. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.0-flash-exp

Single-model run

No unambiguous catalog match

22.2%2024-12-22

whole edits · 225/225 reported test cases

Run details: gemini-2.0-flash-exp

Aider 0.69.2.dev · harness commit b1bc2f8

aider --model gemini/gemini-2.0-flash-exp

Published settings: No additional settings reported

Run ID: 2024-12-22-20-08-13--gemini-2.0-flash-exp-polyglot-whole. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

qwen-max-2025-01-25

Single-model run

No unambiguous catalog match

21.8%2025-01-28

diff edits · 225/225 reported test cases

Run details: qwen-max-2025-01-25

Aider 0.72.4.dev · harness commit ae7d459

OPENAI_API_BASE=https://dashscope-intl.aliyuncs.com/compatible-mode/v1 aider --model openai/qwen-max-2025-01-25

Published settings: No additional settings reported

Run ID: 2025-01-28-16-00-03--qwen-max-2025-01-25-polyglot-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

QwQ-32B

Single-model run

No unambiguous catalog match

20.9%2025-03-06

diff edits · 225/225 reported test cases

Run details: QwQ-32B

Aider 0.75.3.dev · harness commit 51d118f-dirty

aider --model fireworks_ai/accounts/fireworks/models/qwq-32b

Published settings: No additional settings reported

Run ID: 2025-03-06-17-40-24--qwq32b-diff-temp-topp-ex-sys-remind-user-for-real. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4o-2024-11-20

Single-model run

Catalog: GPT-4o (2024-11-20)		18.2%2024-12-30

diff edits · 225/225 reported test cases

Run details: gpt-4o-2024-11-20

Aider 0.70.1.dev · harness commit 09ee197-dirty

aider --model gpt-4o-2024-11-20

Published settings: No additional settings reported

Run ID: 2024-12-30-20-57-12--gpt-4o-2024-11-20-ex-as-sys. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemini-2.0-flash-thinking-exp-01-21

Single-model run

No unambiguous catalog match

18.2%2025-01-21

diff edits · 225/225 reported test cases

Run details: gemini-2.0-flash-thinking-exp-01-21

Aider 0.72.2.dev · harness commit 843720a

aider --model gemini/gemini-2.0-flash-thinking-exp-01-21

Published settings: No additional settings reported

Run ID: 2025-01-21-22-51-49--gemini-2.0-flash-thinking-exp-01-21-polyglot-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

DeepSeek Chat V2.5

Single-model run

No unambiguous catalog match

17.8%2024-12-21

diff edits · 225/225 reported test cases

Run details: DeepSeek Chat V2.5

Aider 0.69.2.dev · harness commit a755079-dirty

aider --model deepseek/deepseek-chat

Published settings: No additional settings reported

Run ID: 2024-12-21-20-56-21--polyglot-deepseek-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Qwen2.5-Coder-32B-Instruct

Single-model run

No unambiguous catalog match

16.4%2024-12-26

whole edits · 225/225 reported test cases

Run details: Qwen2.5-Coder-32B-Instruct

Aider 0.69.2.dev · harness commit b51768b0

aider --model openai/Qwen2.5-Coder-32B-Instruct

Published settings: No additional settings reported

Run ID: 2024-12-26-00-55-20--Qwen2.5-Coder-32B-Instruct. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Llama 4 Maverick

Single-model run

No unambiguous catalog match

15.6%2025-04-06

whole edits · 225/225 reported test cases

Run details: Llama 4 Maverick

Aider 0.81.2.dev · harness commit 9445a31

aider --model nvidia_nim/meta/llama-4-maverick-17b-128e-instruct

Published settings: No additional settings reported

Run ID: 2025-04-06-08-39-52--llama-4-maverick-17b-128e-instruct-polyglot-whole. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

yi-lightning

Single-model run

No unambiguous catalog match

12.9%2024-12-23

whole edits · 225/225 reported test cases

Run details: yi-lightning

Aider 0.69.2.dev · harness commit 2b1625e

aider --model openai/yi-lightning

Published settings: No additional settings reported

Run ID: 2024-12-23-01-11-56--yi-test. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

command-a-03-2025-quality

Single-model run

No unambiguous catalog match

12.0%2025-03-14

whole edits · 225/225 reported test cases

Run details: command-a-03-2025-quality

Aider 0.77.1.dev · harness commit a1aa63f

OPENAI_API_BASE=https://api.cohere.ai/compatibility/v1 aider --model openai/command-a-03-2025-quality

Published settings: No additional settings reported

Run ID: 2025-03-14-23-40-00--cmda-quality-whole2. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Codestral 25.01

Single-model run

No unambiguous catalog match

11.1%2025-01-13

whole edits · 225/225 reported test cases

Run details: Codestral 25.01

Aider 0.71.2.dev · harness commit 0cba898-dirty

aider --model mistral/codestral-latest

Published settings: No additional settings reported

Run ID: 2025-01-13-18-17-25--codestral-whole2. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

openhands-lm-32b-v0.1

Single-model run

No unambiguous catalog match

10.2%2025-04-19

whole edits · 225/225 reported test cases

Run details: openhands-lm-32b-v0.1

Aider 0.82.2.dev · harness commit c08336f

aider --model openrouter/all-hands/openhands-lm-32b-v0.1

Published settings: No additional settings reported

Run ID: 2025-04-19-14-43-04--o4-mini-patch. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4.1-nano

Single-model run

Catalog: GPT-4.1 nano		8.9%2025-04-14

whole edits · 225/225 reported test cases

Run details: gpt-4.1-nano

Aider 0.81.4.dev · harness commit 71d1591-dirty

aider --model gpt-4.1-nano

Published settings: No additional settings reported

Run ID: 2025-04-14-22-46-01--gpt41nano-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Qwen2.5-Coder-32B-Instruct

Single-model run

No unambiguous catalog match

8.0%2024-12-22

diff edits · 225/225 reported test cases

Run details: Qwen2.5-Coder-32B-Instruct

Aider 0.69.2.dev · harness commit 6d7e8be-dirty

aider --model openai/Qwen/Qwen2.5-Coder-32B-Instruct # via hyperbolic

Published settings: No additional settings reported

Run ID: 2024-12-22-13-22-32--polyglot-qwen-diff. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gemma-3-27b-it

Single-model run

No unambiguous catalog match

4.9%2025-03-15

whole edits · 225/225 reported test cases

Run details: gemma-3-27b-it

Aider 0.77.1.dev · harness commit fd21f51-dirty

aider --model openrouter/google/gemma-3-27b-it

Published settings: No additional settings reported

Run ID: 2025-03-15-01-21-24--gemma3-27b-or. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

gpt-4o-mini-2024-07-18

Single-model run

No unambiguous catalog match

3.6%2024-12-21

whole edits · 225/225 reported test cases

Run details: gpt-4o-mini-2024-07-18

Aider 0.69.2.dev · harness commit a755079-dirty

aider --model gpt-4o-mini-2024-07-18

Published settings: No additional settings reported

Run ID: 2024-12-21-18-41-18--polyglot-gpt-4o-mini. Dataset revision: 5dc9490bb35f9729ef2c95d00a19ccd30c26339c.

Retrieved 2026-09-17T10:38:47.056Z.

SHA-256: 85a50b25953512d18ba4bb0c23c0b8e626fcf9a5b52d287644b8a0b44b9535de

Original source data · Aider Polyglot · Apache-2.0

Fields normalized and catalog links added by AI Language Models Index; source scores unchanged.

Ordering: aider-run-explorer-v1. Score and date sort descending; name sorts ascending; ties use run ID ascending. Every configuration is retained. Sponsorship does not affect the order.