Soft reset hides last commit - identify commit hash to recover
Tests ability to recover a commit lost via soft reset by inspecting reflog. Evaluates basic reflog navigation for recovery.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'Initial content' > file.txt
  5. 05 git add file.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 echo 'Second version' > file.txt
  8. 08 git add file.txt
  9. 09 git commit -m 'Add second version'
  10. 10 echo 'Third version' > file.txt
  11. 11 git add file.txt
  12. 12 git commit -m 'Add third version - important changes'
  13. 13 git reset --soft HEAD~1
Prompt
You ran 'git reset --soft HEAD~1' which moved HEAD back one commit but kept the changes staged. Using git reflog, identify the full commit hash of the commit you just reset away. Provide ONLY the 40-character commit hash.
Expected
Add third version - important changes

Scoped model quality, cost, API time, and token usage for reflog/f001.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 280 in → 79 out (14 reasoning)
8911ce4e45209d8c6d89b76977d0e143b1f7f2d8
anthropic/claude-fable-5:high__json_schema PASS 100% 489 in → 35 out (0 reasoning)
682c75087e009f4a29ef5f50cdbf562599cd2993
JSON Schema Structured Output
(raw) {"ref":"682c75087e009f4a29ef5f50cdbf562599cd2993"}
anthropic/claude-fable-5:low PASS 100% 287 in → 29 out (0 reasoning)
4c472e24fcf168a6782b1f5b757040ddcaad4d21
anthropic/claude-fable-5:low__json_schema PASS 100% 497 in → 35 out (0 reasoning)
2f61542f84682bfb8e50cd83f103faf03955ed77
JSON Schema Structured Output
(raw) {"ref": "2f61542f84682bfb8e50cd83f103faf03955ed77"}
anthropic/claude-fable-5:max PASS 100% 287 in → 345 out (67 reasoning)
12359ef13d3ea255b8cb3b7a112b8a891297d462
anthropic/claude-fable-5:max__json_schema PASS 100% 500 in → 311 out (59 reasoning)
e1c0663692c27cefc86bc3132cdabe8214ed62ef
JSON Schema Structured Output
(raw) {"ref": "e1c0663692c27cefc86bc3132cdabe8214ed62ef"}
anthropic/claude-fable-5:medium PASS 100% 282 in → 26 out (0 reasoning)
49401f663c810a7d79e14b9cce35598e95c305d0
anthropic/claude-fable-5:medium__json_schema PASS 100% 497 in → 35 out (0 reasoning)
9a118abcadf1463f356e1af201ac538ad5ad4708
JSON Schema Structured Output
(raw) {"ref": "9a118abcadf1463f356e1af201ac538ad5ad4708"}
anthropic/claude-fable-5:xhigh PASS 100% 286 in → 88 out (33 reasoning)
9b192ec5f893eec2b8c4d2dae490cc0af4e94d1d
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 507 in → 135 out (25 reasoning)
2a8f4d2ed4db64b6bffd6786e004e7d9af01fe7d
JSON Schema Structured Output
(raw) {"ref": "2a8f4d2ed4db64b6bffd6786e004e7d9af01fe7d"}
anthropic/claude-haiku-4.5:high PASS 100% 234 in → 452 out (309 reasoning)
7ea4fefe02c9ccb838dc5cb881b88ed4d677059e
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 403 in → 685 out (486 reasoning)
7776fd961632c42efebc8ec9140ecf333d830e9d
JSON Schema Structured Output
(raw) {"ref": "7776fd961632c42efebc8ec9140ecf333d830e9d"}
anthropic/claude-haiku-4.5:low PASS 100% 238 in → 562 out (407 reasoning)
ab31d852582a4853d160fe00a95b54c7d1c05d77
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 411 in → 994 out (489 reasoning)
618bc669243e69ecd278bee808e039b52ff0da45
JSON Schema Structured Output
(raw) {"ref": "618bc669243e69ecd278bee808e039b52ff0da45"}
anthropic/claude-haiku-4.5:medium PASS 100% 247 in → 301 out (197 reasoning)
9d519c4e91d0e7e2b2792e1885a98297650846c1
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 399 in → 453 out (337 reasoning)
89ad0b109badd2f3ab97fdbdfda5cc907f953df7
JSON Schema Structured Output
(raw) {"ref":"89ad0b109badd2f3ab97fdbdfda5cc907f953df7"}
anthropic/claude-haiku-4.5:none PASS 100% 220 in → 32 out (0 reasoning)
3b4f64ae2650245ba1ce0f81b3c2b7e4f7f49345
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 378 in → 33 out (0 reasoning)
7e2df761a7c2d7319bc8d395c0eb968836c28a79
JSON Schema Structured Output
(raw) {"ref": "7e2df761a7c2d7319bc8d395c0eb968836c28a79"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 240 in → 328 out (263 reasoning)
abb274028be5dea58ae9c8fa55c1212f4559084f
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 403 in → 766 out (335 reasoning)
8d1d31c27c079239f89753b9a801e392b58c5f50
JSON Schema Structured Output
(raw) {"ref": "8d1d31c27c079239f89753b9a801e392b58c5f50"}
anthropic/claude-opus-4.6:high PASS 100% 217 in → 26 out (0 reasoning)
c9d18107d8fe565c0514790bad7a528c3fd60c77
anthropic/claude-opus-4.6:high__json_schema PASS 100% 385 in → 99 out (39 reasoning)
9143bf26acbfd43ffaa5a44bde77c1e5f493ff77
JSON Schema Structured Output
(raw) {"ref":"9143bf26acbfd43ffaa5a44bde77c1e5f493ff77"}
anthropic/claude-opus-4.6:low PASS 100% 219 in → 29 out (0 reasoning)
ebfe2a8ae8b3b2bfd4ccb6074f022ebb81dd9447
anthropic/claude-opus-4.6:low__json_schema PASS 100% 381 in → 31 out (0 reasoning)
fc04b7883cb557ea6fc08e8f8b35c1d99700953b
JSON Schema Structured Output
(raw) {"ref":"fc04b7883cb557ea6fc08e8f8b35c1d99700953b"}
anthropic/claude-opus-4.6:max PASS 100% 215 in → 300 out (135 reasoning)
`3f72c725d76760088a8bfb4ecd7719e1bbe0f536` This is the commit at **HEAD@{1}** ("Add third version - important changes"), which is where HEAD pointed **before** the reset. The reset moved HEAD back to `c966f57...` (the "Add second version" commit), but the reflog preserved a record of the original commit.
anthropic/claude-opus-4.6:max__json_schema PASS 100% 366 in → 202 out (115 reasoning)
5365e695fd0a6d582141c76f84391ff0aed9d0b4
JSON Schema Structured Output
(raw) {"ref":"5365e695fd0a6d582141c76f84391ff0aed9d0b4"}
anthropic/claude-opus-4.6:medium PASS 100% 209 in → 23 out (0 reasoning)
51074148d078911d928ff38a56251188db0c4506
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 387 in → 32 out (0 reasoning)
d4d4a94ae4474d8a7df5be63dfbb17714e6651ce
JSON Schema Structured Output
(raw) {"ref":"d4d4a94ae4474d8a7df5be63dfbb17714e6651ce"}
anthropic/claude-opus-4.6:none PASS 100% 214 in → 28 out (0 reasoning)
4c912d65e7ce21de8b7033093426d423e66cad7b
anthropic/claude-opus-4.6:none__json_schema PASS 100% 373 in → 34 out (0 reasoning)
2e9b19c7e1fe670dc0264f78dcb623b1fcd4ff1a
JSON Schema Structured Output
(raw) {"ref":"2e9b19c7e1fe670dc0264f78dcb623b1fcd4ff1a"}
anthropic/claude-opus-4.6:xhigh PASS 100% 210 in → 26 out (0 reasoning)
5db66d44d16c976dffdc95d0903360f8ee111134
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 386 in → 32 out (0 reasoning)
f3f1d2977b47e936f526ed9275e4600d59bb0ec9
JSON Schema Structured Output
(raw) {"ref":"f3f1d2977b47e936f526ed9275e4600d59bb0ec9"}
anthropic/claude-opus-4.7:high PASS 100% 293 in → 34 out (0 reasoning)
0a1d9b23e18ebbc10a4109e4eac112b26452a3fd
anthropic/claude-opus-4.7:high__json_schema PASS 100% 495 in → 37 out (0 reasoning)
993f725726efbf36f41fed14960b35c55e7f2ef4
JSON Schema Structured Output
(raw) {"ref":"993f725726efbf36f41fed14960b35c55e7f2ef4"}
anthropic/claude-opus-4.7:low PASS 100% 282 in → 32 out (0 reasoning)
b1e9124da96d6f48d39f8dcb41a534b13a51eedd
anthropic/claude-opus-4.7:low__json_schema PASS 100% 494 in → 38 out (0 reasoning)
42aafb58cd3996f8b319b4c857e93c86edebeb12
JSON Schema Structured Output
(raw) {"ref":"42aafb58cd3996f8b319b4c857e93c86edebeb12"}
anthropic/claude-opus-4.7:max PASS 100% 287 in → 479 out (68 reasoning)
6a54df53a339ea6314442c562ede50def9dfd589
anthropic/claude-opus-4.7:max__json_schema PASS 100% 499 in → 37 out (0 reasoning)
9820b0bf1077854e27aa839ca68d8b99237d1138
JSON Schema Structured Output
(raw) {"ref":"9820b0bf1077854e27aa839ca68d8b99237d1138"}
anthropic/claude-opus-4.7:medium PASS 100% 293 in → 30 out (0 reasoning)
abf4c066b39f101d973d7b1eaead94ecee29d64f
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 506 in → 43 out (0 reasoning)
9d13f2002e30c33d5c982d54f0bc69e4f1e2c9f2
JSON Schema Structured Output
(raw) {"ref":"9d13f2002e30c33d5c982d54f0bc69e4f1e2c9f2"}
anthropic/claude-opus-4.7:none PASS 100% 279 in → 28 out (0 reasoning)
f0562f4cda505fb13c047870af5116504fc51bf3
anthropic/claude-opus-4.7:none__json_schema PASS 100% 507 in → 38 out (0 reasoning)
f36a41cdb13fc9e843871f67765d55e4d71f086e
JSON Schema Structured Output
(raw) {"ref":"f36a41cdb13fc9e843871f67765d55e4d71f086e"}
anthropic/claude-opus-4.7:xhigh PASS 100% 292 in → 35 out (0 reasoning)
7bf6c895b4fea61f71d5a1ddd1e3e365b619ef70
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 498 in → 39 out (0 reasoning)
c57cf2f6aeedc510ba95d8ca7b7667678e456901
JSON Schema Structured Output
(raw) {"ref":"c57cf2f6aeedc510ba95d8ca7b7667678e456901"}
anthropic/claude-opus-4.8:high PASS 100% 286 in → 116 out (58 reasoning)
2eac776b1529b05c537a12663ddf249758b4bbeb
anthropic/claude-opus-4.8:high__json_schema PASS 100% 494 in → 35 out (0 reasoning)
cba9f4e501bfaf08219d9fb328258886f1ade231
JSON Schema Structured Output
(raw) {"ref":"cba9f4e501bfaf08219d9fb328258886f1ade231"}
anthropic/claude-opus-4.8:low PASS 100% 282 in → 29 out (0 reasoning)
f59636eda33d22b0349ce03ddab136fb1bfe9f00
anthropic/claude-opus-4.8:low__json_schema PASS 100% 478 in → 35 out (0 reasoning)
dbb90458f5bbe23c59ffb045978a876913682ba3
JSON Schema Structured Output
(raw) {"ref":"dbb90458f5bbe23c59ffb045978a876913682ba3"}
anthropic/claude-opus-4.8:max PASS 100% 281 in → 1,165 out (91 reasoning)
**668050ac42ed704a5f1aac49586460f9e0a78f66** This is the commit that was reset away. Here's the reasoning: - Before the reset, `HEAD` pointed to `668050ac...` (the "Add third version - important changes" commit) — shown at `HEAD@{1}`. - The `reset: moving to HEAD~1` operation at `HEAD@{0}` moved `HEAD` back to `135082f7...` (the "Add second version" commit). So the "Add third version" commit (`668050ac42ed704a5f1aac49586460f9e0a78f66`) is the one you reset away. Its changes are still staged, and the commit itself remains recoverable via this hash (e.g., `git reset --soft 668050ac...` or `git cherry-pick`).
anthropic/claude-opus-4.8:max__json_schema PASS 100% 507 in → 550 out (65 reasoning)
5dadd4286c0c02e7b928f0dba30b4d93781566e0
JSON Schema Structured Output
(raw) {"ref": "5dadd4286c0c02e7b928f0dba30b4d93781566e0"}
anthropic/claude-opus-4.8:medium PASS 100% 288 in → 105 out (40 reasoning)
c429b7b92588e18c123a1c7bd5e7f10300a618ae
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 490 in → 36 out (0 reasoning)
d6673edaa5675e5ffa021cd744b2a75836ba248f
JSON Schema Structured Output
(raw) {"ref": "d6673edaa5675e5ffa021cd744b2a75836ba248f"}
anthropic/claude-opus-4.8:none PASS 100% 280 in → 24 out (0 reasoning)
c78a72295d18108f080f3664097fddea2b022df7
anthropic/claude-opus-4.8:none__json_schema PASS 100% 479 in → 31 out (0 reasoning)
864161c3652415f7b75653f312423ae14c4966e1
JSON Schema Structured Output
(raw) {"ref":"864161c3652415f7b75653f312423ae14c4966e1"}
anthropic/claude-opus-4.8:xhigh PASS 100% 286 in → 289 out (74 reasoning)
40f74a2a9ff037c2d4e1267851ed5d0175c15a2f
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 498 in → 191 out (54 reasoning)
f4caf34653f800e34f26bf02e548c08afaea5472
JSON Schema Structured Output
(raw) {"ref":"f4caf34653f800e34f26bf02e548c08afaea5472"}
anthropic/claude-sonnet-4.6:high PASS 100% 205 in → 221 out (158 reasoning)
`bddb9ddf0938d9b672620f8b58492f79cfafe86a`
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 376 in → 32 out (0 reasoning)
be9f1626ccb47b61a4cf601f557e5e3dabe3920d
JSON Schema Structured Output
(raw) {"ref":"be9f1626ccb47b61a4cf601f557e5e3dabe3920d"}
anthropic/claude-sonnet-4.6:low PASS 100% 200 in → 22 out (0 reasoning)
cb74a82acf88fc26148979de64922e3915574025
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 366 in → 32 out (0 reasoning)
f05d5a935622ab990e18a92f64bec7a96cf96d8d
JSON Schema Structured Output
(raw) {"ref":"f05d5a935622ab990e18a92f64bec7a96cf96d8d"}
anthropic/claude-sonnet-4.6:max PASS 100% 208 in → 480 out (143 reasoning)
## Identifying the Reset Commit Looking at the reflog: - `HEAD@{0}`: The **reset operation** — HEAD moved **to** `d2db63bd...` (HEAD~1) - `HEAD@{1}`: The **commit that was reset away** — `bf56da806b842cad9d9ecefeb68b93f1e73e41fb` The `git reset --soft HEAD~1` moved HEAD **away from** `HEAD@{1}` back to `HEAD@{2}`, meaning the "Add third version - important changes" commit is the one that was undone. ``` bf56da806b842cad9d9ecefeb68b93f1e73e41fb ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 372 in → 230 out (129 reasoning)
363762c880b98de9f4c1792c5949c3c166166f67
JSON Schema Structured Output
(raw) {"ref":"363762c880b98de9f4c1792c5949c3c166166f67"}
anthropic/claude-sonnet-4.6:medium PASS 100% 217 in → 32 out (0 reasoning)
fecf4ad3c77b8d48ced4a1daa64ec2d82ce11d7f
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 382 in → 31 out (0 reasoning)
56185e148766de6bfeb6b5777367962b1d7a0fb4
JSON Schema Structured Output
(raw) {"ref":"56185e148766de6bfeb6b5777367962b1d7a0fb4"}
anthropic/claude-sonnet-4.6:none PASS 100% 212 in → 26 out (0 reasoning)
65d33789dc6624554a554ea22ece89d0dc5babd1
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 379 in → 30 out (0 reasoning)
b4a41294e73d83c7510432ee90ebe2549c738a55
JSON Schema Structured Output
(raw) {"ref":"b4a41294e73d83c7510432ee90ebe2549c738a55"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 213 in → 199 out (116 reasoning)
`3ef6818dfd5d653200999ceb77101b2339b36969`
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 374 in → 29 out (0 reasoning)
684422caacd86a2ccabdc050f64b0003e6e68090
JSON Schema Structured Output
(raw) {"ref":"684422caacd86a2ccabdc050f64b0003e6e68090"}
anthropic/claude-sonnet-5:high PASS 100% 289 in → 31 out (0 reasoning)
cf0550f6fa7e6cfb45e3eb7589804717a20cdc02
anthropic/claude-sonnet-5:high__json_schema PASS 100% 488 in → 271 out (35 reasoning)
a20e2cbc7e90927def5a657027aa93d9df61d2bc
JSON Schema Structured Output
(raw) {"ref":"a20e2cbc7e90927def5a657027aa93d9df61d2bc"}
anthropic/claude-sonnet-5:low PASS 100% 287 in → 28 out (0 reasoning)
e8ef748285c0f655e033ca9b6d16557fb8481d46
anthropic/claude-sonnet-5:low__json_schema PASS 100% 496 in → 239 out (61 reasoning)
36b9de46a1ef68c87c3bfa64855928569728953b
JSON Schema Structured Output
(raw) {"ref":"36b9de46a1ef68c87c3bfa64855928569728953b"}
anthropic/claude-sonnet-5:medium PASS 100% 274 in → 72 out (0 reasoning)
The commit that was reset away is the one referenced at HEAD@{1}, which was the commit before the reset moved HEAD back: **5e23896c0c28e422ed09ee133df1726dac58e9e8**
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 492 in → 153 out (50 reasoning)
2adbfbc1faed05b60ea0321b3cafbd05e1c7bd5f
JSON Schema Structured Output
(raw) {"ref": "2adbfbc1faed05b60ea0321b3cafbd05e1c7bd5f"}
anthropic/claude-sonnet-5:none PASS 100% 281 in → 29 out (0 reasoning)
fa867b9a7b40981a3db388bf9e567bc87bdf654d
anthropic/claude-sonnet-5:none__json_schema PASS 100% 495 in → 35 out (0 reasoning)
447d76aaf465bfb5e8b22534675560ea211b2d2b
JSON Schema Structured Output
(raw) {"ref":"447d76aaf465bfb5e8b22534675560ea211b2d2b"}
anthropic/claude-sonnet-5:xhigh PASS 100% 287 in → 1,458 out (112 reasoning)
**fc385c5ed55cdab99346834c3692886e12c507ab** This is the commit hash that HEAD pointed to just before the `git reset --soft HEAD~1` command was run (visible at `HEAD@{1}` in the reflog, labeled "Add third version - important changes"). The reset moved HEAD back to `3e1c0eb0f41ea3bdf8122062fdffa21878589bf1` (the "Add second version" commit), while keeping the changes from the reset-away commit staged.
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 497 in → 36 out (0 reasoning)
a00552763a1cdb1e21b38cb1aca4132cf791256a
JSON Schema Structured Output
(raw) {"ref": "a00552763a1cdb1e21b38cb1aca4132cf791256a"}
arcee-ai/trinity-large-thinking:high PASS 100% 202 in → 395 out (378 reasoning)
695d8b61c54dad837883a0f57543f0ad328a5d9e
arcee-ai/trinity-large-thinking:low PASS 100% 203 in → 410 out (389 reasoning)
2edce0d1525571d621c9fed92516d9c5352a72f9
arcee-ai/trinity-large-thinking:medium PASS 100% 210 in → 403 out (387 reasoning)
861c7bd17e603f64f2ac21f20e513bc6b2463461
arcee-ai/trinity-large-thinking:xhigh PASS 100% 187 in → 359 out (343 reasoning)
b66650fdc23c50e41ad89af4da58623619fea9ea
arcee-ai/trinity-mini:high PASS 100% 195 in → 660 out (632 reasoning)
eef268c9cae945989073aaaae507f9c5a07cd0cf
arcee-ai/trinity-mini:low PASS 100% 205 in → 434 out (401 reasoning)
36cde36d8955fc5b8d1a9dfbd3b644a623c77a21
arcee-ai/trinity-mini:low__json_schema PASS 100% 196 in → 334 out (313 reasoning)
HEAD@{1}: commit: Add third version - important changes
JSON Schema Structured Output
(raw) { "ref": "HEAD@{1}: commit: Add third version - important changes" }
arcee-ai/trinity-mini:medium PASS 100% 198 in → 391 out (378 reasoning)
f97c0ead2dda8ddfbbc515fb471fcd373d3e1720
arcee-ai/trinity-mini:medium__json_schema PASS 100% 202 in → 383 out (344 reasoning)
aa70c27a955e6f5ca101fb393e73977e57c75bb3
JSON Schema Structured Output
(raw) { "ref": "aa70c27a955e6f5ca101fb393e73977e57c75bb3" }
arcee-ai/trinity-mini:xhigh PASS 100% 209 in → 1,091 out (1,085 reasoning)
e3f66e3251ea0f49ff1497817faec1bd7cb0386b
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 205 in → 338 out (304 reasoning)
HEAD@{1}: commit: Add third version - important changes
JSON Schema Structured Output
(raw) { "ref": "HEAD@{1}: commit: Add third version - important changes" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 276 in → 107 out (85 reasoning)
73b9731324a719367671e0d5422adb0af66223e9
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 278 in → 118 out (83 reasoning)
9a8464985ab420606e5056523b91c407818b408b
JSON Schema Structured Output
(raw) {"ref":"9a8464985ab420606e5056523b91c407818b408b"}
deepseek/deepseek-v4-flash-0731:low PASS 100% 204 in → 643 out (518 reasoning)
c618c6ee0f16057dc1df7d132a3b383acf113c3f
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 206 in → 297 out (225 reasoning)
82de801b6956eaf3cdf1585b85194d64c4fd2267
JSON Schema Structured Output
(raw) {"ref":"82de801b6956eaf3cdf1585b85194d64c4fd2267"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 280 in → 255 out (233 reasoning)
e7dc7077777b4c52de166537baacb421ebabd91b
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 284 in → 118 out (75 reasoning)
94117dfdd3ad61e55c728d036ce8deb6cab65b08
JSON Schema Structured Output
(raw) { "ref": "94117dfdd3ad61e55c728d036ce8deb6cab65b08" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 190 in → 79 out (57 reasoning)
0cf61aa0aa653252e2da954e87358902acfc9fbb
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 204 in → 293 out (223 reasoning)
b11ebf30eeda63c27c3da1c4dd00325fbf4fe10e
JSON Schema Structured Output
(raw) { "ref": "b11ebf30eeda63c27c3da1c4dd00325fbf4fe10e" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 196 in → 26 out (0 reasoning)
8ade2c53924c867730f0d4e1dc88b9be8b302ec0
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 285 in → 115 out (72 reasoning)
672b0974f6f0879e14529a79f5cb2fd6754d1d59
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 193 in → 183 out (153 reasoning)
1c917845bdd7090494c5703d0baa3e61886ea251
JSON Schema Structured Output
(raw) { "ref": "1c917845bdd7090494c5703d0baa3e61886ea251" }
deepseek/deepseek-v4-flash:high PASS 100% 200 in → 322 out (239 reasoning)
e251f36e5dac9bcea9a65538d7b8e196e0842f10
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 291 in → 27 out (0 reasoning)
8fb241da721dd0708b4035d9270b7edc476983e2
JSON Schema Structured Output
(raw) {"ref":"8fb241da721dd0708b4035d9270b7edc476983e2"}
deepseek/deepseek-v4-flash:low PASS 100% 197 in → 380 out (357 reasoning)
faf13fd19cb140210e2020f48ba74a0ab0256d6b
deepseek/deepseek-v4-flash:medium PASS 100% 199 in → 154 out (109 reasoning)
b4f2302c19cb7a4f2d3890cf488f1647532fead6
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 197 in → 212 out (181 reasoning)
2a0786198d7179c72ed0049b22af6d66def991db
JSON Schema Structured Output
(raw) { "ref": "2a0786198d7179c72ed0049b22af6d66def991db" }
deepseek/deepseek-v4-flash:none PASS 100% 205 in → 21 out (0 reasoning)
9a03368735ce65adbb28d0ff43580a553935cc20
deepseek/deepseek-v4-flash:xhigh PASS 100% 201 in → 339 out (316 reasoning)
e877fcec7b3315c77fd28dca2aab608a60062919
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 340 in → 642 out (527 reasoning)
81840fe222c04c902082e958061077ade4e859ac
JSON Schema Structured Output
(raw) { "ref": "81840fe222c04c902082e958061077ade4e859ac" }
deepseek/deepseek-v4-pro:high PASS 100% 203 in → 166 out (142 reasoning)
207faec1a95dd0574e3991a4dcc7642ed56280be
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 205 in → 419 out (281 reasoning)
7edd3afaf9d415f8bc85e89eb76aeb92af6e3a92
JSON Schema Structured Output
(raw) { "ref": "7edd3afaf9d415f8bc85e89eb76aeb92af6e3a92" }
deepseek/deepseek-v4-pro:low PASS 100% 197 in → 390 out (363 reasoning)
3c5d035c326d43c2047de0619a7e65424b4aa3d2
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 196 in → 207 out (173 reasoning)
7fd7867cc8842cadbdf9f34d0315e778534e52a9
JSON Schema Structured Output
(raw) { "ref": "7fd7867cc8842cadbdf9f34d0315e778534e52a9" }
deepseek/deepseek-v4-pro:medium PASS 100% 196 in → 338 out (314 reasoning)
acb3538d19f5a543a087e9638bb078685136e60e
deepseek/deepseek-v4-pro:none PASS 100% 197 in → 22 out (0 reasoning)
68dce525f60394b92a852eee58d3e63220e063b1
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 387 in → 27 out (0 reasoning)
7f65f774690eb9fb290eca8b0f9812616632fc11
JSON Schema Structured Output
(raw) {"ref": "7f65f774690eb9fb290eca8b0f9812616632fc11"}
deepseek/deepseek-v4-pro:xhigh PASS 100% 281 in → 374 out (347 reasoning)
7bdeef601b9db1d7e772c02ccdf5a991f35bdd45
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 460 in → 596 out (564 reasoning)
905b5d11f7fb46cee8786aaf1b72a789eaf8edce
JSON Schema Structured Output
(raw) {"ref": "905b5d11f7fb46cee8786aaf1b72a789eaf8edce"}
google/gemini-3-flash-preview:high PASS 100% 256 in → 450 out (414 reasoning)
52d898dc4668291c6256f3673856ff402bf4df64
google/gemini-3-flash-preview:high__json_schema PASS 100% 303 in → 444 out (403 reasoning)
8de5bb63014ea33231717a038073956f79a01c6f
JSON Schema Structured Output
(raw) {"ref":"8de5bb63014ea33231717a038073956f79a01c6f"}
google/gemini-3-flash-preview:low PASS 100% 251 in → 580 out (542 reasoning)
c27ed1278bf37846595190c89e134c067f9e7a9d
google/gemini-3-flash-preview:low__json_schema PASS 100% 295 in → 569 out (524 reasoning)
cb71f23d3cc5d0ecf4cb7e51d537f63f8e1b8840
JSON Schema Structured Output
(raw) { "ref": "cb71f23d3cc5d0ecf4cb7e51d537f63f8e1b8840" }
google/gemini-3-flash-preview:medium PASS 100% 249 in → 477 out (445 reasoning)
ab9532ee67dcbf59f2181ffbd36fcd8867ad8121
google/gemini-3-flash-preview:medium__json_schema PASS 100% 299 in → 835 out (792 reasoning)
4086b069a896bf4f9b2827c401dc99afacb3bac9
JSON Schema Structured Output
(raw) { "ref": "4086b069a896bf4f9b2827c401dc99afacb3bac9" }
google/gemini-3-flash-preview:xhigh PASS 100% 241 in → 608 out (572 reasoning)
bcc0da1b0b68019c345e8c81774e780453455aa1
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 302 in → 559 out (521 reasoning)
ad04f9f2f57ffc18dac6308106b67e1a0436ad40
JSON Schema Structured Output
(raw) {"ref":"ad04f9f2f57ffc18dac6308106b67e1a0436ad40"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 242 in → 551 out (515 reasoning)
f83cf04125a1a056d66ed543fe45ddc84263c31f
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 304 in → 747 out (708 reasoning)
39e47aae7a4afbef6a8a8f9a8e123000c6ba0e3a
JSON Schema Structured Output
(raw) {"ref":"39e47aae7a4afbef6a8a8f9a8e123000c6ba0e3a"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 243 in → 176 out (141 reasoning)
d32f124ef1f996af422e5c21fb105eb42176af31
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 240 in → 196 out (153 reasoning)
be462ff8410cbf410b48762699ac4b61bb8bcd5c
JSON Schema Structured Output
(raw) { "ref": "be462ff8410cbf410b48762699ac4b61bb8bcd5c" }
google/gemini-3.1-flash-lite-preview:medium PASS 100% 251 in → 500 out (461 reasoning)
5d60f6f348e60597c3e7b5b168bd27e841d83372
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 306 in → 616 out (576 reasoning)
0d26cff9fd5b0be0aaa4b49a95f4034c29698495
JSON Schema Structured Output
(raw) {"ref": "0d26cff9fd5b0be0aaa4b49a95f4034c29698495"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 243 in → 847 out (813 reasoning)
aa4d343087115a70012218c15a5bea37cdfed05e
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 301 in → 875 out (838 reasoning)
bcad71fbea3afb5d7a985a8b9d8222021eff909d
JSON Schema Structured Output
(raw) {"ref":"bcad71fbea3afb5d7a985a8b9d8222021eff909d"}
google/gemini-3.1-pro-preview:high PASS 100% 242 in → 912 out (876 reasoning)
fe64f9b0d63598179d88a9eb5fdd0896c6fc3750
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 298 in → 416 out (375 reasoning)
eca44a3877daf1d0c9a9d4322a207789f8e559f5
JSON Schema Structured Output
(raw) {"ref": "eca44a3877daf1d0c9a9d4322a207789f8e559f5"}
google/gemini-3.1-pro-preview:low PASS 100% 245 in → 317 out (279 reasoning)
d0192f002ea624c29e30266427dd4540e8778b19
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 305 in → 231 out (190 reasoning)
57a90c666fa78054838db72a729b123283eff9a8
JSON Schema Structured Output
(raw) {"ref": "57a90c666fa78054838db72a729b123283eff9a8"}
google/gemini-3.1-pro-preview:medium PASS 100% 233 in → 325 out (292 reasoning)
beba4d4fa9d02fcba70737a587b7f253ddd70577
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 296 in → 233 out (192 reasoning)
a2022fb24835bb1a8680fe52e1fae598240b9d7b
JSON Schema Structured Output
(raw) {"ref": "a2022fb24835bb1a8680fe52e1fae598240b9d7b"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 247 in → 711 out (675 reasoning)
4f56a9c2d8fcf6da75c611ce241c65f12bc73231
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 307 in → 744 out (703 reasoning)
552367761ca18e905af09045d4133c133133ff60
JSON Schema Structured Output
(raw) {"ref":"552367761ca18e905af09045d4133c133133ff60"}
google/gemini-3.5-flash-lite:high PASS 100% 231 in → 817 out (784 reasoning)
6da46ba2238a97e7676831f0869ccd4daac1cf18
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 249 in → 801 out (762 reasoning)
7bb2d8eb02c81e356422514fcabb2956f4050a41
JSON Schema Structured Output
(raw) {"ref":"7bb2d8eb02c81e356422514fcabb2956f4050a41"}
google/gemini-3.5-flash-lite:low PASS 100% 238 in → 34 out (0 reasoning)
13dcb7dd8072afd72823557e0378b66caaa384b8
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 295 in → 40 out (0 reasoning)
f23617864e87dca95bd602e9207724ef0e14fd1c
JSON Schema Structured Output
(raw) {"ref": "f23617864e87dca95bd602e9207724ef0e14fd1c"}
google/gemini-3.5-flash-lite:max PASS 100% 250 in → 559 out (522 reasoning)
a64b53cde23177d1a85e0c057946540789ebfb12
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 247 in → 607 out (560 reasoning)
059a95ea8559c05d909dcd5b8ed2890e0a07d3d0
JSON Schema Structured Output
(raw) { "ref": "059a95ea8559c05d909dcd5b8ed2890e0a07d3d0" }
google/gemini-3.5-flash-lite:medium PASS 100% 247 in → 536 out (503 reasoning)
5eafede39d5e7cdebd6fa4258964c26391f558be
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 305 in → 532 out (488 reasoning)
615e7593cac5dabf53c0ce68096b32f1c90090cb
JSON Schema Structured Output
(raw) { "ref": "615e7593cac5dabf53c0ce68096b32f1c90090cb" }
google/gemini-3.5-flash-lite:xhigh PASS 100% 244 in → 752 out (723 reasoning)
9eda1bea65aac1cdbbd2dd8f21b2597b09f9feb8
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 244 in → 595 out (556 reasoning)
5d51bfd345248d472746c16035ff82277ccc99ba
JSON Schema Structured Output
(raw) {"ref":"5d51bfd345248d472746c16035ff82277ccc99ba"}
google/gemini-3.5-flash:high PASS 100% 246 in → 1,045 out (1,010 reasoning)
e3160445e535dad644fd660d04fbde9f0b92a281
google/gemini-3.5-flash:high__json_schema PASS 100% 310 in → 968 out (929 reasoning)
1f1aa0264286cb7b3f84af4138ffd3115f26030f
JSON Schema Structured Output
(raw) {"ref":"1f1aa0264286cb7b3f84af4138ffd3115f26030f"}
google/gemini-3.5-flash:low PASS 100% 248 in → 371 out (333 reasoning)
d0c41e3105c2972e863390098947640e5dfcd728
google/gemini-3.5-flash:low__json_schema PASS 100% 304 in → 273 out (235 reasoning)
eee67fb0afd63e3c7cc2f09f552448aa91207031
JSON Schema Structured Output
(raw) {"ref": "eee67fb0afd63e3c7cc2f09f552448aa91207031"}
google/gemini-3.5-flash:medium PASS 100% 246 in → 604 out (570 reasoning)
34a2cc27b847d50e7b910865f4803ead6baf5adc
google/gemini-3.5-flash:medium__json_schema PASS 100% 292 in → 624 out (587 reasoning)
196381b53caf45ae77ae4e0e46ffdd3aab00e30d
JSON Schema Structured Output
(raw) {"ref":"196381b53caf45ae77ae4e0e46ffdd3aab00e30d"}
google/gemini-3.5-flash:xhigh PASS 100% 245 in → 761 out (725 reasoning)
8ee2c68a9a0b2c00e12a8bc161cd3dcb751f956a
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 303 in → 642 out (605 reasoning)
2933ea439aae01cab05829fc6ad038dadb431331
JSON Schema Structured Output
(raw) {"ref": "2933ea439aae01cab05829fc6ad038dadb431331"}
google/gemini-3.6-flash:high PASS 100% 238 in → 640 out (606 reasoning)
3eaa697f434de994c975f07f41a891441efcfe40
google/gemini-3.6-flash:high__json_schema PASS 100% 298 in → 421 out (386 reasoning)
3fec84716bef36ff931012cf99ef79527aa72bcb
JSON Schema Structured Output
(raw) {"ref":"3fec84716bef36ff931012cf99ef79527aa72bcb"}
google/gemini-3.6-flash:low PASS 100% 247 in → 183 out (151 reasoning)
18dcee29df6f5dbfe63af0029cd2ccf073137e1f
google/gemini-3.6-flash:low__json_schema PASS 100% 247 in → 230 out (191 reasoning)
64fb6c837cf44b81a93d1317b3e81fceb1bea937
JSON Schema Structured Output
(raw) {"ref":"64fb6c837cf44b81a93d1317b3e81fceb1bea937"}
google/gemini-3.6-flash:max PASS 100% 250 in → 800 out (764 reasoning)
ff206c92162c03fdc18df7c1c836b73f3de2c333
google/gemini-3.6-flash:max__json_schema PASS 100% 234 in → 499 out (463 reasoning)
aaea4164bbc6b6341207102d3b00aadd4ace02b0
JSON Schema Structured Output
(raw) {"ref":"aaea4164bbc6b6341207102d3b00aadd4ace02b0"}
google/gemini-3.6-flash:medium PASS 100% 248 in → 617 out (578 reasoning)
2a2d9c12846e71b0b1f34d3a9b273859d32b5cb8
google/gemini-3.6-flash:medium__json_schema PASS 100% 303 in → 452 out (411 reasoning)
efc4b5261e3ed98e2935b485ef75f52224b1d851
JSON Schema Structured Output
(raw) {"ref":"efc4b5261e3ed98e2935b485ef75f52224b1d851"}
google/gemini-3.6-flash:xhigh PASS 100% 246 in → 673 out (637 reasoning)
602065a0bc2c3a9561dc80154ae87b80dd9f5046
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 244 in → 444 out (404 reasoning)
097577d876f6fd7ae6436069a07039b6f595ffcd
JSON Schema Structured Output
(raw) {"ref":"097577d876f6fd7ae6436069a07039b6f595ffcd"}
google/gemma-4-26b-a4b-it:high PASS 100% 261 in → 798 out (471 reasoning)
00a6d5f69a2a2e008287f550d351a97529531208
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 262 in → 534 out (326 reasoning)
205ac23aebb88ac84f5dc707be96d63b24db4efb
JSON Schema Structured Output
(raw) {"ref": "205ac23aebb88ac84f5dc707be96d63b24db4efb"}
google/gemma-4-26b-a4b-it:low PASS 100% 269 in → 570 out (336 reasoning)
d0730cef74a7f4886ca503a1f21b10f76e27f4e6
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 253 in → 720 out (431 reasoning)
554952e78efda42db5ac7fa80b0ff582062a04cd
JSON Schema Structured Output
(raw) { "ref": "554952e78efda42db5ac7fa80b0ff582062a04cd" }
google/gemma-4-26b-a4b-it:medium PASS 100% 258 in → 621 out (386 reasoning)
4d7add40bb4618a12305cf3b8bcd53f263d57313
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 259 in → 561 out (342 reasoning)
b4bffbde9125f71c5ea28dd8f3ca5dc9cfb04e19
JSON Schema Structured Output
(raw) { "ref": "b4bffbde9125f71c5ea28dd8f3ca5dc9cfb04e19" }
google/gemma-4-26b-a4b-it:none PASS 100% 256 in → 37 out (0 reasoning)
336fb180137a4a1554a1d88583f0feded966c74b
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 257 in → 49 out (0 reasoning)
00e6deb9a86102569b66f092d8f5d2785a21c358
JSON Schema Structured Output
(raw) { "ref": "00e6deb9a86102569b66f092d8f5d2785a21c358" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 253 in → 680 out (436 reasoning)
ed0609504b1f23abf55db0498572a7bf01dc3d9d
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 253 in → 526 out (340 reasoning)
8460100d6bc38feb67e5cf6ac90017a692cc0e22
JSON Schema Structured Output
(raw) { "ref": "8460100d6bc38feb67e5cf6ac90017a692cc0e22" }
google/gemma-4-31b-it:high PASS 100% 257 in → 486 out (301 reasoning)
380ea19078f3f20eb9993937c69097591a635ed0
google/gemma-4-31b-it:high__json_schema PASS 100% 258 in → 456 out (259 reasoning)
02eb5b6727f86e74a6617f771e6a345f7a377b11
JSON Schema Structured Output
(raw) { "ref": "02eb5b6727f86e74a6617f771e6a345f7a377b11" }
google/gemma-4-31b-it:low PASS 100% 262 in → 522 out (298 reasoning)
bea91ce2eeab99c7e94554c2c9aad810050b7810
google/gemma-4-31b-it:low__json_schema PASS 100% 267 in → 548 out (300 reasoning)
aee47b7263ec77b3dd59040e4d3076644d280950
JSON Schema Structured Output
(raw) { "ref": "aee47b7263ec77b3dd59040e4d3076644d280950" }
google/gemma-4-31b-it:medium PASS 100% 265 in → 513 out (286 reasoning)
9f060cfc1f17b803abee041c758a02ce27e8da94
google/gemma-4-31b-it:medium__json_schema PASS 100% 261 in → 528 out (489 reasoning)
bae07f0bb94f1bec46be3541da5b1003f29e4956
JSON Schema Structured Output
(raw) {"ref":"bae07f0bb94f1bec46be3541da5b1003f29e4956"}
google/gemma-4-31b-it:none PASS 100% 258 in → 37 out (0 reasoning)
198c08be202bf60b6b1db6320493e3ed1e219441
google/gemma-4-31b-it:none__json_schema PASS 100% 260 in → 44 out (0 reasoning)
038b4db8ec9a78c60db0635416c195fb7d8f172f
JSON Schema Structured Output
(raw) { "ref": "038b4db8ec9a78c60db0635416c195fb7d8f172f" }
google/gemma-4-31b-it:xhigh PASS 100% 262 in → 563 out (1 reasoning)
405e94214b0fba72885da4862fe8c2d64ccfb09d
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 269 in → 577 out (534 reasoning)
4b17a6707e8a843ad23c6008d19d4372b6373feb
JSON Schema Structured Output
(raw) {"ref":"4b17a6707e8a843ad23c6008d19d4372b6373feb"}
inclusionai/ling-2.6-flash PASS 100% 265 in → 35 out
593e9652efeba109955de5ebc5a662113afd11d8
minimax/minimax-m2.5:high PASS 100% 229 in → 434 out (332 reasoning)
91b81cbd1c8c320295402d0a51661c3096ca4a9c
minimax/minimax-m2.5:high__json_schema PASS 100% 227 in → 390 out (293 reasoning)
3df3db63bce1f864b338e2938e53a4909466815f
JSON Schema Structured Output
(raw) { "ref": "3df3db63bce1f864b338e2938e53a4909466815f" }
minimax/minimax-m2.5:low PASS 100% 235 in → 620 out (595 reasoning)
bc13987597e9d07186f6a92b8c1b2390c8441665
minimax/minimax-m2.5:low__json_schema PASS 100% 239 in → 495 out (388 reasoning)
6d3e6672b62864d3af4dddf93dc3b82d123f29d9
JSON Schema Structured Output
(raw) { "ref": "6d3e6672b62864d3af4dddf93dc3b82d123f29d9" }
minimax/minimax-m2.5:medium PASS 100% 223 in → 655 out (632 reasoning)
341907869e62aaf05c3d884b42c4605cf30da836
minimax/minimax-m2.5:medium__json_schema PASS 100% 235 in → 480 out (345 reasoning)
2d6bc54ce3d92dea46399791aae17c0c86d8e9e9
JSON Schema Structured Output
(raw) { "ref": "2d6bc54ce3d92dea46399791aae17c0c86d8e9e9" }
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 216 in → 637 out (621 reasoning)
ee407036c73a737597ddeb9ae513cd354562565c
JSON Schema Structured Output
(raw) { "ref": "ee407036c73a737597ddeb9ae513cd354562565c" }
minimax/minimax-m2.7:high PASS 100% 237 in → 387 out (362 reasoning)
45064f19f2c9fdd01ac9253afe9b5456698b31fd
minimax/minimax-m2.7:high__json_schema PASS 100% 367 in → 352 out (323 reasoning)
d29d4f6be23eb4486df10308a3b7de96579347ba
JSON Schema Structured Output
(raw) {"ref":"d29d4f6be23eb4486df10308a3b7de96579347ba"}
minimax/minimax-m2.7:low PASS 100% 233 in → 555 out (532 reasoning)
77073c86dd096af7bef836925dd408eb1f6933f4
minimax/minimax-m2.7:low__json_schema PASS 100% 358 in → 650 out (624 reasoning)
babfd946daf190f260d38f5a918222b013af271c
JSON Schema Structured Output
(raw) {"ref":"babfd946daf190f260d38f5a918222b013af271c"}
minimax/minimax-m2.7:medium PASS 100% 228 in → 380 out (357 reasoning)
e160571cc17b2725a089c90c325df0396c72e909
minimax/minimax-m2.7:medium__json_schema PASS 100% 363 in → 676 out (640 reasoning)
8f8f80a9a738ed855e09b2f65d3a38a26626382b
JSON Schema Structured Output
(raw) { "ref": "8f8f80a9a738ed855e09b2f65d3a38a26626382b" }
minimax/minimax-m2.7:xhigh PASS 100% 223 in → 767 out (744 reasoning)
13a61ddffa5dbd20524a17a454e6bd64ba21b716
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 363 in → 536 out (505 reasoning)
aa1d861de7e5d21ed8c4670ca1bed67fe0341b0e
JSON Schema Structured Output
(raw) {"ref":"aa1d861de7e5d21ed8c4670ca1bed67fe0341b0e"}
minimax/minimax-m3:high PASS 100% 370 in → 190 out (126 reasoning)
0ff08c3cac310f9f61670d3a66eda76efec0c5d9
minimax/minimax-m3:low PASS 100% 381 in → 116 out (87 reasoning)
57648721192ad4ce2a9b5202f8367b621abb8a64
minimax/minimax-m3:medium PASS 100% 370 in → 190 out (154 reasoning)
dc7a7547f4c7095e16ca91a04f20de34ee68e57b
minimax/minimax-m3:medium__json_schema PASS 100% 362 in → 37 out (0 reasoning)
f78727d7d30f9af70b939f9d4e2abba0f290e0de
JSON Schema Structured Output
(raw) { "ref": "f78727d7d30f9af70b939f9d4e2abba0f290e0de" }
minimax/minimax-m3:xhigh PASS 100% 378 in → 385 out (276 reasoning)
64d4d19b524f5618bf923ce7f193a2691e134809
minimax/minimax-m3:xhigh__json_schema PASS 100% 357 in → 30 out (0 reasoning)
44ca1d81ff45ef6bed9b5209267daa7c214ef8e2
JSON Schema Structured Output
(raw) {"ref": "44ca1d81ff45ef6bed9b5209267daa7c214ef8e2"}
mistralai/devstral-2512 PASS 100% 244 in → 50 out
The full commit hash of the commit you just reset away is: abc994a624108108a3379e98be852dd245f9d958
mistralai/devstral-2512__json_schema PASS 100% 240 in → 42 out
e9d3e00a7bbc5661cec489080a8ba66444d06450
JSON Schema Structured Output
(raw) {"ref":"e9d3e00a7bbc5661cec489080a8ba66444d06450"}
mistralai/mistral-medium-3-5:high PASS 100% 264 in → 1,222 out (946 reasoning)
def74c4b065fd6e18f321302f332eec92f8b587d
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 251 in → 283 out (198 reasoning)
58f8f79e3fd957a3293dfeb0d4ac8e156ef5c859
JSON Schema Structured Output
(raw) {"ref": "58f8f79e3fd957a3293dfeb0d4ac8e156ef5c859"}
mistralai/mistral-medium-3-5:low PASS 100% 257 in → 2,003 out (1,490 reasoning)
2c73a31d47b5b0bcb90e6a6e4f564f2d4979769e
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 250 in → 341 out (224 reasoning)
9bc546b50f125553a2640b5df4b99ab5beb67410
JSON Schema Structured Output
(raw) {"ref": "9bc546b50f125553a2640b5df4b99ab5beb67410"}
mistralai/mistral-medium-3-5:medium PASS 100% 263 in → 721 out (591 reasoning)
aa331544a03f5f56762d799cebf3d70f5fdd09ca
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 242 in → 687 out (480 reasoning)
0abb05a22e185fbb0a245460bb9652c30ba9663c
JSON Schema Structured Output
(raw) {"ref": "0abb05a22e185fbb0a245460bb9652c30ba9663c"}
mistralai/mistral-medium-3-5:none PASS 100% 258 in → 35 out (0 reasoning)
b63910bf3e25ad1bc0f418fe7a46d067ec2fc790
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 241 in → 39 out (0 reasoning)
9f2f98cf50bbc7d99cebf396eb065cf1ba1e5368
JSON Schema Structured Output
(raw) {"ref": "9f2f98cf50bbc7d99cebf396eb065cf1ba1e5368"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 254 in → 1,221 out (1,038 reasoning)
95945f9a4768abf99ff49417df0470534544a5ea
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 250 in → 421 out (255 reasoning)
c16f3cb25c880d61294eab56dac31a2e85e711ef
JSON Schema Structured Output
(raw) {"ref": "c16f3cb25c880d61294eab56dac31a2e85e711ef"}
moonshotai/kimi-k2.6:high PASS 100% 212 in → 416 out (393 reasoning)
1e19aa73332f1e2f0c1705ed04d9a1be9df76252
moonshotai/kimi-k2.6:high__json_schema PASS 100% 194 in → 664 out (264 reasoning)
I'm looking at this reflog to identify the commit that was reset away. The user ran `git reset --soft HEAD~1`, which moved HEAD back one commit. Before the reset, HEAD was at `HEAD@{1}` which is `f31aca4901a9314f92dd709653a239e8478285de`. This is the commit that was "reset away" (the third version commit). The reset moved HEAD to `849a018286e9dbbcd036369f00a1a28ca8ff24bb` (which was HEAD~1). So the answer should be the 40-character hash: f31aca4901a9314f92dd709653a239e8478285de.
JSON Schema Structured Output
(raw) { "ref": "I'm looking at this reflog to identify the commit that was reset away. The user ran `git reset --soft HEAD~1`, which moved HEAD back one commit. Before the reset, HEAD was at `HEAD@{1}` which is `f31aca4901a9314f92dd709653a239e8478285de`. This is the commit that was \"reset away\" (the third version commit). The reset moved HEAD to `849a018286e9dbbcd036369f00a1a28ca8ff24bb` (which was HEAD~1). So the answer should be the 40-character hash: f31aca4901a9314f92dd709653a239e8478285de." }
moonshotai/kimi-k2.6:low PASS 100% 203 in → 379 out (292 reasoning)
a468a01aea46d6950908dcc42fb788057503ecfb
moonshotai/kimi-k2.6:low__json_schema PASS 100% 200 in → 690 out (658 reasoning)
5c15bbd276b992f06006e249f2654a518db2c0b7
JSON Schema Structured Output
(raw) { "ref": "5c15bbd276b992f06006e249f2654a518db2c0b7" }
moonshotai/kimi-k2.6:medium PASS 100% 193 in → 396 out (376 reasoning)
7eaf4a0e0560afa10b46fdb298cb71419f628689
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 204 in → 1,481 out (423 reasoning)
Answer: 7d8d5a7d1b1e896564b9a90ede30e9094a58d6e7
JSON Schema Structured Output
(raw) { "ref": "Answer: 7d8d5a7d1b1e896564b9a90ede30e9094a58d6e7" }
moonshotai/kimi-k2.6:xhigh PASS 100% 193 in → 378 out (353 reasoning)
faae5a9bc2f20d3f3f04484210e900798edf6dcb
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 199 in → 275 out (238 reasoning)
003c83c38cd7d7810ed17c6d8ee195822ed2221f
JSON Schema Structured Output
(raw) { "ref": "003c83c38cd7d7810ed17c6d8ee195822ed2221f" }
moonshotai/kimi-k2.7-code:high PASS 100% 189 in → 252 out (228 reasoning)
cce5297b9056a189cc46f249f97581d2da1c961e
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 285 in → 150 out (118 reasoning)
f2c88d90ba00ef21139d7074c00a4a46a6266e93
JSON Schema Structured Output
(raw) {"ref": "f2c88d90ba00ef21139d7074c00a4a46a6266e93"}
moonshotai/kimi-k2.7-code:low PASS 100% 200 in → 196 out (171 reasoning)
94f4d5df777071e6b58a67a398782e638e110ef4
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 207 in → 166 out (110 reasoning)
8bd4a35ac9ffbf913d4a5fdef4cca86713c47601
JSON Schema Structured Output
(raw) { "ref": "8bd4a35ac9ffbf913d4a5fdef4cca86713c47601" }
moonshotai/kimi-k2.7-code:medium PASS 100% 197 in → 200 out (177 reasoning)
d60e35651231f1955120fcc0a6fde298b40203b7
moonshotai/kimi-k2.7-code:xhigh PASS 100% 196 in → 128 out (99 reasoning)
65c706d6ee2f6d6f73ec2421211f5084b3c6b97d
moonshotai/kimi-k3:high PASS 100% 286 in → 317 out (276 reasoning)
1c077b4f6a0441fa07f87d69024d21c9e4ca2325
moonshotai/kimi-k3:high__json_schema PASS 100% 280 in → 201 out (163 reasoning)
517a2fc0b388c0b8744c494eee9322ed82c609e0
moonshotai/kimi-k3:low PASS 100% 266 in → 205 out (168 reasoning)
36bd20ee8b6a73a56451bc668b042aa278d35f60
moonshotai/kimi-k3:low__json_schema PASS 100% 277 in → 195 out (157 reasoning)
8a853e6aecb22fa0f14514f4269d8ec31674626a
moonshotai/kimi-k3:max PASS 100% 272 in → 254 out (221 reasoning)
043e9367c7898fb54592c550100d093acd207843
moonshotai/kimi-k3:max__json_schema PASS 100% 276 in → 322 out (281 reasoning)
92e88b977db3582dd1169a03dd3d52c9f03c8a4d
moonshotai/kimi-k3:medium PASS 100% 271 in → 256 out (221 reasoning)
41e1352277ff879fed6bafdd0552607c673053b7
moonshotai/kimi-k3:medium__json_schema PASS 100% 281 in → 244 out (204 reasoning)
88cb9d46a9c7b74d8d313c70be0b47438cd35578
moonshotai/kimi-k3:xhigh PASS 100% 264 in → 161 out (126 reasoning)
a9074d466f28e0184facb88aaf41986e37135215
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 275 in → 163 out (129 reasoning)
644a1011ec48c268bac2007b59656178d663cf41
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 257 in → 429 out (369 reasoning)
7fb65639a48e884614a42ca2c9fa6f9d9d188769
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 267 in → 774 out (521 reasoning)
c840d513a44ab8de27909084def667324bb7d968
JSON Schema Structured Output
(raw) { "ref": "c840d513a44ab8de27909084def667324bb7d968" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 259 in → 703 out (448 reasoning)
e63ef592ea7c5e43d360f162cb4b3e3884c2e614
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 263 in → 490 out (307 reasoning)
f139658800b8d6cac18f01ef33ac843be6ce1558
JSON Schema Structured Output
(raw) { "ref": "f139658800b8d6cac18f01ef33ac843be6ce1558" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 263 in → 368 out (197 reasoning)
0a13d4b58c409e2cb7bf4649c8685bb93c24325f
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 262 in → 628 out (418 reasoning)
855ac85fe35a3e2748005bb2254dab3b131cd6fd
JSON Schema Structured Output
(raw) {"ref":"855ac85fe35a3e2748005bb2254dab3b131cd6fd" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 257 in → 33 out (0 reasoning)
623c38fd44afc6ffc2e4cc1391cec8beff9ea610
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 258 in → 45 out (0 reasoning)
aa510b0a3391a1e0e0e71b6a3dd3d3742dbb0a1a
JSON Schema Structured Output
(raw) { "ref": "aa510b0a3391a1e0e0e71b6a3dd3d3742dbb0a1a" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 259 in → 747 out (397 reasoning)
4525fc17e4558fc39a163fdc0705c6b4ca762581
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 269 in → 295 out (198 reasoning)
d7622c1c82928c52e8380405d60e981f1e495a70
JSON Schema Structured Output
(raw) { "ref": "d7622c1c82928c52e8380405d60e981f1e495a70" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 270 in → 307 out (177 reasoning)
70195f54e976f97647679ffc313f32cde5ad1685
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 260 in → 361 out (276 reasoning)
51a4c6e8ed67ed05ed6d7d2a71d77442e600781e
JSON Schema Structured Output
(raw) { "ref": "51a4c6e8ed67ed05ed6d7d2a71d77442e600781e" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 260 in → 482 out (392 reasoning)
b543fc548401262b6aa617b0c49f68bce4ff3a55
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 265 in → 217 out (156 reasoning)
ea912fec541ff4e74781941d48a12a2835329a30
JSON Schema Structured Output
(raw) { "ref": "ea912fec541ff4e74781941d48a12a2835329a30" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 255 in → 393 out (321 reasoning)
edcab421487682d6a667b06cca817d4bf2b09fe8
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 262 in → 39 out (0 reasoning)
9dffc27c28100528fafc8d112b292e055263a8e4
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 258 in → 44 out (0 reasoning)
50b9fe6010221165385215f7cde9ebe544241d3c
JSON Schema Structured Output
(raw) { "ref": "50b9fe6010221165385215f7cde9ebe544241d3c" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 261 in → 432 out (308 reasoning)
9ae05414f3ff9da636872a18bd98392ea52fa215
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 263 in → 634 out (473 reasoning)
be3b67418f69991f46a812bbe175b2bc937d17b4
JSON Schema Structured Output
(raw) { "ref": "be3b67418f69991f46a812bbe175b2bc937d17b4" }
openai/gpt-5.4-mini:high PASS 100% 206 in → 110 out (78 reasoning)
3c1d6a69e54973a7b08513ff2aa821e9a6e581fe
openai/gpt-5.4-mini:high__json_schema PASS 100% 251 in → 400 out (358 reasoning)
9dda4b69e7b22a7fabfda16d9efc04041d7419f7
JSON Schema Structured Output
(raw) {"ref":"9dda4b69e7b22a7fabfda16d9efc04041d7419f7"}
openai/gpt-5.4-mini:low PASS 100% 203 in → 59 out (31 reasoning)
8ac80d91ad1cae76ac252469da02c5fa0608ce28
openai/gpt-5.4-mini:low__json_schema PASS 100% 243 in → 81 out (43 reasoning)
a0000469a2899614a41d40b6326df258581f8a09
JSON Schema Structured Output
(raw) {"ref":"a0000469a2899614a41d40b6326df258581f8a09"}
openai/gpt-5.4-mini:medium PASS 100% 199 in → 89 out (58 reasoning)
8a90d7e5224e52dd7bfd93024a27963c7b19ac5d
openai/gpt-5.4-mini:medium__json_schema PASS 100% 243 in → 124 out (89 reasoning)
a62189168860bd330a4e643aec743eb8096398e0
JSON Schema Structured Output
(raw) {"ref":"a62189168860bd330a4e643aec743eb8096398e0"}
openai/gpt-5.4-mini:none PASS 100% 201 in → 27 out (0 reasoning)
ce3cd3fae731b77a57ecb6128503333a172c3a47
openai/gpt-5.4-mini:none__json_schema PASS 100% 235 in → 38 out (0 reasoning)
d180df2c01fd3dae90dcc3d5ec47e5c1dfeb1441
JSON Schema Structured Output
(raw) {"ref":"d180df2c01fd3dae90dcc3d5ec47e5c1dfeb1441"}
openai/gpt-5.4-mini:xhigh PASS 100% 195 in → 100 out (70 reasoning)
4933a2c8188a1f4dd7c475f13250ca73304353c0
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 245 in → 212 out (175 reasoning)
58c73b217ae38eb411e8923b3b350993717efff4
JSON Schema Structured Output
(raw) {"ref":"58c73b217ae38eb411e8923b3b350993717efff4"}
openai/gpt-5.4-nano:high__json_schema PASS 100% 233 in → 145 out (108 reasoning)
9b8c88a731115efd87878118bd087fbf97ae7fd4
JSON Schema Structured Output
(raw) {"ref":"9b8c88a731115efd87878118bd087fbf97ae7fd4"}
openai/gpt-5.4-nano:low PASS 100% 205 in → 26 out (0 reasoning)
92ffe16ef81e7455a428de560efb3cdeb5f156d1
openai/gpt-5.4-nano:low__json_schema PASS 100% 241 in → 34 out (0 reasoning)
f429046f7ed84f3832174689e29169ea107c7571
JSON Schema Structured Output
(raw) {"ref":"f429046f7ed84f3832174689e29169ea107c7571"}
openai/gpt-5.4-nano:medium PASS 100% 200 in → 31 out (0 reasoning)
be62bd4c2373d1c431b5e5d512b5e6059ed8ecf2
openai/gpt-5.4-nano:medium__json_schema PASS 100% 234 in → 38 out (0 reasoning)
776dc9033a3a07c6b40dba1ac05555d9bd73f493
JSON Schema Structured Output
(raw) {"ref":"776dc9033a3a07c6b40dba1ac05555d9bd73f493"}
openai/gpt-5.4-nano:none PASS 100% 206 in → 27 out (0 reasoning)
a1910eb29a80eaa9d9cfd21b27dddece569aded9
openai/gpt-5.4-nano:none__json_schema PASS 100% 237 in → 38 out (0 reasoning)
e9ada1a0b97b1354745d20ed756f50522d9192a7
JSON Schema Structured Output
(raw) {"ref":"e9ada1a0b97b1354745d20ed756f50522d9192a7"}
openai/gpt-5.4-nano:xhigh PASS 100% 209 in → 198 out (166 reasoning)
5bf86f1ad286adf0818b4d699bd5b6a1c2c21030
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 236 in → 598 out (560 reasoning)
5342167185fa4cc43a6c653dcac5d67cb1d00469
JSON Schema Structured Output
(raw) {"ref":"5342167185fa4cc43a6c653dcac5d67cb1d00469"}
openai/gpt-5.4:high PASS 100% 208 in → 112 out (82 reasoning)
1772f6a6905d7e27f58c6d023da983751eadd57e
openai/gpt-5.4:high__json_schema PASS 100% 238 in → 93 out (57 reasoning)
d591ce3679f140628d94a05ffd4436431b981cd8
JSON Schema Structured Output
(raw) {"ref":"d591ce3679f140628d94a05ffd4436431b981cd8"}
openai/gpt-5.4:low PASS 100% 204 in → 25 out (0 reasoning)
8bd5612c211d0e8eb43790b80cbe840206919edf
openai/gpt-5.4:low__json_schema PASS 100% 242 in → 39 out (0 reasoning)
fca1b4804c900b50608c25f0eae15d25d85f16bf
JSON Schema Structured Output
(raw) {"ref":"fca1b4804c900b50608c25f0eae15d25d85f16bf"}
openai/gpt-5.4:medium PASS 100% 195 in → 60 out (34 reasoning)
93633abe2be96fa7227d012f473e347c20a09495
openai/gpt-5.4:medium__json_schema PASS 100% 235 in → 88 out (52 reasoning)
075208438243a4b36db2a01010601be63a7cc502
JSON Schema Structured Output
(raw) {"ref":"075208438243a4b36db2a01010601be63a7cc502"}
openai/gpt-5.4:none PASS 100% 204 in → 30 out (0 reasoning)
3bf751d221e7bd0f4e3b6e722fe3ad3dc6d49acc
openai/gpt-5.4:none__json_schema PASS 100% 228 in → 35 out (0 reasoning)
5db813236c6319db28f29c5337396958ef0fc68c
JSON Schema Structured Output
(raw) {"ref":"5db813236c6319db28f29c5337396958ef0fc68c"}
openai/gpt-5.4:xhigh PASS 100% 193 in → 114 out (86 reasoning)
8760126765191c3e30bf8c8e9714fed775b97f14
openai/gpt-5.4:xhigh__json_schema PASS 100% 242 in → 132 out (91 reasoning)
8bfc05b1c7387a40441f799d2dbc01e5680c5c87
JSON Schema Structured Output
(raw) {"ref":"8bfc05b1c7387a40441f799d2dbc01e5680c5c87"}
openai/gpt-5.5:high PASS 100% 204 in → 114 out (86 reasoning)
d3c09a5de993576421802cc10e63f5f852fa6218
openai/gpt-5.5:high__json_schema PASS 100% 243 in → 112 out (68 reasoning)
fe15cba2c9ef6bb2b87d724b1e0b2ea4c0ad04ef
JSON Schema Structured Output
(raw) {"ref":"fe15cba2c9ef6bb2b87d724b1e0b2ea4c0ad04ef"}
openai/gpt-5.5:low PASS 100% 200 in → 31 out (0 reasoning)
94707cf8b17758f4b7bc8bff23e6a3cdd0c710a0
openai/gpt-5.5:low__json_schema PASS 100% 239 in → 36 out (0 reasoning)
ef9976e6f83519491beea0a505b6319d44448bf6
JSON Schema Structured Output
(raw) {"ref":"ef9976e6f83519491beea0a505b6319d44448bf6"}
openai/gpt-5.5:medium PASS 100% 200 in → 63 out (34 reasoning)
7b63abe0f23c0fc74205f1f72ec59069cc52b060
openai/gpt-5.5:medium__json_schema PASS 100% 239 in → 82 out (42 reasoning)
b16b589ff942e959d605f45e182ff0828e2d2f2b
JSON Schema Structured Output
(raw) {"ref":"b16b589ff942e959d605f45e182ff0828e2d2f2b"}
openai/gpt-5.5:none PASS 100% 205 in → 25 out (0 reasoning)
9e8e3cb76eac431ac32b865218485133bfca82cc
openai/gpt-5.5:none__json_schema PASS 100% 243 in → 37 out (0 reasoning)
78df5a072805202afcd0e42c8c5778c9ca49a303
JSON Schema Structured Output
(raw) {"ref":"78df5a072805202afcd0e42c8c5778c9ca49a303"}
openai/gpt-5.5:xhigh PASS 100% 197 in → 153 out (129 reasoning)
381090f066896908c5359d00fd333ced108446a1
openai/gpt-5.5:xhigh__json_schema PASS 100% 235 in → 149 out (112 reasoning)
99ce8c94a776710343b532bb150af68f8ea8e603
JSON Schema Structured Output
(raw) {"ref":"99ce8c94a776710343b532bb150af68f8ea8e603"}
openai/gpt-5.6-luna:high PASS 100% 201 in → 27 out (0 reasoning)
8572ab15c600e8d3e5008a5a6d79615815297682
openai/gpt-5.6-luna:high__json_schema PASS 100% 242 in → 40 out (0 reasoning)
c8d9941797f1469fd5f0009c16a1a4dff410e36c
JSON Schema Structured Output
(raw) {"ref":"c8d9941797f1469fd5f0009c16a1a4dff410e36c"}
openai/gpt-5.6-luna:low PASS 100% 197 in → 27 out (0 reasoning)
288c32f4deb1a5a21cda6e011698bba512e45917
openai/gpt-5.6-luna:low__json_schema PASS 100% 235 in → 36 out (0 reasoning)
f2a370cf2b8a01462950298baf14ea2af798adfd
JSON Schema Structured Output
(raw) {"ref":"f2a370cf2b8a01462950298baf14ea2af798adfd"}
openai/gpt-5.6-luna:max PASS 100% 200 in → 94 out (65 reasoning)
8e0545ac3336a14224613b4a7b5dce7291109784
openai/gpt-5.6-luna:max__json_schema PASS 100% 235 in → 67 out (28 reasoning)
d65292f350b1bf13c8f03749593a67b84efecfdc
JSON Schema Structured Output
(raw) {"ref":"d65292f350b1bf13c8f03749593a67b84efecfdc"}
openai/gpt-5.6-luna:medium PASS 100% 203 in → 26 out (0 reasoning)
6b803846864a01dcd93826a463f88c8e82b64990
openai/gpt-5.6-luna:medium__json_schema PASS 100% 240 in → 37 out (0 reasoning)
82960a6ed9cb956600f2dd80c8cf27bfb85af98a
JSON Schema Structured Output
(raw) {"ref":"82960a6ed9cb956600f2dd80c8cf27bfb85af98a"}
openai/gpt-5.6-luna:none PASS 100% 199 in → 28 out (0 reasoning)
c6cd0909abdd8bf7165ac4c5d1774678199a7fc3
openai/gpt-5.6-luna:none__json_schema PASS 100% 249 in → 35 out (0 reasoning)
d2cd37044f39819da7ba685bce6cd8faa90237fc
JSON Schema Structured Output
(raw) {"ref":"d2cd37044f39819da7ba685bce6cd8faa90237fc"}
openai/gpt-5.6-luna:xhigh PASS 100% 201 in → 66 out (38 reasoning)
022202ffa1a9eba4d42acf59e86f2d8d14750068
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 243 in → 35 out (0 reasoning)
0f84cee1e5ce870b5cd92603dd832fb54879e678
JSON Schema Structured Output
(raw) {"ref":"0f84cee1e5ce870b5cd92603dd832fb54879e678"}
openai/gpt-5.6-sol:high PASS 100% 202 in → 29 out (0 reasoning)
7a32e0ea2997566e0c9d17c1aa21ca0bac556cfc
openai/gpt-5.6-sol:high__json_schema PASS 100% 236 in → 35 out (0 reasoning)
1751eac691835f82a43f88e626cc900a061925a5
JSON Schema Structured Output
(raw) {"ref":"1751eac691835f82a43f88e626cc900a061925a5"}
openai/gpt-5.6-sol:low PASS 100% 199 in → 28 out (0 reasoning)
bfeb670c7b793d17553c5771c042ae6b80e4b93a
openai/gpt-5.6-sol:low__json_schema PASS 100% 237 in → 38 out (0 reasoning)
942a46ea6c2f5e1251e07100fea6fb4301f97e05
JSON Schema Structured Output
(raw) {"ref":"942a46ea6c2f5e1251e07100fea6fb4301f97e05"}
openai/gpt-5.6-sol:max PASS 100% 201 in → 23 out (0 reasoning)
859b8945e4081d0511658e40945392473eee1390
openai/gpt-5.6-sol:max__json_schema PASS 100% 233 in → 33 out (0 reasoning)
6e031716271a642f0dfa95004cf315fbc850612d
JSON Schema Structured Output
(raw) {"ref":"6e031716271a642f0dfa95004cf315fbc850612d"}
openai/gpt-5.6-sol:medium PASS 100% 209 in → 28 out (0 reasoning)
62eb5c56e56b92e11da186f4d5fc36f5ef624323
openai/gpt-5.6-sol:medium__json_schema PASS 100% 228 in → 34 out (0 reasoning)
85672951a03303c25ae642a8758308c4c74fd210
JSON Schema Structured Output
(raw) {"ref":"85672951a03303c25ae642a8758308c4c74fd210"}
openai/gpt-5.6-sol:none PASS 100% 199 in → 26 out (0 reasoning)
417b1aa98ffe8924ba4684775fe1fb1e3fbe1eba
openai/gpt-5.6-sol:none__json_schema PASS 100% 237 in → 35 out (0 reasoning)
779c795eb3ed2103fc3ca9dc143dc02e642c7307
JSON Schema Structured Output
(raw) {"ref":"779c795eb3ed2103fc3ca9dc143dc02e642c7307"}
openai/gpt-5.6-sol:xhigh PASS 100% 201 in → 30 out (0 reasoning)
dbae917c912e0e34201e09d4e7d21b5d7b10660c
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 235 in → 34 out (0 reasoning)
80619c0ef600ca882105746bfcc15e59f882c30d
JSON Schema Structured Output
(raw) {"ref":"80619c0ef600ca882105746bfcc15e59f882c30d"}
openai/gpt-5.6-terra:high PASS 100% 215 in → 29 out (0 reasoning)
d8e0da33fd26acf8d765db53e5e7f24153e7da88
openai/gpt-5.6-terra:high__json_schema PASS 100% 243 in → 36 out (0 reasoning)
5e76fc3af19502af8e887334688578ca2f465a8c
JSON Schema Structured Output
(raw) {"ref":"5e76fc3af19502af8e887334688578ca2f465a8c"}
openai/gpt-5.6-terra:low PASS 100% 207 in → 26 out (0 reasoning)
d409361ecad7a4b1946b121e036a0ff562533fe5
openai/gpt-5.6-terra:low__json_schema PASS 100% 238 in → 37 out (0 reasoning)
60e216f0847cc7d4ed146508d892b5b69edc4686
JSON Schema Structured Output
(raw) {"ref":"60e216f0847cc7d4ed146508d892b5b69edc4686"}
openai/gpt-5.6-terra:max PASS 100% 203 in → 27 out (0 reasoning)
0f11fe4111289a5c76dc41f996fa5db4fb40224c
openai/gpt-5.6-terra:max__json_schema PASS 100% 244 in → 36 out (0 reasoning)
aeaa040c026de7db768f0840e47dc38ee2d326b0
JSON Schema Structured Output
(raw) {"ref":"aeaa040c026de7db768f0840e47dc38ee2d326b0"}
openai/gpt-5.6-terra:medium PASS 100% 198 in → 26 out (0 reasoning)
1e1bb215aabc761e9becb3f7504eb071f21abfac
openai/gpt-5.6-terra:medium__json_schema PASS 100% 232 in → 36 out (0 reasoning)
1f021995d6987afd8f22160de11cf7c23ebfdf32
JSON Schema Structured Output
(raw) {"ref":"1f021995d6987afd8f22160de11cf7c23ebfdf32"}
openai/gpt-5.6-terra:none PASS 100% 199 in → 27 out (0 reasoning)
d784b1973883d1b9cda951e0e47891ae642d366b
openai/gpt-5.6-terra:none__json_schema PASS 100% 244 in → 39 out (0 reasoning)
d90a3b19042e1cb2c6c068fff00d2c321d14bee5
JSON Schema Structured Output
(raw) {"ref":"d90a3b19042e1cb2c6c068fff00d2c321d14bee5"}
openai/gpt-5.6-terra:xhigh PASS 100% 207 in → 27 out (0 reasoning)
3f057cc4e77efdceabf62cbbaf32df66e9979bfa
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 239 in → 38 out (0 reasoning)
b0f6324c99eebda82cce1213237e132cc14aa9e9
JSON Schema Structured Output
(raw) {"ref":"b0f6324c99eebda82cce1213237e132cc14aa9e9"}
openai/gpt-oss-120b:high PASS 100% 243 in → 518 out (435 reasoning)
70830ca043ca330e954a47236481b1e552f36900
openai/gpt-oss-120b:high__json_schema PASS 100% 268 in → 1,035 out (803 reasoning)
6b04d2106d64f5ce3f5ff86d3f39a35d38260b1f
JSON Schema Structured Output
(raw) {"ref":"6b04d2106d64f5ce3f5ff86d3f39a35d38260b1f"}
openai/gpt-oss-120b:low PASS 100% 257 in → 93 out (39 reasoning)
161f0b3d81d2e2d8ffc8c37a4ff96eac2753e238
openai/gpt-oss-120b:low__json_schema PASS 100% 268 in → 100 out (37 reasoning)
b67a4c7b8c2599dc76c293ad0f78e645c84a1eaa
JSON Schema Structured Output
(raw) { "ref": "b67a4c7b8c2599dc76c293ad0f78e645c84a1eaa" }
openai/gpt-oss-120b:medium PASS 100% 262 in → 483 out (353 reasoning)
a7fc50608083d72f320ef4afe194b8e1ee301bcf
openai/gpt-oss-120b:medium__json_schema PASS 100% 275 in → 370 out (261 reasoning)
bdfe3e5f83bf3ef5d2346fca99dd17205adeae00
JSON Schema Structured Output
(raw) { "ref": "bdfe3e5f83bf3ef5d2346fca99dd17205adeae00" }
openai/gpt-oss-120b:xhigh PASS 100% 261 in → 212 out (189 reasoning)
794af16f9bb703bf0c4eceeea71b0d1441eed32b
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 260 in → 556 out (400 reasoning)
e0518425f1e018d6429ffee2b1922879d0582603
JSON Schema Structured Output
(raw) { "ref": "e0518425f1e018d6429ffee2b1922879d0582603" }
openai/gpt-oss-20b:high PASS 100% 246 in → 616 out (526 reasoning)
d0aca6b598f9227245ac49299aefc12cd8f166e7
openai/gpt-oss-20b:low PASS 100% 251 in → 82 out (36 reasoning)
0f1d5f43af321d8e342061932af41dd24cc60a15
openai/gpt-oss-20b:low__json_schema PASS 100% 315 in → 82 out (26 reasoning)
57246184590d22fe45b8d4d43536a92030d1f6f6
JSON Schema Structured Output
(raw) {"ref":"57246184590d22fe45b8d4d43536a92030d1f6f6"}
openai/gpt-oss-20b:medium PASS 100% 260 in → 235 out (180 reasoning)
972c1d2dc68bff5bbec8d2911556bd055d3969f2
openai/gpt-oss-20b:xhigh PASS 100% 267 in → 266 out (239 reasoning)
e4c1a6edb8ed9d3e523796f4704d4c5de0258bad
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 266 in → 559 out (514 reasoning)
5abceff5c10d11e04bcc07702d549cba6cd93ade
JSON Schema Structured Output
(raw) { "ref" : "5abceff5c10d11e04bcc07702d549cba6cd93ade" }
poolside/laguna-m.1:high PASS 100% 246 in → 1,366 out (1,326 reasoning)
6905838e5979515375ba51fd82e8784043ea4a77
poolside/laguna-m.1:high__json_schema PASS 100% 252 in → 345 out (299 reasoning)
ddef71352c6c6b4c7409fb3f131ba975d5fe17e6
JSON Schema Structured Output
(raw) { "ref": "ddef71352c6c6b4c7409fb3f131ba975d5fe17e6" }
poolside/laguna-m.1:low PASS 100% 248 in → 661 out (624 reasoning)
13ed818dfe3e00d5a7aa53ceb748d9c96275601e
poolside/laguna-m.1:low__json_schema PASS 100% 254 in → 455 out (410 reasoning)
71cc20baa9ec8b39d65ec617d2dbb3c6e2ff7224
JSON Schema Structured Output
(raw) { "ref": "71cc20baa9ec8b39d65ec617d2dbb3c6e2ff7224" }
poolside/laguna-m.1:medium PASS 100% 243 in → 288 out (252 reasoning)
1fb24fe68ba62f98040dd6457becebc4c120c11a
poolside/laguna-m.1:none PASS 100% 247 in → 34 out (0 reasoning)
fb9dfaf4d3e768a806aa91bba8a673c19e959dfb
poolside/laguna-m.1:xhigh PASS 100% 260 in → 371 out (329 reasoning)
e45e2658f47e3d324859d27988bc9e69775e3c56
poolside/laguna-m.1:xhigh__json_schema PASS 100% 262 in → 391 out (343 reasoning)
1a7d6fd8f9db39582348eac224a4e7297986e85d
JSON Schema Structured Output
(raw) { "ref": "1a7d6fd8f9db39582348eac224a4e7297986e85d" }
poolside/laguna-xs-2.1:high PASS 100% 260 in → 991 out (953 reasoning)
b9830934fc82916b2686c7d560fe75d132148bb1
poolside/laguna-xs-2.1:high__json_schema PASS 100% 253 in → 636 out (589 reasoning)
520044c30660d5ae18611cf82728f0dffde51516
JSON Schema Structured Output
(raw) { "ref": "520044c30660d5ae18611cf82728f0dffde51516" }
poolside/laguna-xs-2.1:low PASS 100% 246 in → 524 out (489 reasoning)
68221ab48acb2d672a05892eaec601473745dc32
poolside/laguna-xs-2.1:low__json_schema PASS 100% 254 in → 473 out (428 reasoning)
0031b54d5ca3cb65e1fdf4cb87560098688a0bcf
JSON Schema Structured Output
(raw) { "ref": "0031b54d5ca3cb65e1fdf4cb87560098688a0bcf" }
poolside/laguna-xs-2.1:medium PASS 100% 244 in → 726 out (691 reasoning)
6d4bbc8c0757ae586257dd644d9e7c902932caa8
poolside/laguna-xs-2.1:none PASS 100% 249 in → 35 out (0 reasoning)
e6c1d28097217863e1cfe8c7e91a4a597fbcda20
poolside/laguna-xs-2.1:none__json_schema PASS 100% 255 in → 42 out (0 reasoning)
0862bec6ff22fde64f02aa3f70bb7e7ef7f3ca51
JSON Schema Structured Output
(raw) { "ref": "0862bec6ff22fde64f02aa3f70bb7e7ef7f3ca51" }
poolside/laguna-xs-2.1:xhigh PASS 100% 253 in → 500 out (464 reasoning)
748335c85d1bb03480669efae3e1aa8f952ef581
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 258 in → 460 out (417 reasoning)
16cefe2ee82fdc6525b0537fe1028ecceb1d480f
JSON Schema Structured Output
(raw) { "ref": "16cefe2ee82fdc6525b0537fe1028ecceb1d480f" }
poolside/laguna-xs.2:high PASS 100% 287 in → 435 out (397 reasoning)
7743632b7d74107901afc8e854edfae5a0e34b44
poolside/laguna-xs.2:high__json_schema PASS 100% 294 in → 344 out (249 reasoning)
git reset --soft HEAD~1 moved HEAD back one commit, so the commit that was just reset away is the one that was previously at HEAD@{1}, which corresponds to the commit 'Add third version - important changes' with hash 51898e251412f347a69a4aee18943e3c753e21e7
JSON Schema Structured Output
(raw) {"ref": "git reset --soft HEAD~1 moved HEAD back one commit, so the commit that was just reset away is the one that was previously at HEAD@{1}, which corresponds to the commit 'Add third version - important changes' with hash 51898e251412f347a69a4aee18943e3c753e21e7"}
poolside/laguna-xs.2:low PASS 100% 290 in → 300 out (265 reasoning)
53e1876befff3f1a0df648ca4fc4e361accb8247
poolside/laguna-xs.2:medium PASS 100% 283 in → 261 out (223 reasoning)
16c3b49af2eebf4b527b961fb1d09c0b2cd9b623
poolside/laguna-xs.2:medium__json_schema PASS 100% 289 in → 301 out (257 reasoning)
a2e139ec4733f3830a42debae65d68b939a06e29
JSON Schema Structured Output
(raw) {"ref": "a2e139ec4733f3830a42debae65d68b939a06e29"}
poolside/laguna-xs.2:none PASS 100% 301 in → 37 out (0 reasoning)
d07ba60d7198cf73245162fe827dcf40230b29f2
poolside/laguna-xs.2:none__json_schema PASS 100% 296 in → 42 out (0 reasoning)
4b15205d3616f64fcd27a8269e222bec5819fb8e
JSON Schema Structured Output
(raw) {"ref": "4b15205d3616f64fcd27a8269e222bec5819fb8e"}
poolside/laguna-xs.2:xhigh PASS 100% 295 in → 217 out (180 reasoning)
ec290884c25fefff6e7986e14909716ac14a9cd0
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 280 in → 410 out (362 reasoning)
048592ddfc63e281b94e214796a4cb23ef47c20b
JSON Schema Structured Output
(raw) { "ref": "048592ddfc63e281b94e214796a4cb23ef47c20b" }
qwen/qwen3.6-27b:high PASS 100% 256 in → 1,488 out (968 reasoning)
7c8b8fa65963c528d8467dc56afb1599e2a014dc
qwen/qwen3.6-27b:high__json_schema PASS 100% 260 in → 45 out (0 reasoning)
3471e227a73d79f1ec9bcb150c7dd0cadd7f8d0f
JSON Schema Structured Output
(raw) { "ref": "3471e227a73d79f1ec9bcb150c7dd0cadd7f8d0f" }
qwen/qwen3.6-27b:low PASS 100% 264 in → 324 out (188 reasoning)
43a940c8166134d5bc3e2ba338969b09710848f3
qwen/qwen3.6-27b:low__json_schema PASS 100% 243 in → 1,363 out (868 reasoning)
35340277ce50dd9742e5e589571251a98aaa5ead
JSON Schema Structured Output
(raw) { "ref": "35340277ce50dd9742e5e589571251a98aaa5ead" }
qwen/qwen3.6-27b:medium PASS 100% 267 in → 1,604 out (1,011 reasoning)
ba8e2a7c9d87ced7ddc9a9f16ef0799210b233e6
qwen/qwen3.6-27b:medium__json_schema PASS 100% 253 in → 1,195 out (829 reasoning)
06dbe0bbf7696df6ce13bac92edad11b87cf1d23
JSON Schema Structured Output
(raw) { "ref": "06dbe0bbf7696df6ce13bac92edad11b87cf1d23" }
qwen/qwen3.6-27b:none PASS 100% 258 in → 35 out (0 reasoning)
1323ab9ee9394c85386d59706cee3ac02bf8b31d
qwen/qwen3.6-27b:none__json_schema PASS 100% 265 in → 44 out (0 reasoning)
762185bd690a473bae54f86c916e5ecde072a5a2
JSON Schema Structured Output
(raw) {"ref": "762185bd690a473bae54f86c916e5ecde072a5a2"}
qwen/qwen3.6-27b:xhigh PASS 100% 262 in → 2,094 out (1,327 reasoning)
44b423f5acb1be0b70f8eb4b155c3d83116a2b9e
qwen/qwen3.6-35b-a3b:high PASS 100% 259 in → 1,193 out (1,154 reasoning)
495d7c3f4cba66f38a5d6bc89725eb70eeb033dd
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 247 in → 1,762 out (1,193 reasoning)
95edddb81ca2d5199db69ca6ee57105d7bcbf170
JSON Schema Structured Output
(raw) { "ref": "95edddb81ca2d5199db69ca6ee57105d7bcbf170" }
qwen/qwen3.6-35b-a3b:low PASS 100% 259 in → 1,361 out (1,319 reasoning)
3b2efc7e769c3395179ce97187331e5802fd5d9f
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 252 in → 1,448 out (947 reasoning)
d9671ba9dedaa935cf99924e8884913dedec2f9a
JSON Schema Structured Output
(raw) { "ref": "d9671ba9dedaa935cf99924e8884913dedec2f9a" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 265 in → 370 out (232 reasoning)
1d052561b18877884de926d60a674999386d47e5
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 256 in → 1,672 out (1,103 reasoning)
0262b75b97f16b95e7e9e82e6baa9b3aa6c5fb8c
JSON Schema Structured Output
(raw) { "ref": "0262b75b97f16b95e7e9e82e6baa9b3aa6c5fb8c" }
qwen/qwen3.6-35b-a3b:none PASS 100% 257 in → 36 out (0 reasoning)
9bf747cca5e5f1563a40564c5c70c485c027bb1a
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 264 in → 290 out (189 reasoning)
f240123483b995218f991642f244b4e6f6b203a3
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 253 in → 1,448 out (979 reasoning)
a6b1f8fc1fae881dd443fe0c58d135f23d16bf5a
JSON Schema Structured Output
(raw) { "ref": "a6b1f8fc1fae881dd443fe0c58d135f23d16bf5a" }
qwen/qwen3.6-flash:high PASS 100% 261 in → 1,528 out (1,487 reasoning)
b09dc363f9f8ba935f9976c230060b11294ebb72
qwen/qwen3.6-flash:low PASS 100% 260 in → 1,451 out (1,411 reasoning)
c80001a969d0836ed5218473af5b5bf15b35acc0
qwen/qwen3.6-flash:medium PASS 100% 246 in → 1,376 out (1,340 reasoning)
5f0862be6fca65c67b4a2cec521cba85ce20ebae
qwen/qwen3.6-flash:none PASS 100% 259 in → 34 out (0 reasoning)
201d6ab0594aa00f0aff072f88b0bb2c5fbf2104
qwen/qwen3.6-flash:xhigh PASS 100% 258 in → 1,232 out (1,192 reasoning)
762cb817e9444c322fd8f16a0de8cb720f47af1c
qwen/qwen3.7-flash:high PASS 100% 258 in → 787 out (748 reasoning)
7e44dc62be5a6e451bc4015ce740ffce3c65a37b
qwen/qwen3.7-flash:low PASS 100% 261 in → 1,519 out (1,476 reasoning)
b9ba0d8615fc35c512f4b8251e987a33f82347a5
qwen/qwen3.7-flash:max PASS 100% 259 in → 1,270 out (1,228 reasoning)
f573d1ca1b867a7b5ee1d3e635ebc38c84381d54
qwen/qwen3.7-flash:medium PASS 100% 264 in → 861 out (817 reasoning)
987a3e41c80d00d5e86c310a49ac44a1b41835d0
qwen/qwen3.7-flash:none PASS 100% 257 in → 36 out (0 reasoning)
9eac83421337cee70f8b6575b3f8a774b7520cb5
qwen/qwen3.7-flash:xhigh PASS 100% 258 in → 1,039 out (1,000 reasoning)
11357e83047a8e9fe5cb553e38992eadba8fd42a
qwen/qwen3.7-max:high PASS 100% 254 in → 400 out (362 reasoning)
5818affcd590a3c0bb9e52d32da5a9bcf91ec655
qwen/qwen3.7-max:low PASS 100% 261 in → 356 out (317 reasoning)
67823a757ae43f32c84eae2cb9b26cb4924cfddc
qwen/qwen3.7-max:medium PASS 100% 254 in → 487 out (444 reasoning)
0338f899d58a92281ca0943130bcf7d7d0932059
qwen/qwen3.7-max:none PASS 100% 255 in → 36 out (0 reasoning)
c8f1f5a66323afdfb818de58791667b5cb081a0a
qwen/qwen3.7-max:xhigh PASS 100% 248 in → 477 out (440 reasoning)
bf0afe59b3ccc8f037ade98a3f58e5b20ad22743
qwen/qwen3.7-plus:high PASS 100% 265 in → 384 out (340 reasoning)
61814d4321fe0d6f0672702f766480b660e77a0e
qwen/qwen3.7-plus:low PASS 100% 252 in → 258 out (219 reasoning)
e51cd46f8af7ce4bf87fc4395fbc236e62e53369
qwen/qwen3.7-plus:medium PASS 100% 258 in → 410 out (370 reasoning)
9e3a85dfa38077ca79d5716a8b03ac3ff5f84f15
qwen/qwen3.7-plus:none PASS 100% 261 in → 34 out (0 reasoning)
a49602a3dd791c64b41015de7dac8fe12a03e8ef
qwen/qwen3.7-plus:xhigh PASS 100% 260 in → 467 out (427 reasoning)
195000c0ac7151a0b40783e81ccc824a34c9ada9
tencent/hy3:high PASS 100% 207 in → 327 out (301 reasoning)
ea103ad5c0bbebc18fa7a7edab08baf5bd188b0b
tencent/hy3:high__json_schema PASS 100% 202 in → 467 out (314 reasoning)
1b2b8480b0abe92ecfb9851ad15d30001f98c6d7
JSON Schema Structured Output
(raw) {"ref":"1b2b8480b0abe92ecfb9851ad15d30001f98c6d7"}
tencent/hy3:low PASS 100% 203 in → 292 out (264 reasoning)
1a7b9f16764f4cccde5d42a1cf37f1f557cf6077
tencent/hy3:low__json_schema PASS 100% 204 in → 350 out (234 reasoning)
3ed026853f7acd31cf22bb5afb8f3dd5bfece8b4
JSON Schema Structured Output
(raw) { "ref": "3ed026853f7acd31cf22bb5afb8f3dd5bfece8b4" }
tencent/hy3:medium PASS 100% 212 in → 292 out (267 reasoning)
ad01a6ea9defb48bf8343c7fb4993fb134b0af99
tencent/hy3:medium__json_schema PASS 100% 206 in → 514 out (305 reasoning)
3f5b33903e00819b7b42b98dd173d0bdd8114075
JSON Schema Structured Output
(raw) { "ref": "3f5b33903e00819b7b42b98dd173d0bdd8114075" }
tencent/hy3:none PASS 100% 207 in → 23 out (0 reasoning)
0a235062ba070202aaf5e30e32342e7924c4677f
tencent/hy3:xhigh PASS 100% 202 in → 392 out (364 reasoning)
66f5fc52d5110b64e8e58a1476fd2fdb91584d8f
tencent/hy3:xhigh__json_schema PASS 100% 209 in → 339 out (202 reasoning)
41d72eefd7a7bca04ca4c2dd9dbe25d69bcf0ce1
JSON Schema Structured Output
(raw) { "ref": "41d72eefd7a7bca04ca4c2dd9dbe25d69bcf0ce1" }
thinkingmachines/inkling-small:high PASS 100% 210 in → 190 out (162 reasoning)
f4d996b9473dc0d1c48c36e208beada442eee366
thinkingmachines/inkling-small:high__json_schema PASS 100% 206 in → 201 out (169 reasoning)
02c701f8f0104bcd8306b0501d25cce213f15566
JSON Schema Structured Output
(raw) {"ref":"02c701f8f0104bcd8306b0501d25cce213f15566"}
thinkingmachines/inkling-small:low PASS 100% 213 in → 75 out (45 reasoning)
c8fd165076e6923f7ba6cedd516c0b2922ed6a74
thinkingmachines/inkling-small:low__json_schema PASS 100% 210 in → 76 out (43 reasoning)
d9c361c7269564cc83e72672f629b658fe240e10
JSON Schema Structured Output
(raw) {"ref": "d9c361c7269564cc83e72672f629b658fe240e10"}
thinkingmachines/inkling-small:max PASS 100% 216 in → 246 out (153 reasoning)
e1c2ac48163b2d0f18bef9a8e17cbc9675357d9f
thinkingmachines/inkling-small:max__json_schema PASS 100% 206 in → 293 out (256 reasoning)
0f60a2444a48231a84f6232bcf54b88ad9aa629d
JSON Schema Structured Output
(raw) { "ref": "0f60a2444a48231a84f6232bcf54b88ad9aa629d" }
thinkingmachines/inkling-small:medium PASS 100% 214 in → 73 out (45 reasoning)
72c473162097e39aea52e50a6128a7173a8072e3
thinkingmachines/inkling-small:medium__json_schema PASS 100% 207 in → 171 out (135 reasoning)
fa6c9188e4bbee1384c1883d721be0ea92398ea8
JSON Schema Structured Output
(raw) { "ref": "fa6c9188e4bbee1384c1883d721be0ea92398ea8" }
thinkingmachines/inkling-small:none PASS 100% 209 in → 27 out (0 reasoning)
cc9903a9d290304dc9ad7e6af40c69c409c6be17<|end_message|>
thinkingmachines/inkling-small:xhigh PASS 100% 214 in → 346 out (318 reasoning)
d65a3d9cd86b8272b071603cae32c63033414464
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 199 in → 292 out (257 reasoning)
74be67fe1bc92d98db305f748be7411b0fb56ddc
JSON Schema Structured Output
(raw) { "ref": "74be67fe1bc92d98db305f748be7411b0fb56ddc" }
thinkingmachines/inkling:high PASS 100% 210 in → 215 out (156 reasoning)
b88845b290560ed910ff5fdce203d3d952adafce
thinkingmachines/inkling:high__json_schema PASS 100% 215 in → 237 out (198 reasoning)
44e0ced40d37a1e8a972ded0c9a6dce651c0057b
JSON Schema Structured Output
(raw) { "ref": "44e0ced40d37a1e8a972ded0c9a6dce651c0057b" }
thinkingmachines/inkling:low PASS 100% 204 in → 67 out (42 reasoning)
8044a7a45714a699fd55336ad11139802acc416c
thinkingmachines/inkling:max PASS 100% 206 in → 221 out (151 reasoning)
09f1e6a183352ec67f1e0acaf86882a4ebba7a7f
thinkingmachines/inkling:max__json_schema PASS 100% 208 in → 353 out (314 reasoning)
692866c70a1d8992d82cc179b5d70c1c34f93bb0
JSON Schema Structured Output
(raw) { "ref": "692866c70a1d8992d82cc179b5d70c1c34f93bb0" }
thinkingmachines/inkling:medium PASS 100% 203 in → 221 out (190 reasoning)
6a179823ae341a42f0c1fec179dd86b9af2b27d5
thinkingmachines/inkling:none PASS 100% 207 in → 23 out (0 reasoning)
1744024464e8e7f6114463335253ebee65218b27<|end_message|>
thinkingmachines/inkling:xhigh PASS 100% 217 in → 292 out (259 reasoning)
ce0779a47f2785c7714ad2a9d2b4eeef7ad719a7
thinkingmachines/inkling:xhigh__json_schema PASS 100% 218 in → 115 out (79 reasoning)
14081af53faf9ab5f400ae4bc09b8eb3139c7ef6
JSON Schema Structured Output
(raw) { "ref": "14081af53faf9ab5f400ae4bc09b8eb3139c7ef6" }
x-ai/grok-4.3:high PASS 100% 373 in → 500 out (478 reasoning)
4a93ed1e4ffed35aec1863349311e07926d8ecc0
x-ai/grok-4.3:high__json_schema PASS 100% 432 in → 1,035 out (1,027 reasoning)
HEAD@{1}
JSON Schema Structured Output
(raw) {"ref":"HEAD@{1}"}
x-ai/grok-4.3:low PASS 100% 380 in → 800 out (656 reasoning)
62b42d8e6efeaa3676b165d68841e3e6532c54ed The reflog shows the sequence of HEAD movements (newest first). The reset action recorded at HEAD@{0} moved HEAD to the parent of the prior tip. The immediately preceding entry (HEAD@{1}) records the commit that was at HEAD just before the reset occurred; that is exactly the commit removed from the branch history. Its parent is the commit now shown at both HEAD@{0} and HEAD@{2}, confirming the one-commit rewind described by the `reset --soft HEAD~1` command. The 40-character SHA-1 from HEAD@{1} is therefore the answer.
x-ai/grok-4.3:low__json_schema PASS 100% 427 in → 745 out (717 reasoning)
fd6ff5750fc53b8ce9e58384bd63abc68c3d4f22
JSON Schema Structured Output
(raw) {"ref":"fd6ff5750fc53b8ce9e58384bd63abc68c3d4f22"}
x-ai/grok-4.3:max PASS 100% 370 in → 397 out (375 reasoning)
f47dd65861ad25a1a4ccb9686f1bd28232cfc0ff
x-ai/grok-4.3:max__json_schema PASS 100% 433 in → 1,367 out (1,338 reasoning)
d90a3b19042e1cb2c6c068fff00d2c321d14bee5
JSON Schema Structured Output
(raw) {"ref":"d90a3b19042e1cb2c6c068fff00d2c321d14bee5"}
x-ai/grok-4.3:medium PASS 100% 377 in → 1,268 out (1,053 reasoning)
4d55cf75aed74ab0373a2f700c27f3575bd0c6a6 The reflog records HEAD movement in reverse chronological order. The most recent entry (HEAD@{0}) shows the reset action and the resulting position 62daf89f694a845b631a663407ec31bf502bb85b. The immediately preceding entry (HEAD@{1}) shows the commit that was at HEAD right before the reset (4d55cf75aed74ab0373a2f700c27f3575bd0c6a6). Because the performed command was `reset --soft HEAD~1`, the position before the reset must have been exactly one commit beyond 62daf89f694a845b631a663407ec31bf502bb85b; the only commit that satisfies this relationship is 4d55cf75aed74ab0373a2f700c27f3575bd0c6a6. This is therefore the commit that was “reset away.”
x-ai/grok-4.3:medium__json_schema PASS 100% 442 in → 1,594 out (1,586 reasoning)
HEAD@{1}
JSON Schema Structured Output
(raw) {"ref":"HEAD@{1}"}
x-ai/grok-4.3:none PASS 100% 373 in → 20 out (0 reasoning)
45503023f760b6b8519f010ab201547670fd6c33
x-ai/grok-4.3:xhigh PASS 100% 379 in → 651 out (629 reasoning)
60523e22a6bc5e541747c7dd6f3dec895a953cac
x-ai/grok-4.3:xhigh__json_schema PASS 100% 432 in → 2,393 out (2,365 reasoning)
eea07c14a9b2a1196bc9bbefde29a12570f8315b
JSON Schema Structured Output
(raw) {"ref":"eea07c14a9b2a1196bc9bbefde29a12570f8315b"}
x-ai/grok-4.5:high PASS 100% 395 in → 415 out (395 reasoning)
740190c35c000b943b932c7284771a415cd22ab1
x-ai/grok-4.5:high__json_schema PASS 100% 466 in → 672 out (645 reasoning)
8458947f56f6a3dc7117376dc54f45ac1a826bbb
JSON Schema Structured Output
(raw) {"ref":"8458947f56f6a3dc7117376dc54f45ac1a826bbb"}
x-ai/grok-4.5:low PASS 100% 403 in → 308 out (285 reasoning)
c38251d2bd4faaf522bc437e25a557a5eeba2cd9
x-ai/grok-4.5:low__json_schema PASS 100% 462 in → 219 out (194 reasoning)
15d45fd4afc6425aa107d86f601e00428859acca
JSON Schema Structured Output
(raw) {"ref":"15d45fd4afc6425aa107d86f601e00428859acca"}
x-ai/grok-4.5:max PASS 100% 402 in → 457 out (433 reasoning)
7afaf6b5894d66fa330d8c24baed3a54b55156cc
x-ai/grok-4.5:max__json_schema PASS 100% 466 in → 674 out (647 reasoning)
955fb5502c47dff456abc8e72c54f22f7da6dc82
JSON Schema Structured Output
(raw) {"ref":"955fb5502c47dff456abc8e72c54f22f7da6dc82"}
x-ai/grok-4.5:medium PASS 100% 392 in → 410 out (390 reasoning)
6bb196933d6439ada149db9434d5b197933da41f
x-ai/grok-4.5:medium__json_schema PASS 100% 464 in → 538 out (510 reasoning)
28d6f9ae92c861bb921611e5e99ebd9d07194df8
JSON Schema Structured Output
(raw) {"ref":"28d6f9ae92c861bb921611e5e99ebd9d07194df8"}
x-ai/grok-4.5:xhigh PASS 100% 398 in → 475 out (451 reasoning)
7f3004e12284df9bfa1af2623782df4196a1a4fc
x-ai/grok-4.5:xhigh__json_schema PASS 100% 473 in → 486 out (457 reasoning)
b7ca7657324bdf486fca89a280cf282ba1324e35
JSON Schema Structured Output
(raw) { "ref": "b7ca7657324bdf486fca89a280cf282ba1324e35" }
z-ai/glm-4.7-flash:low PASS 100% 223 in → 684 out (526 reasoning)
f233979e01dffa8ce0e177ef2002646d2829be4e
z-ai/glm-4.7-flash:medium PASS 100% 217 in → 690 out (549 reasoning)
c8ea890a4e0074f37cd00b1b1af8a1ebac8c723f
z-ai/glm-4.7-flash:none PASS 100% 212 in → 26 out (0 reasoning)
f0da1eacd93c965ecc20a00fa4be458a5c672916
z-ai/glm-4.7-flash:xhigh PASS 100% 212 in → 2,730 out (2,212 reasoning)
75dd3fc07841f54427cc0f8271434df8cd6409f1
z-ai/glm-5.2:high PASS 100% 215 in → 291 out (264 reasoning)
74de608a56a47c290f930a0c70c293104d77c65b
z-ai/glm-5.2:high__json_schema PASS 100% 224 in → 1,460 out (1,427 reasoning)
4a5cb130aa59eed786c964f28d6d72bf80d8de4c
JSON Schema Structured Output
(raw) {"ref": "4a5cb130aa59eed786c964f28d6d72bf80d8de4c"}
z-ai/glm-5.2:low PASS 100% 212 in → 141 out (114 reasoning)
2ef7c7a7604dfa85af74486dcac1daa83ebaf172
z-ai/glm-5.2:medium PASS 100% 220 in → 222 out (150 reasoning)
02cfc0fd750fbdbfac0ee2b8c0e607dd39c2e2b6
z-ai/glm-5.2:medium__json_schema PASS 100% 213 in → 335 out (242 reasoning)
8c62af23c150ea7cab7e102460a1926fa8e22bf5
JSON Schema Structured Output
(raw) { "ref": "8c62af23c150ea7cab7e102460a1926fa8e22bf5" }
z-ai/glm-5.2:none PASS 100% 215 in → 29 out (0 reasoning)
94f4d5df777071e6b58a67a398782e638e110ef4
z-ai/glm-5.2:none__json_schema PASS 100% 216 in → 32 out (0 reasoning)
1cb45a9ba77eb3ceacbd2bc907f699fb03eaf4bf
JSON Schema Structured Output
(raw) { "ref": "1cb45a9ba77eb3ceacbd2bc907f699fb03eaf4bf" }
z-ai/glm-5.2:xhigh PASS 100% 233 in → 781 out (502 reasoning)
baa4f8da5119ae6eb0c52ead5e4cd2b5fe58a2bf
z-ai/glm-5.2:xhigh__json_schema PASS 100% 224 in → 635 out (598 reasoning)
54447150c599f6e2a69b530764d156b0b447d5fe
JSON Schema Structured Output
(raw) { "ref": "54447150c599f6e2a69b530764d156b0b447d5fe" }
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
arcee-ai/trinity-mini:high__json_schema FAIL 0% 210 in → 551 out (496 reasoning)
da4b0f66d8520e2fb28a4e5e64d1ecd6337a2a81
JSON Schema Structured Output
(raw) { "ref": "da4b0f66d8520e2fb28a4e5e64d1ecd6337a2a81" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
deepseek/deepseek-v4-flash-0731:none__json_schema FAIL 0% 196 in → 13 out (0 reasoning)
refs/heads/main
JSON Schema Structured Output
(raw) { "ref": "refs/heads/main" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output: 87ecf8ca82bca601aaac48fd928882b51d4c93e1
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 3 (char 2)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 3 (char 2)
deepseek/deepseek-v4-flash:none__json_schema FAIL 0% 203 in → 33 out (0 reasoning)
564cf0bc930badd8b09f1409e0a3ec281bf6b6e0
JSON Schema Structured Output
(raw) { "ref": "564cf0bc930badd8b09f1409e0a3ec281bf6b6e0" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output: 613b15e3a227d7ec097e3fd20fed930256a8d268
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 4 (char 3)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 4 (char 3)
ibm-granite/granite-4.1-8b FAIL 0% 206 in → 24 out
0ab11b454b4d20801544113dc40a7aabf85f6b610
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
51ef95b5eae24a869c59dd9f3d107b64292fb83d
JSON Schema Structured Output
(raw) { "ref": "51ef95b5eae24a869c59dd9f3d107b64292fb83d" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
gentleman/do_reflog_analytics
JSON Schema Structured Output
(raw) { "ref": "gentleman/do_reflog_analytics" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
liquid/lfm-2-24b-a2b FAIL 0% 198 in → 28 out
The commit hash you reset away is: b02df364208ee9879066361560132da392942e6c
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output: The commit hash you reset away is: **fe390f80ab4145fe0848492a923578a34c0ff7f9**
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
minimax/minimax-m2.5:xhigh FAIL 0% 232 in → 2,728 out (2,703 reasoning)
0d3423101c70b91d38a7f49860dcf0427d3fcc
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
minimax/minimax-m3:high__json_schema FAIL 0% 368 in → 36 out (0 reasoning)
8dcd9f020be1ee7cc9bd2a8b4a54027af2fd8be8
JSON Schema Structured Output
(raw) { "ref": "8dcd9f020be1ee7cc9bd2a8b4a54027af2fd8be8" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
minimax/minimax-m3:low__json_schema FAIL 0% 353 in → 13 out (0 reasoning)
d5b13fb
JSON Schema Structured Output
(raw) { "ref": "d5b13fb" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
moonshotai/kimi-k2.7-code:xhigh__json_schema FAIL 0% 195 in → 77 out (43 reasoning)
Strict instructions: Provide ONLY the 40-character commit hash. So output just that.
JSON Schema Structured Output
(raw) {"ref": "Strict instructions: Provide ONLY the 40-character commit hash. So output just that."}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
nvidia/nemotron-3-super-120b-a12b:medium__json_schema FAIL 0% 261 in → 525 out (450 reasoning)
(empty output)
JSON Schema Structured Output
(raw) { "ref": "" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
openai/gpt-5.4-nano:high FAIL 0% 209 in → 32 out (0 reasoning)
d65836c7ea2979d7072bb4f7d4c69b0b9ac8b5ad
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
openai/gpt-oss-20b:high__json_schema FAIL 0% 268 in → 10 out (0 reasoning)
13
JSON Schema Structured Output
(raw) { "ref" : "13"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
openai/gpt-oss-20b:medium__json_schema FAIL 0% 267 in → 6 out (0 reasoning)
analysis
JSON Schema Structured Output
(raw) {"ref":"analysis"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
poolside/laguna-m.1:medium__json_schema FAIL 0% 260 in → 363 out (348 reasoning)
refs/heads/main
JSON Schema Structured Output
(raw) { "ref": "refs/heads/main" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
poolside/laguna-m.1:none__json_schema FAIL 0% 250 in → 39 out (0 reasoning)
a7fa160f41dbe94077e76bca2f7c1905d668abd1
JSON Schema Structured Output
(raw) {"ref": "a7fa160f41dbe94077e76bca2f7c1905d668abd1"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
poolside/laguna-xs-2.1:medium__json_schema FAIL 0% 255 in → 531 out (517 reasoning)
refs/heads/main
JSON Schema Structured Output
(raw) { "ref": "refs/heads/main" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
poolside/laguna-xs.2:low__json_schema FAIL 0% 292 in → 292 out (282 reasoning)
git-reset
JSON Schema Structured Output
(raw) {"ref": "git-reset"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 0% 259 in → 39 out (0 reasoning)
fd6876e293dcdfb957732acf47c95373004ed94b
JSON Schema Structured Output
(raw) {"ref":"fd6876e293dcdfb957732acf47c95373004ed94b"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0e218536-3af5-9f33-b09e-4e17e4c484be","request_id":"0e218536-3af5-9f33-b09e-4e17e4c484be"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d09978de-a9c3-9df0-9d32-a6e797c8fb6e","request_id":"d09978de-a9c3-9df0-9d32-a6e797c8fb6e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-859245e9-d13a-9f43-ac0b-9a5de6effbf6","request_id":"859245e9-d13a-9f43-ac0b-9a5de6effbf6"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e6a43022-1ea1-979f-b978-c40895d85c23","request_id":"e6a43022-1ea1-979f-b978-c40895d85c23"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a7f80656-d9b6-9a0c-8ef8-cf731d9506da","request_id":"a7f80656-d9b6-9a0c-8ef8-cf731d9506da"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-2bb1cef1-22ea-9f6c-b541-7c14c02af67f"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-b638a507-6398-9432-ab60-e3a2bde3b17e"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ef71dcb6-a9f4-90bf-af7a-3a9578728efb"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-edced7d3-0c03-9dce-91f0-05e6a502acc0"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-7919c88d-e4a9-9216-b6ef-91c2221ad72b"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-d3ab0b3c-679d-9181-b4a8-06a7bc70d2bd"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8f6713c6-6453-9a84-aeb2-baa8fb4adc8b","request_id":"8f6713c6-6453-9a84-aeb2-baa8fb4adc8b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d600c7c5-b396-9c6b-9f6b-96c345f89e09","request_id":"d600c7c5-b396-9c6b-9f6b-96c345f89e09"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-ac860cc3-3a86-90a5-a866-adc7bfa74880","request_id":"ac860cc3-3a86-90a5-a866-adc7bfa74880"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-43402174-e874-9d0d-861e-e5424906b5d7","request_id":"43402174-e874-9d0d-861e-e5424906b5d7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b21c86e1-445b-9ce8-9efe-c1438a7fdaab","request_id":"b21c86e1-445b-9ce8-9efe-c1438a7fdaab"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-fda0f87c-d627-9122-ae7d-7f62ec33f79b","request_id":"fda0f87c-d627-9122-ae7d-7f62ec33f79b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5385f937-61b1-9147-984c-c7bd24734300","request_id":"5385f937-61b1-9147-984c-c7bd24734300"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8f7734b7-72b6-97d8-b13e-90fb5cd90261","request_id":"8f7734b7-72b6-97d8-b13e-90fb5cd90261"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7ba93714-715c-9b6b-928e-85a49428d135","request_id":"7ba93714-715c-9b6b-928e-85a49428d135"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8ad1de6b-7116-97be-ae5c-1687a2f1f627","request_id":"8ad1de6b-7116-97be-ae5c-1687a2f1f627"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Model 'tencent/hy3:none' violated the reasoning_level=none invariant: provider reported 265 reasoning token(s)
Invalid JSON. Output: f632569cda3c51f5d85c3de43d6f4d9168e81d7c{"ref":"e725ade512274767e83a8a6febe7352377bd0756"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: 0c3f2a3b90ac2af48e2de00bd156483678a5b7d8
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
thinkingmachines/inkling:medium__json_schema FAIL 0% 222 in → 160 out (117 reasoning)
98b3060b1d4f3bc5d46f37a710a84b4b5a4ab9a9
JSON Schema Structured Output
(raw) { "ref": "98b3060b1d4f3bc5d46f37a710a84b4b5a4ab9a9" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output: c2f2ee8262ad9e4f9a62fdfc87f32f289a58b7b9{"ref":"c2f2ee8262ad9e4f9a62fdfc87f32f289a58b7b9"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
x-ai/grok-4.3:none__json_schema FAIL 0% 437 in → 31 out (0 reasoning)
5788d7f2c4a42d3809fa38d9720b0a731d4d4928
JSON Schema Structured Output
(raw) {"ref":"5788d7f2c4a42d3809fa38d9720b0a731d4d4928"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
z-ai/glm-4.7-flash:high FAIL 0% 218 in → 832 out (643 reasoning)
ce0bc3a8d3e07f97cb426322fbf78ddd53a75cf2
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-4.7-flash:none__json_schema FAIL 0% 216 in → 13 out (0 reasoning)
HEAD@{0}
JSON Schema Structured Output
(raw) { "ref": "HEAD@{0}" }
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-5.2:low__json_schema FAIL 0% 216 in → 20 out (0 reasoning)
https://github.com/git/git/blob/master/Documentation/git-reflog.txt
JSON Schema Structured Output
(raw) {"ref":"https://github.com/git/git/blob/master/Documentation/git-reflog.txt"}
Failure: Expected output to mention target hash prefix/full hash or matching HEAD@{n} selector for 'Add third version - important changes'