Remove ignored files only
Tests ability to remove only ignored files with git clean -fX. Evaluates understanding of gitignore interaction.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 printf '*.log node_modules/ ' > .gitignore
  5. 05 git add .gitignore
  6. 06 git commit -m 'Add gitignore'
  7. 07 echo 'data' > app.log
  8. 08 mkdir -p node_modules/pkg
  9. 09 echo '{}' > node_modules/pkg/index.js
  10. 10 echo 'untracked' > untracked.txt
Prompt
Remove only ignored files and directories while keeping untracked files. Output ONLY the git command, nothing else.
Expected
git clean -f -X -d

Scoped model quality, cost, API time, and token usage for git_clean/f004.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 92 in → 18 out (30 reasoning)
git clean -fdX
anthropic/claude-fable-5:high__json_schema PASS 100% 302 in → 37 out (33 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-fable-5:low PASS 100% 92 in → 9 out (0 reasoning)
git clean -fdX
anthropic/claude-fable-5:low__json_schema PASS 100% 302 in → 16 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-fable-5:max PASS 100% 92 in → 177 out (60 reasoning)
git clean -fdX
anthropic/claude-fable-5:max__json_schema PASS 100% 302 in → 191 out (68 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-fable-5:medium PASS 100% 92 in → 18 out (30 reasoning)
git clean -fdX
anthropic/claude-fable-5:medium__json_schema PASS 100% 302 in → 16 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-fable-5:xhigh PASS 100% 92 in → 36 out (27 reasoning)
git clean -fdX
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 302 in → 36 out (35 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 265 in → 712 out (544 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-haiku-4.5:low PASS 100% 102 in → 174 out (150 reasoning)
``` git clean -fdX ```
anthropic/claude-haiku-4.5:medium PASS 100% 102 in → 193 out (167 reasoning)
``` git clean -fdX ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 265 in → 307 out (277 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-haiku-4.5:none PASS 100% 73 in → 12 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 235 in → 13 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 102 in → 466 out (418 reasoning)
``` git clean -Xfd ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 265 in → 319 out (286 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-opus-4.6:high PASS 100% 73 in → 14 out (0 reasoning)
``` git clean -fXd ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 236 in → 88 out (65 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
anthropic/claude-opus-4.6:low PASS 100% 73 in → 13 out (0 reasoning)
``` git clean -fXd ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 236 in → 13 out (0 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
anthropic/claude-opus-4.6:max PASS 100% 73 in → 86 out (64 reasoning)
``` git clean -fdX ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 236 in → 103 out (80 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.6:medium PASS 100% 73 in → 13 out (0 reasoning)
``` git clean -fXd ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 236 in → 13 out (0 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
anthropic/claude-opus-4.6:none PASS 100% 73 in → 13 out (0 reasoning)
``` git clean -fXd ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 236 in → 13 out (0 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
anthropic/claude-opus-4.6:xhigh PASS 100% 73 in → 14 out (0 reasoning)
``` git clean -fXd ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 236 in → 113 out (91 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
anthropic/claude-opus-4.7:high PASS 100% 97 in → 11 out (0 reasoning)
git clean -fdX
anthropic/claude-opus-4.7:high__json_schema PASS 100% 307 in → 18 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.7:low PASS 100% 97 in → 11 out (0 reasoning)
git clean -fdX
anthropic/claude-opus-4.7:low__json_schema PASS 100% 307 in → 18 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.7:max PASS 100% 97 in → 148 out (74 reasoning)
`git clean -fdX`
anthropic/claude-opus-4.7:max__json_schema PASS 100% 307 in → 18 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.7:medium PASS 100% 97 in → 11 out (0 reasoning)
git clean -fdX
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 307 in → 18 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.7:none PASS 100% 97 in → 13 out (0 reasoning)
`git clean -fdX`
anthropic/claude-opus-4.7:none__json_schema PASS 100% 307 in → 18 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.7:xhigh PASS 100% 97 in → 13 out (0 reasoning)
`git clean -fdX`
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 307 in → 18 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.8:high PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 302 in → 17 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
anthropic/claude-opus-4.8:low PASS 100% 92 in → 9 out (0 reasoning)
git clean -fdX
anthropic/claude-opus-4.8:low__json_schema PASS 100% 302 in → 17 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
anthropic/claude-opus-4.8:max PASS 100% 92 in → 279 out (54 reasoning)
`git clean -Xdf`
anthropic/claude-opus-4.8:max__json_schema PASS 100% 302 in → 17 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
anthropic/claude-opus-4.8:medium PASS 100% 92 in → 9 out (0 reasoning)
git clean -fdX
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 302 in → 17 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
anthropic/claude-opus-4.8:none PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 302 in → 15 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-opus-4.8:xhigh PASS 100% 92 in → 9 out (0 reasoning)
git clean -fdX
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 302 in → 17 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
anthropic/claude-sonnet-4.6:high PASS 100% 73 in → 102 out (78 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 236 in → 99 out (72 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
anthropic/claude-sonnet-4.6:max PASS 100% 73 in → 140 out (114 reasoning)
``` git clean -Xdf ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 236 in → 106 out (81 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 236 in → 12 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-sonnet-4.6:none PASS 100% 73 in → 12 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 236 in → 13 out (0 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 73 in → 94 out (70 reasoning)
``` git clean -Xfd ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 236 in → 89 out (65 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command":"git clean -Xfd"}
anthropic/claude-sonnet-5:high PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-5:low PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-5:low__json_schema PASS 100% 302 in → 16 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
anthropic/claude-sonnet-5:medium PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 302 in → 16 out (0 reasoning)
git clean -dfX
JSON Schema Structured Output
(raw) {"command":"git clean -dfX"}
anthropic/claude-sonnet-5:none PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 302 in → 15 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-sonnet-5:xhigh PASS 100% 92 in → 13 out (0 reasoning)
``` git clean -fdX ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 302 in → 16 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
arcee-ai/trinity-large-thinking:low PASS 100% 58 in → 1,103 out (1,099 reasoning)
git clean -Xdf
arcee-ai/trinity-large-thinking:xhigh PASS 100% 58 in → 1,446 out (1,444 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash-0731:high PASS 100% 136 in → 372 out (377 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 136 in → 297 out (279 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 199 in → 198 out (181 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 155 in → 185 out (177 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 136 in → 265 out (256 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 63 in → 131 out (126 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 57 in → 315 out (299 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 57 in → 6 out (0 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 136 in → 141 out (147 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 57 in → 256 out (253 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX" }
deepseek/deepseek-v4-flash:high PASS 100% 57 in → 273 out (266 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 57 in → 128 out (116 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
deepseek/deepseek-v4-flash:low PASS 100% 57 in → 141 out (135 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash:none PASS 100% 57 in → 6 out (0 reasoning)
git clean -fdX
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 59 in → 13 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 136 in → 807 out (800 reasoning)
git clean -fdX
deepseek/deepseek-v4-pro:high PASS 100% 57 in → 299 out (292 reasoning)
git clean -fdX
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 244 in → 141 out (129 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 59 in → 741 out (726 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
deepseek/deepseek-v4-pro:medium PASS 100% 57 in → 385 out (378 reasoning)
git clean -fdX
deepseek/deepseek-v4-pro:none PASS 100% 57 in → 6 out (0 reasoning)
git clean -fdX
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 59 in → 12 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 57 in → 413 out (406 reasoning)
git clean -fdX
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 322 in → 630 out (618 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command": "git clean -fXd"}
google/gemini-3-flash-preview:high PASS 100% 62 in → 265 out (260 reasoning)
git clean -fdX
google/gemini-3-flash-preview:high__json_schema PASS 100% 119 in → 445 out (435 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
google/gemini-3-flash-preview:low PASS 100% 62 in → 375 out (370 reasoning)
git clean -fdX
google/gemini-3-flash-preview:low__json_schema PASS 100% 119 in → 270 out (261 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemini-3-flash-preview:medium PASS 100% 62 in → 235 out (230 reasoning)
git clean -fdX
google/gemini-3-flash-preview:medium__json_schema PASS 100% 119 in → 367 out (358 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemini-3-flash-preview:xhigh PASS 100% 62 in → 250 out (245 reasoning)
git clean -fdX
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 119 in → 310 out (301 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 62 in → 544 out (539 reasoning)
git clean -fdX
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 63 in → 556 out (547 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command":"git clean -Xfd"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 63 in → 146 out (141 reasoning)
git clean -fdX
google/gemini-3.1-flash-lite-preview:medium PASS 100% 63 in → 355 out (350 reasoning)
git clean -fdX
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 119 in → 225 out (215 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command": "git clean -Xfd"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 63 in → 615 out (610 reasoning)
git clean -fdX
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 63 in → 233 out (223 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command": "git clean -fXd"}
google/gemini-3.1-pro-preview:high PASS 100% 62 in → 398 out (393 reasoning)
git clean -fdX
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 119 in → 399 out (389 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
google/gemini-3.1-pro-preview:low PASS 100% 62 in → 301 out (296 reasoning)
git clean -Xdf
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 119 in → 214 out (204 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command": "git clean -Xdf"}
google/gemini-3.1-pro-preview:medium PASS 100% 62 in → 291 out (286 reasoning)
git clean -Xdf
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 119 in → 291 out (281 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command": "git clean -Xfd"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 62 in → 668 out (663 reasoning)
git clean -fdX
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 119 in → 296 out (287 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
google/gemini-3.5-flash-lite:high PASS 100% 62 in → 815 out (810 reasoning)
git clean -Xdf
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 119 in → 537 out (528 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command":"git clean -Xfd"}
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 63 in → 15 out (0 reasoning)
git clean -X -f -f -d
JSON Schema Structured Output
(raw) {"command": "git clean -X -f -f -d"}
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 63 in → 339 out (326 reasoning)
git clean -X -d -f
JSON Schema Structured Output
(raw) {"command": "git clean -X -d -f"}
google/gemini-3.5-flash-lite:medium PASS 100% 63 in → 456 out (451 reasoning)
git clean -Xfd
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 119 in → 284 out (274 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command": "git clean -Xfd"}
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 63 in → 343 out (334 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemini-3.5-flash:high PASS 100% 62 in → 709 out (704 reasoning)
git clean -fdX
google/gemini-3.5-flash:high__json_schema PASS 100% 119 in → 595 out (585 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
google/gemini-3.5-flash:low PASS 100% 62 in → 459 out (454 reasoning)
git clean -fXd
google/gemini-3.5-flash:low__json_schema PASS 100% 119 in → 176 out (166 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
google/gemini-3.5-flash:medium PASS 100% 62 in → 543 out (538 reasoning)
git clean -fdX
google/gemini-3.5-flash:medium__json_schema PASS 100% 119 in → 396 out (386 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command": "git clean -fXd"}
google/gemini-3.5-flash:xhigh PASS 100% 62 in → 609 out (604 reasoning)
git clean -fdX
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 119 in → 621 out (611 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
google/gemini-3.6-flash:high PASS 100% 63 in → 387 out (382 reasoning)
git clean -fdX
google/gemini-3.6-flash:high__json_schema PASS 100% 119 in → 484 out (475 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemini-3.6-flash:low PASS 100% 63 in → 349 out (344 reasoning)
git clean -fdX
google/gemini-3.6-flash:max PASS 100% 63 in → 398 out (393 reasoning)
git clean -fdX
google/gemini-3.6-flash:max__json_schema PASS 100% 119 in → 333 out (318 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemini-3.6-flash:medium PASS 100% 63 in → 494 out (489 reasoning)
git clean -fdX
google/gemini-3.6-flash:medium__json_schema PASS 100% 119 in → 324 out (315 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemini-3.6-flash:xhigh PASS 100% 63 in → 402 out (397 reasoning)
git clean -fdX
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 63 in → 406 out (397 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemma-4-26b-a4b-it:high PASS 100% 78 in → 451 out (445 reasoning)
git clean -fdX
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 77 in → 580 out (519 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemma-4-26b-a4b-it:low PASS 100% 78 in → 736 out (672 reasoning)
git clean -fdX
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 77 in → 361 out (294 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemma-4-26b-a4b-it:medium PASS 100% 78 in → 937 out (875 reasoning)
git clean -fdX
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 75 in → 16 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 77 in → 326 out (276 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemma-4-31b-it:high PASS 100% 78 in → 252 out (214 reasoning)
git clean -fdX
google/gemma-4-31b-it:high__json_schema PASS 100% 78 in → 302 out (256 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemma-4-31b-it:low__json_schema PASS 100% 78 in → 228 out (192 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
google/gemma-4-31b-it:medium PASS 100% 78 in → 214 out (187 reasoning)
git clean -fdX
google/gemma-4-31b-it:medium__json_schema PASS 100% 81 in → 240 out (229 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
google/gemma-4-31b-it:xhigh PASS 100% 77 in → 230 out (199 reasoning)
git clean -fdX
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 77 in → 207 out (170 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
inclusionai/ling-2.6-flash PASS 100% 74 in → 18 out
``` git clean -fdx -- . ':!untracked.txt' ```
minimax/minimax-m2.7:high PASS 100% 88 in → 1,763 out (1,757 reasoning)
git clean -fdX
minimax/minimax-m2.7:high__json_schema PASS 100% 218 in → 1,742 out (1,731 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
minimax/minimax-m2.7:low PASS 100% 88 in → 607 out (599 reasoning)
git clean -fX -d
minimax/minimax-m2.7:low__json_schema PASS 100% 218 in → 800 out (781 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
minimax/minimax-m2.7:medium PASS 100% 88 in → 417 out (411 reasoning)
git clean -fdX
minimax/minimax-m2.7:medium__json_schema PASS 100% 218 in → 3,036 out (3,021 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) { "command": "git clean -fXd" }
minimax/minimax-m2.7:xhigh PASS 100% 86 in → 1,315 out (1,308 reasoning)
git clean -fdX
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 218 in → 1,374 out (1,363 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) {"command":"git clean -fXd"}
minimax/minimax-m3:low PASS 100% 224 in → 385 out (419 reasoning)
`git clean -fdX`
minimax/minimax-m3:xhigh PASS 100% 224 in → 85 out (83 reasoning)
`git clean -fXd`
mistralai/devstral-2512 PASS 100% 56 in → 11 out
```sh git clean -fdX ```
mistralai/mistral-medium-3-5:high PASS 100% 68 in → 515 out (552 reasoning)
git clean -fd -X
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 56 in → 693 out (763 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
mistralai/mistral-medium-3-5:low PASS 100% 68 in → 430 out (457 reasoning)
git clean -fdX
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 56 in → 613 out (643 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
mistralai/mistral-medium-3-5:medium PASS 100% 68 in → 501 out (533 reasoning)
git clean -Xdf
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 56 in → 1,002 out (1,085 reasoning)
git clean -fd -X
JSON Schema Structured Output
(raw) {"command": "git clean -fd -X"}
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 56 in → 17 out (0 reasoning)
git clean -f -d --exclude=. -X
JSON Schema Structured Output
(raw) {"command": "git clean -f -d --exclude=. -X"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 68 in → 3,112 out (3,547 reasoning)
git clean -fdX
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 56 in → 1,225 out (1,307 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
moonshotai/kimi-k2.6:high PASS 100% 59 in → 1,193 out (1,187 reasoning)
git clean -Xfd
moonshotai/kimi-k2.6:high__json_schema PASS 100% 241 in → 2,265 out (2,254 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
moonshotai/kimi-k2.6:low PASS 100% 59 in → 3,482 out (3,476 reasoning)
git clean -Xdf
moonshotai/kimi-k2.6:low__json_schema PASS 100% 241 in → 2,726 out (2,715 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
moonshotai/kimi-k2.6:medium PASS 100% 59 in → 3,298 out (3,678 reasoning)
git clean -fdX
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 58 in → 1,352 out (1,141 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX"}
moonshotai/kimi-k2.6:xhigh PASS 100% 59 in → 3,056 out (3,049 reasoning)
git clean -fXd
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 59 in → 8,067 out (8,053 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
moonshotai/kimi-k2.7-code:high PASS 100% 59 in → 1,031 out (1,024 reasoning)
git clean -fdX
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 59 in → 383 out (389 reasoning)
git clean -f -X -d
JSON Schema Structured Output
(raw) { "command": "git clean -f -X -d" }
moonshotai/kimi-k2.7-code:low PASS 100% 58 in → 127 out (117 reasoning)
git clean -X -d -f
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 59 in → 1,075 out (1,186 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
moonshotai/kimi-k2.7-code:medium PASS 100% 59 in → 182 out (175 reasoning)
git clean -fdX
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 142 in → 189 out (174 reasoning)
git clean -X -d -f
JSON Schema Structured Output
(raw) {"command": "git clean -X -d -f"}
moonshotai/kimi-k2.7-code:xhigh PASS 100% 59 in → 357 out (350 reasoning)
git clean -fdX
moonshotai/kimi-k3:high PASS 100% 136 in → 163 out (143 reasoning)
git clean -fdX
moonshotai/kimi-k3:high__json_schema PASS 100% 136 in → 194 out (174 reasoning)
git clean -fdX
moonshotai/kimi-k3:low PASS 100% 136 in → 191 out (171 reasoning)
git clean -fdX
moonshotai/kimi-k3:low__json_schema PASS 100% 136 in → 236 out (216 reasoning)
git clean -fdX
moonshotai/kimi-k3:max PASS 100% 136 in → 179 out (159 reasoning)
git clean -fdX
moonshotai/kimi-k3:medium PASS 100% 136 in → 181 out (161 reasoning)
git clean -fdX
moonshotai/kimi-k3:medium__json_schema PASS 100% 136 in → 143 out (123 reasoning)
git clean -fdX
moonshotai/kimi-k3:xhigh PASS 100% 136 in → 145 out (125 reasoning)
git clean -fdX
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 136 in → 193 out (173 reasoning)
git clean -fdX
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 69 in → 582 out (638 reasoning)
git clean -Xfd
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 69 in → 521 out (534 reasoning)
git clean -f -X .
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 69 in → 14 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 69 in → 216 out (228 reasoning)
git clean -Xfd
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 69 in → 470 out (442 reasoning)
git clean -X -f -d
JSON Schema Structured Output
(raw) { "command": "git clean -X -f -d" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 69 in → 204 out (188 reasoning)
git clean -Xfd
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 69 in → 274 out (245 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) { "command": "git clean -fXd" }
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 69 in → 175 out (155 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) { "command": "git clean -Xfd" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 69 in → 6 out (0 reasoning)
git clean -Xfd
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 69 in → 14 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) { "command": "git clean -Xdf" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 69 in → 205 out (230 reasoning)
git clean -Xfd
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 69 in → 218 out (195 reasoning)
git clean -X -f -d
JSON Schema Structured Output
(raw) { "command": "git clean -X -f -d" }
openai/gpt-5.4-mini:high PASS 100% 54 in → 82 out (71 reasoning)
git clean -fdX
openai/gpt-5.4-mini:high__json_schema PASS 100% 89 in → 87 out (68 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-mini:low PASS 100% 54 in → 50 out (39 reasoning)
git clean -fdX
openai/gpt-5.4-mini:low__json_schema PASS 100% 89 in → 64 out (45 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-mini:medium PASS 100% 54 in → 81 out (70 reasoning)
git clean -fdX
openai/gpt-5.4-mini:medium__json_schema PASS 100% 89 in → 92 out (73 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-mini:none PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.4-mini:none__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-mini:xhigh PASS 100% 54 in → 219 out (208 reasoning)
git clean -fdX
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 89 in → 535 out (516 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-nano:high PASS 100% 54 in → 100 out (86 reasoning)
git clean -d -X -f
openai/gpt-5.4-nano:high__json_schema PASS 100% 89 in → 157 out (138 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-nano:low__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-nano:medium PASS 100% 54 in → 90 out (79 reasoning)
git clean -fdX
openai/gpt-5.4-nano:medium__json_schema PASS 100% 89 in → 100 out (81 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4-nano:xhigh PASS 100% 54 in → 509 out (498 reasoning)
git clean -fdX
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 89 in → 1,462 out (1,443 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4:high PASS 100% 54 in → 68 out (57 reasoning)
git clean -fdX
openai/gpt-5.4:high__json_schema PASS 100% 89 in → 123 out (104 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4:low PASS 100% 54 in → 43 out (32 reasoning)
git clean -fdX
openai/gpt-5.4:low__json_schema PASS 100% 89 in → 47 out (28 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4:medium PASS 100% 54 in → 64 out (53 reasoning)
git clean -fdX
openai/gpt-5.4:medium__json_schema PASS 100% 89 in → 70 out (51 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4:none PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.4:none__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.4:xhigh PASS 100% 54 in → 135 out (124 reasoning)
git clean -fdX
openai/gpt-5.4:xhigh__json_schema PASS 100% 89 in → 175 out (156 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.5:high PASS 100% 54 in → 85 out (74 reasoning)
git clean -fdX
openai/gpt-5.5:high__json_schema PASS 100% 89 in → 156 out (137 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.5:low PASS 100% 54 in → 44 out (33 reasoning)
git clean -fdX
openai/gpt-5.5:low__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.5:medium PASS 100% 54 in → 57 out (46 reasoning)
git clean -fdX
openai/gpt-5.5:medium__json_schema PASS 100% 89 in → 69 out (50 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.5:none PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.5:none__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.5:xhigh PASS 100% 54 in → 131 out (120 reasoning)
git clean -fdX
openai/gpt-5.5:xhigh__json_schema PASS 100% 89 in → 279 out (260 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-luna:high PASS 100% 54 in → 72 out (61 reasoning)
git clean -fdX
openai/gpt-5.6-luna:high__json_schema PASS 100% 89 in → 152 out (133 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-luna:low PASS 100% 54 in → 48 out (37 reasoning)
git clean -fdX
openai/gpt-5.6-luna:max PASS 100% 54 in → 75 out (64 reasoning)
git clean -fdX
openai/gpt-5.6-luna:max__json_schema PASS 100% 89 in → 535 out (516 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-luna:medium PASS 100% 54 in → 42 out (31 reasoning)
git clean -fdX
openai/gpt-5.6-luna:medium__json_schema PASS 100% 89 in → 54 out (35 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-luna:none PASS 100% 54 in → 9 out (0 reasoning)
git clean -Xdf
openai/gpt-5.6-luna:none__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-luna:xhigh PASS 100% 54 in → 83 out (72 reasoning)
git clean -fdX
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 89 in → 70 out (51 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-sol:high PASS 100% 54 in → 41 out (30 reasoning)
git clean -fdX
openai/gpt-5.6-sol:high__json_schema PASS 100% 89 in → 43 out (24 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-sol:low PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.6-sol:low__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-sol:max PASS 100% 54 in → 59 out (48 reasoning)
git clean -fdX
openai/gpt-5.6-sol:max__json_schema PASS 100% 89 in → 123 out (104 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-sol:medium PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.6-sol:medium__json_schema PASS 100% 89 in → 80 out (61 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-sol:none PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.6-sol:none__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-sol:xhigh PASS 100% 54 in → 39 out (28 reasoning)
git clean -fdX
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 89 in → 125 out (106 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-terra:high PASS 100% 54 in → 49 out (38 reasoning)
git clean -fdX
openai/gpt-5.6-terra:high__json_schema PASS 100% 89 in → 45 out (26 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-terra:low PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.6-terra:low__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-terra:max PASS 100% 54 in → 67 out (56 reasoning)
git clean -fdX
openai/gpt-5.6-terra:max__json_schema PASS 100% 89 in → 106 out (87 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-terra:medium PASS 100% 54 in → 39 out (28 reasoning)
git clean -fdX
openai/gpt-5.6-terra:medium__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-terra:none PASS 100% 54 in → 9 out (0 reasoning)
git clean -fdX
openai/gpt-5.6-terra:none__json_schema PASS 100% 89 in → 17 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-5.6-terra:xhigh PASS 100% 54 in → 61 out (50 reasoning)
git clean -fdX
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 89 in → 92 out (73 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-120b:high PASS 100% 115 in → 120 out (115 reasoning)
git clean -fdX
openai/gpt-oss-120b:high__json_schema PASS 100% 117 in → 257 out (258 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-120b:low PASS 100% 115 in → 46 out (32 reasoning)
git clean -fdX
openai/gpt-oss-120b:low__json_schema PASS 100% 115 in → 45 out (26 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-120b:medium PASS 100% 113 in → 92 out (91 reasoning)
git clean -fdX
openai/gpt-oss-120b:medium__json_schema PASS 100% 113 in → 112 out (100 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-120b:xhigh PASS 100% 115 in → 84 out (79 reasoning)
git clean -fdX
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 113 in → 329 out (346 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-20b:high PASS 100% 113 in → 296 out (309 reasoning)
git clean -fdX
openai/gpt-oss-20b:high__json_schema PASS 100% 113 in → 250 out (249 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-20b:low PASS 100% 120 in → 46 out (28 reasoning)
git clean -X -d -f .
openai/gpt-oss-20b:low__json_schema PASS 100% 115 in → 9 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-20b:medium PASS 100% 102 in → 152 out (141 reasoning)
git clean -fdX
openai/gpt-oss-20b:medium__json_schema PASS 100% 115 in → 279 out (262 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
openai/gpt-oss-20b:xhigh PASS 100% 115 in → 1,260 out (1,256 reasoning)
git clean -fdX
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 102 in → 919 out (932 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
poolside/laguna-m.1:high PASS 100% 66 in → 836 out (827 reasoning)
git clean -Xd -f
poolside/laguna-m.1:high__json_schema PASS 100% 66 in → 1,484 out (1,471 reasoning)
git clean -Xd -f
JSON Schema Structured Output
(raw) {"command": "git clean -Xd -f"}
poolside/laguna-m.1:low PASS 100% 66 in → 1,109 out (1,100 reasoning)
git clean -Xd -f
poolside/laguna-m.1:low__json_schema PASS 100% 66 in → 269 out (253 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
poolside/laguna-m.1:medium PASS 100% 66 in → 273 out (265 reasoning)
git clean -Xfd
poolside/laguna-m.1:none__json_schema PASS 100% 66 in → 11 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
poolside/laguna-m.1:xhigh PASS 100% 66 in → 2,194 out (2,187 reasoning)
git clean -fXd
poolside/laguna-m.1:xhigh__json_schema PASS 100% 66 in → 428 out (411 reasoning)
git clean -dXf
JSON Schema Structured Output
(raw) { "command": "git clean -dXf" }
poolside/laguna-xs-2.1:low__json_schema PASS 100% 66 in → 297 out (280 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
poolside/laguna-xs-2.1:medium PASS 100% 66 in → 261 out (255 reasoning)
git clean -fdX
poolside/laguna-xs-2.1:none PASS 100% 66 in → 5 out (0 reasoning)
git clean -fdX
poolside/laguna-xs.2:high PASS 100% 103 in → 1,095 out (1,087 reasoning)
git clean -fdX
qwen/qwen3.6-27b:high PASS 100% 62 in → 2,134 out (1,926 reasoning)
git clean -Xfd
qwen/qwen3.6-27b:low PASS 100% 62 in → 1,047 out (964 reasoning)
git clean -Xfd
qwen/qwen3.6-27b:medium PASS 100% 62 in → 1,882 out (1,721 reasoning)
git clean -fXd
qwen/qwen3.6-27b:medium__json_schema PASS 100% 62 in → 2,201 out (1,947 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) { "command": "git clean -Xdf" }
qwen/qwen3.6-27b:xhigh PASS 100% 62 in → 1,679 out (1,532 reasoning)
git clean -fdX
qwen/qwen3.6-35b-a3b:high PASS 100% 62 in → 1,093 out (1,083 reasoning)
git clean -Xdf
qwen/qwen3.6-35b-a3b:medium PASS 100% 62 in → 1,516 out (1,405 reasoning)
git clean -fdX
qwen/qwen3.6-35b-a3b:none PASS 100% 64 in → 6 out (0 reasoning)
git clean -fdX
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 62 in → 1,394 out (1,292 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) { "command": "git clean -Xdf" }
qwen/qwen3.6-flash:low PASS 100% 62 in → 3,275 out (3,265 reasoning)
git clean -Xfd
qwen/qwen3.6-flash:medium PASS 100% 62 in → 2,805 out (2,795 reasoning)
git clean -fdX
qwen/qwen3.6-flash:none PASS 100% 64 in → 5 out (0 reasoning)
git clean -fdX
qwen/qwen3.6-flash:xhigh PASS 100% 62 in → 1,949 out (1,939 reasoning)
git clean -fdX
qwen/qwen3.7-flash:high PASS 100% 62 in → 1,867 out (1,857 reasoning)
git clean -Xdf
qwen/qwen3.7-flash:low PASS 100% 62 in → 1,185 out (1,175 reasoning)
git clean -Xdf
qwen/qwen3.7-flash:max PASS 100% 62 in → 2,189 out (2,179 reasoning)
git clean -Xdf
qwen/qwen3.7-flash:medium PASS 100% 62 in → 1,173 out (1,163 reasoning)
git clean -Xdf
qwen/qwen3.7-max:high PASS 100% 62 in → 750 out (740 reasoning)
git clean -fdX
qwen/qwen3.7-max:low PASS 100% 62 in → 277 out (267 reasoning)
git clean -fdX
qwen/qwen3.7-max:medium PASS 100% 62 in → 1,036 out (1,026 reasoning)
git clean -fdX
qwen/qwen3.7-max:none PASS 100% 64 in → 5 out (0 reasoning)
git clean -fdX
qwen/qwen3.7-max:xhigh PASS 100% 62 in → 985 out (975 reasoning)
git clean -fdX
qwen/qwen3.7-plus:high PASS 100% 62 in → 912 out (902 reasoning)
git clean -fdX
qwen/qwen3.7-plus:low PASS 100% 62 in → 951 out (941 reasoning)
git clean -fdX
qwen/qwen3.7-plus:medium PASS 100% 62 in → 1,949 out (1,939 reasoning)
git clean -fXd
qwen/qwen3.7-plus:none PASS 100% 64 in → 5 out (0 reasoning)
git clean -fdX
tencent/hy3:high PASS 100% 61 in → 1,189 out (1,182 reasoning)
git clean -fXd
tencent/hy3:high__json_schema PASS 100% 61 in → 1,452 out (1,475 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) { "command": "git clean -fXd" }
tencent/hy3:low__json_schema PASS 100% 61 in → 727 out (760 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) { "command": "git clean -fXd" }
tencent/hy3:medium PASS 100% 61 in → 746 out (752 reasoning)
git clean -fXd
tencent/hy3:medium__json_schema PASS 100% 61 in → 613 out (587 reasoning)
git clean -fXd
JSON Schema Structured Output
(raw) { "command": "git clean -fXd" }
tencent/hy3:none__json_schema PASS 100% 64 in → 16 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
tencent/hy3:xhigh PASS 100% 61 in → 788 out (770 reasoning)
git clean -fdX
tencent/hy3:xhigh__json_schema PASS 100% 61 in → 781 out (738 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
thinkingmachines/inkling-small:high PASS 100% 63 in → 189 out (195 reasoning)
git clean -fdX
thinkingmachines/inkling-small:high__json_schema PASS 100% 63 in → 226 out (211 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
thinkingmachines/inkling-small:low PASS 100% 63 in → 71 out (62 reasoning)
git clean -fdX
thinkingmachines/inkling-small:low__json_schema PASS 100% 63 in → 98 out (81 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
thinkingmachines/inkling-small:max PASS 100% 63 in → 199 out (188 reasoning)
git clean -fdX
thinkingmachines/inkling-small:max__json_schema PASS 100% 63 in → 173 out (155 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
thinkingmachines/inkling-small:medium PASS 100% 63 in → 263 out (272 reasoning)
git clean -fdX
thinkingmachines/inkling-small:medium__json_schema PASS 100% 63 in → 164 out (146 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
thinkingmachines/inkling-small:xhigh PASS 100% 63 in → 223 out (229 reasoning)
git clean -fdX
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 63 in → 225 out (210 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
thinkingmachines/inkling:high PASS 100% 63 in → 335 out (322 reasoning)
git clean -X -d -f
thinkingmachines/inkling:high__json_schema PASS 100% 63 in → 118 out (101 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
thinkingmachines/inkling:low PASS 100% 63 in → 46 out (32 reasoning)
git clean -f -d -X
thinkingmachines/inkling:max PASS 100% 63 in → 225 out (214 reasoning)
git clean -fdX
thinkingmachines/inkling:medium PASS 100% 63 in → 155 out (148 reasoning)
git clean -f -X -d
thinkingmachines/inkling:medium__json_schema PASS 100% 63 in → 334 out (317 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
thinkingmachines/inkling:none__json_schema PASS 100% 61 in → 13 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX"}
thinkingmachines/inkling:xhigh PASS 100% 63 in → 452 out (442 reasoning)
git clean -fdX
x-ai/grok-4.3:high PASS 100% 233 in → 759 out (752 reasoning)
`git clean -Xfd`
x-ai/grok-4.3:high__json_schema PASS 100% 289 in → 822 out (813 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
x-ai/grok-4.3:low PASS 100% 239 in → 429 out (415 reasoning)
git clean -Xdf The final answer is the command above.
x-ai/grok-4.3:low__json_schema PASS 100% 295 in → 627 out (618 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command":"git clean -Xfd"}
x-ai/grok-4.3:max PASS 100% 233 in → 668 out (658 reasoning)
```git git clean -Xfd ```
x-ai/grok-4.3:max__json_schema PASS 100% 289 in → 643 out (634 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command":"git clean -Xfd"}
x-ai/grok-4.3:medium PASS 100% 239 in → 878 out (873 reasoning)
git clean -Xdf
x-ai/grok-4.3:medium__json_schema PASS 100% 295 in → 925 out (912 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) { "command": "git clean -Xdf" }
x-ai/grok-4.3:none PASS 100% 231 in → 8 out (0 reasoning)
git clean -X -d -f
x-ai/grok-4.3:none__json_schema PASS 100% 287 in → 12 out (0 reasoning)
git clean -X -d -f
JSON Schema Structured Output
(raw) {"command":"git clean -X -d -f"}
x-ai/grok-4.3:xhigh PASS 100% 233 in → 314 out (304 reasoning)
```bash git clean -Xdf ```
x-ai/grok-4.3:xhigh__json_schema PASS 100% 289 in → 1,525 out (1,516 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command":"git clean -Xdf"}
x-ai/grok-4.5:high__json_schema PASS 100% 323 in → 459 out (449 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
x-ai/grok-4.5:low PASS 100% 257 in → 306 out (297 reasoning)
``` git clean -fdX ```
x-ai/grok-4.5:low__json_schema PASS 100% 323 in → 220 out (210 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
x-ai/grok-4.5:max PASS 100% 257 in → 356 out (351 reasoning)
git clean -fdX
x-ai/grok-4.5:max__json_schema PASS 100% 323 in → 547 out (537 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
x-ai/grok-4.5:medium PASS 100% 257 in → 541 out (536 reasoning)
git clean -fdX
x-ai/grok-4.5:medium__json_schema PASS 100% 323 in → 808 out (798 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
x-ai/grok-4.5:xhigh PASS 100% 257 in → 347 out (342 reasoning)
git clean -fdX
x-ai/grok-4.5:xhigh__json_schema PASS 100% 323 in → 579 out (569 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
z-ai/glm-4.7-flash:none PASS 100% 54 in → 6 out (0 reasoning)
git clean -Xdf
z-ai/glm-4.7-flash:none__json_schema PASS 100% 54 in → 11 out (0 reasoning)
git clean -Xdf
JSON Schema Structured Output
(raw) {"command": "git clean -Xdf"}
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 54 in → 568 out (570 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) {"command": "git clean -Xfd"}
z-ai/glm-5.2:high PASS 100% 61 in → 649 out (642 reasoning)
git clean -fdX
z-ai/glm-5.2:high__json_schema PASS 100% 54 in → 409 out (383 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) { "command": "git clean -fdX" }
z-ai/glm-5.2:low PASS 100% 61 in → 507 out (498 reasoning)
git clean -fdX
z-ai/glm-5.2:low__json_schema PASS 100% 63 in → 452 out (437 reasoning)
git clean -d -X -f
JSON Schema Structured Output
(raw) {"command": "git clean -d -X -f"}
z-ai/glm-5.2:medium PASS 100% 61 in → 460 out (454 reasoning)
git clean -fdX
z-ai/glm-5.2:medium__json_schema PASS 100% 236 in → 127 out (112 reasoning)
git clean -Xfd
JSON Schema Structured Output
(raw) { "command": "git clean -Xfd" }
z-ai/glm-5.2:none PASS 100% 55 in → 6 out (0 reasoning)
git clean -fdX
z-ai/glm-5.2:none__json_schema PASS 100% 230 in → 10 out (0 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command":"git clean -fdX"}
z-ai/glm-5.2:xhigh PASS 100% 61 in → 408 out (401 reasoning)
git clean -fdX
z-ai/glm-5.2:xhigh__json_schema PASS 100% 61 in → 580 out (568 reasoning)
git clean -fdX
JSON Schema Structured Output
(raw) {"command": "git clean -fdX"}
anthropic/claude-haiku-4.5:high FAIL 66.7% 102 in → 413 out (380 reasoning)
``` git clean -f -X ```
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
anthropic/claude-haiku-4.5:low__json_schema FAIL 66.7% 265 in → 233 out (212 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) {"command": "git clean -fX"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
anthropic/claude-sonnet-4.6:low FAIL 66.7% 73 in → 12 out (0 reasoning)
``` git clean -fX ```
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
anthropic/claude-sonnet-4.6:low__json_schema FAIL 66.7% 236 in → 12 out (0 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) {"command":"git clean -fX"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
anthropic/claude-sonnet-4.6:medium FAIL 66.7% 73 in → 12 out (0 reasoning)
``` git clean -fX ```
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-large-thinking:high FAIL 66.7% 58 in → 1,244 out (1,243 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-large-thinking:medium FAIL 66.7% 58 in → 1,388 out (1,387 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:xhigh__json_schema FAIL 66.7% 58 in → 858 out (966 reasoning)
git clean -fdx
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}]
deepseek/deepseek-v4-flash-0731:low FAIL 66.7% 57 in → 235 out (240 reasoning)
git clean -fX
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-flash-0731:none__json_schema FAIL 66.7% 57 in → 14 out (0 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-flash:medium FAIL 66.7% 57 in → 119 out (112 reasoning)
git clean -fX
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-flash:medium__json_schema FAIL 66.7% 59 in → 325 out (311 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-flash:xhigh__json_schema FAIL 66.7% 59 in → 238 out (224 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemini-3.1-flash-lite-preview:low__json_schema FAIL 66.7% 63 in → 113 out (103 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) {"command": "git clean -fX"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemini-3.5-flash-lite:low FAIL 66.7% 62 in → 7 out (0 reasoning)
`git clean -Xf`
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemini-3.5-flash-lite:max FAIL 66.7% 63 in → 469 out (464 reasoning)
git clean -Xf
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemini-3.5-flash-lite:xhigh FAIL 66.7% 63 in → 353 out (347 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemini-3.6-flash:low__json_schema FAIL 66.7% 119 in → 10 out (0 reasoning)
git clean -f -X
JSON Schema Structured Output
(raw) {"command":"git clean -f -X"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
ibm-granite/granite-4.1-8b FAIL 66.7% 71 in → 6 out
git clean -dfx
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}]
minimax/minimax-m2.5:high__json_schema FAIL 66.7% 86 in → 561 out (660 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:low__json_schema FAIL 66.7% 101 in → 211 out (212 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:medium__json_schema FAIL 66.7% 86 in → 143 out (159 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:xhigh__json_schema FAIL 66.7% 86 in → 295 out (342 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m3:high FAIL 66.7% 224 in → 118 out (126 reasoning)
git clean -fX
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m3:high__json_schema FAIL 66.7% 172 in → 14 out (13 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m3:low__json_schema FAIL 66.7% 211 in → 13 out (0 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m3:medium FAIL 66.7% 224 in → 114 out (120 reasoning)
`git clean -fX`
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m3:medium__json_schema FAIL 66.7% 211 in → 14 out (0 reasoning)
git clean -f -X
JSON Schema Structured Output
(raw) { "command": "git clean -f -X" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m3:xhigh__json_schema FAIL 66.7% 172 in → 14 out (13 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
mistralai/mistral-medium-3-5:none FAIL 66.7% 68 in → 6 out (0 reasoning)
git clean -fX
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
moonshotai/kimi-k2.7-code:xhigh__json_schema FAIL 66.7% 59 in → 150 out (133 reasoning)
git clean -X -f
JSON Schema Structured Output
(raw) { "command": "git clean -X -f" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-nano-30b-a3b:high__json_schema FAIL 66.7% 69 in → 840 out (958 reasoning)
git clean -Xf
JSON Schema Structured Output
(raw) { "command": "git clean -Xf" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-nano-30b-a3b:xhigh FAIL 66.7% 69 in → 375 out (389 reasoning)
git clean -Xf
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-super-120b-a12b:medium FAIL 66.7% 69 in → 545 out (523 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:high FAIL 66.7% 66 in → 486 out (479 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:high__json_schema FAIL 66.7% 66 in → 381 out (370 reasoning)
git clean -Xf
JSON Schema Structured Output
(raw) {"command": "git clean -Xf"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:medium__json_schema FAIL 66.7% 66 in → 262 out (244 reasoning)
git clean -X -f
JSON Schema Structured Output
(raw) { "command": "git clean -X -f" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:xhigh FAIL 66.7% 66 in → 1,160 out (1,153 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:xhigh__json_schema FAIL 66.7% 66 in → 251 out (235 reasoning)
git clean -Xf
JSON Schema Structured Output
(raw) { "command": "git clean -Xf" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:low__json_schema FAIL 66.7% 103 in → 667 out (656 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) {"command": "git clean -fX"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:medium__json_schema FAIL 66.7% 103 in → 329 out (311 reasoning)
git clean -X -f
JSON Schema Structured Output
(raw) { "command": "git clean -X -f" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:xhigh FAIL 66.7% 103 in → 162 out (153 reasoning)
git clean -X -f
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-27b:low__json_schema FAIL 66.7% 62 in → 1,970 out (1,729 reasoning)
git clean -fX
JSON Schema Structured Output
(raw) { "command": "git clean -fX" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-27b:none__json_schema FAIL 66.7% 64 in → 17 out (0 reasoning)
git clean -X -f
JSON Schema Structured Output
(raw) { "command": "git clean -X -f" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-35b-a3b:high__json_schema FAIL 66.7% 62 in → 3,557 out (3,055 reasoning)
git clean -X -f
JSON Schema Structured Output
(raw) { "command": "git clean -X -f" }
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-35b-a3b:xhigh FAIL 66.7% 62 in → 1,139 out (1,027 reasoning)
git clean -Xf
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-flash:high FAIL 66.7% 62 in → 1,223 out (1,213 reasoning)
git clean -Xf
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.7-flash:xhigh FAIL 66.7% 62 in → 1,370 out (1,361 reasoning)
git clean -xfd
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}]
tencent/hy3:low FAIL 66.7% 61 in → 614 out (644 reasoning)
git clean -fX
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
tencent/hy3:none FAIL 66.7% 64 in → 6 out (0 reasoning)
git clean -fX
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
thinkingmachines/inkling-small:none FAIL 66.7% 61 in → 9 out (0 reasoning)
git clean -f -X
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
z-ai/glm-4.7-flash:high FAIL 66.7% 54 in → 705 out (687 reasoning)
git clean -x -f -d
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}]
z-ai/glm-4.7-flash:low__json_schema FAIL 66.7% 54 in → 961 out (975 reasoning)
git clean -Xf
JSON Schema Structured Output
(raw) {"command": "git clean -Xf"}
Failure: Failed assertions: [{'type': 'dir_not_exists', 'path': 'node_modules'}]
z-ai/glm-4.7-flash:xhigh FAIL 66.7% 54 in → 849 out (841 reasoning)
git clean -xf -d
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}]
anthropic/claude-sonnet-5:high__json_schema FAIL 33.3% 302 in → 16 out (0 reasoning)
git clean -Xd
JSON Schema Structured Output
(raw) {"command": "git clean -Xd"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:high FAIL 33.3% 58 in → 1,540 out (1,853 reasoning)
git rm --cached $(gitls-files -i --exclude-standard)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:high__json_schema FAIL 33.3% 58 in → 903 out (1,100 reasoning)
git rm--cached app.log node_modules/pkg/index.js
JSON Schema Structured Output
(raw) { "command": "git rm--cached app.log node_modules/pkg/index.js" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:low FAIL 33.3% 58 in → 847 out (1,011 reasoning)
git rm --cached app.lognode_modules/pkg/index.js
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:low__json_schema FAIL 33.3% 58 in → 899 out (1,094 reasoning)
git rm --cached app.log node_modules/pkg/index.js
JSON Schema Structured Output
(raw) { "command": "git rm --cached app.log node_modules/pkg/index.js" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:medium FAIL 33.3% 58 in → 760 out (905 reasoning)
git rm --cached app.log node_modules/pkg/index.js
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:medium__json_schema FAIL 33.3% 58 in → 968 out (1,184 reasoning)
git rm --cached app.log node_modules/pkg/index.js
JSON Schema Structured Output
(raw) { "command": "git rm --cached app.log node_modules/pkg/index.js" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
arcee-ai/trinity-mini:xhigh FAIL 33.3% 58 in → 936 out (1,139 reasoning)
git rm --cached app.log node_modules/pkg/index.js
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-flash:low__json_schema FAIL 33.3% 57 in → 519 out (487 reasoning)
git clean -Xd
JSON Schema Structured Output
(raw) { "command": "git clean -Xd" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-pro:low FAIL 33.3% 57 in → 461 out (455 reasoning)
git clean -Xd
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
deepseek/deepseek-v4-pro:medium__json_schema FAIL 33.3% 59 in → 724 out (693 reasoning)
git ls-files --ignored --exclude-standard -z | xargs -0 rm -rf
JSON Schema Structured Output
(raw) { "command": "git ls-files --ignored --exclude-standard -z | xargs -0 rm -rf" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemma-4-26b-a4b-it:none FAIL 33.3% 75 in → 5 out (0 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemma-4-26b-a4b-it:xhigh FAIL 33.3% 78 in → 287 out (253 reasoning)
git clean -Xd
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemma-4-31b-it:low FAIL 33.3% 78 in → 254 out (1 reasoning)
git clean -Xd
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemma-4-31b-it:none FAIL 33.3% 75 in → 5 out (0 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
google/gemma-4-31b-it:none__json_schema FAIL 33.3% 75 in → 10 out (0 reasoning)
git clean -X
JSON Schema Structured Output
(raw) {"command":"git clean -X"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
ibm-granite/granite-4.1-8b__json_schema FAIL 33.3% 71 in → 12 out
git clean -fx
JSON Schema Structured Output
(raw) { "command": "git clean -fx" }
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
inclusionai/ling-2.6-flash__json_schema FAIL 33.3% 74 in → 21 out
git clean -fdx -- .gitignore && git checkout -- .gitignore
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- .gitignore && git checkout -- .gitignore"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
liquid/lfm-2-24b-a2b FAIL 33.3% 67 in → 17 out
git rm --ignore-unmatch --cached --quiet untracked.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:high FAIL 33.3% 91 in → 886 out (1,031 reasoning)
clean -fd -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:low FAIL 33.3% 86 in → 195 out (234 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:medium FAIL 33.3% 86 in → 306 out (341 reasoning)
git clean -X -d
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
minimax/minimax-m2.5:xhigh FAIL 33.3% 86 in → 478 out (472 reasoning)
git clean -Xd
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
mistralai/devstral-2512__json_schema FAIL 33.3% 56 in → 12 out
git clean -X -d
JSON Schema Structured Output
(raw) {"command": "git clean -X -d"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-nano-30b-a3b:medium FAIL 33.3% 69 in → 464 out (528 reasoning)
git clean -X -d .
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema FAIL 33.3% 69 in → 374 out (412 reasoning)
git clean -X
JSON Schema Structured Output
(raw) { "command": "git clean -X" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-nano-30b-a3b:none FAIL 33.3% 69 in → 9 out (0 reasoning)
git clean -fdx --ignored
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema FAIL 33.3% 69 in → 383 out (411 reasoning)
git clean -X
JSON Schema Structured Output
(raw) { "command": "git clean -X" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
openai/gpt-5.4-nano:low FAIL 33.3% 54 in → 12 out (0 reasoning)
git clean -fdX --dry-run
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
openai/gpt-5.4-nano:none FAIL 33.3% 54 in → 19 out (0 reasoning)
git clean -ndX -e untracked.txt -e untracked.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
openai/gpt-5.4-nano:none__json_schema FAIL 33.3% 89 in → 20 out (0 reasoning)
git clean -fdX --dry-run
JSON Schema Structured Output
(raw) {"command":"git clean -fdX --dry-run"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
openai/gpt-5.6-luna:low__json_schema FAIL 33.3% 89 in → 18 out (0 reasoning)
git clean -X -d
JSON Schema Structured Output
(raw) {"command":"git clean -X -d"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-m.1:medium__json_schema FAIL 33.3% 66 in → 359 out (348 reasoning)
git clean -Xd
JSON Schema Structured Output
(raw) {"command": "git clean -Xd"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-m.1:none FAIL 33.3% 66 in → 30 out (0 reasoning)
To remove only ignored files while preserving untracked unignored files, use the following command: ```bash git clean -Xd ```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:low FAIL 33.3% 66 in → 275 out (268 reasoning)
git clean -X -d
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs-2.1:none__json_schema FAIL 33.3% 66 in → 23 out (0 reasoning)
git rm -r --cached $(git check-files -i)
JSON Schema Structured Output
(raw) { "command": "git rm -r --cached $(git check-files -i)" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:high__json_schema FAIL 33.3% 103 in → 328 out (315 reasoning)
git clean -d -X
JSON Schema Structured Output
(raw) {"command": "git clean -d -X"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:low FAIL 33.3% 103 in → 249 out (240 reasoning)
git clean -d -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:medium FAIL 33.3% 103 in → 487 out (478 reasoning)
git clean -d -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:none FAIL 33.3% 103 in → 6 out (0 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
poolside/laguna-xs.2:xhigh__json_schema FAIL 33.3% 103 in → 254 out (236 reasoning)
git clean -d -X
JSON Schema Structured Output
(raw) { "command": "git clean -d -X" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-27b:high__json_schema FAIL 33.3% 62 in → 1,284 out (1,263 reasoning)
git clean -X
JSON Schema Structured Output
(raw) { "command": "git clean -X" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-27b:none FAIL 33.3% 64 in → 5 out (0 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-35b-a3b:low FAIL 33.3% 62 in → 1,307 out (1,282 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-35b-a3b:low__json_schema FAIL 33.3% 62 in → 2,033 out (1,951 reasoning)
git clean -X
JSON Schema Structured Output
(raw) { "command": "git clean -X" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-35b-a3b:medium__json_schema FAIL 33.3% 62 in → 1,064 out (993 reasoning)
git clean -X
JSON Schema Structured Output
(raw) { "command": "git clean -X" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 33.3% 64 in → 18 out (0 reasoning)
git clean -Xd .gitignore
JSON Schema Structured Output
(raw) { "command": "git clean -Xd .gitignore" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.7-flash:none FAIL 33.3% 64 in → 4 out (0 reasoning)
git clean -X
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
qwen/qwen3.7-plus:xhigh FAIL 33.3% 62 in → 1,934 out (1,923 reasoning)
git clean -X -d
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
thinkingmachines/inkling:none FAIL 33.3% 61 in → 10 out (0 reasoning)
git clean -X -d
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
x-ai/grok-4.5:high FAIL 33.3% 257 in → 305 out (298 reasoning)
```git clean -fdX```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
z-ai/glm-4.7-flash:high__json_schema FAIL 33.3% 54 in → 884 out (867 reasoning)
git clean -xd
JSON Schema Structured Output
(raw) {"command": "git clean -xd"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
z-ai/glm-4.7-flash:low FAIL 33.3% 54 in → 640 out (640 reasoning)
git clean -Xd
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
z-ai/glm-4.7-flash:medium FAIL 33.3% 54 in → 1,116 out (1,143 reasoning)
git clean -xd
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdX
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git rm --ignore-unmatch -r --prune=nowait untracked.txt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: OpenAI-compatible response contained provider error: Provider returned error
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
poolside/laguna-xs.2:none__json_schema FAIL 0% 103 in → 17 out (0 reasoning)
git clean -d --exclude="." -f
JSON Schema Structured Output
(raw) {"command": "git clean -d --exclude=\".\" -f"}
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}, {'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]
Invalid JSON. Output: git clean -fdX
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9c4f2663-7282-950a-9902-f21cc84233bd","request_id":"9c4f2663-7282-950a-9902-f21cc84233bd"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9ebc5bd7-0ead-9f77-aab0-5f51bec6af5e","request_id":"9ebc5bd7-0ead-9f77-aab0-5f51bec6af5e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-53086ddc-ff4e-9bce-a3a9-429a11b9c528","request_id":"53086ddc-ff4e-9bce-a3a9-429a11b9c528"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8a57f41f-d7bb-9f12-9232-458129e2acc9","request_id":"8a57f41f-d7bb-9f12-9232-458129e2acc9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0355ba82-d815-92e1-a8f7-87fa637a9220","request_id":"0355ba82-d815-92e1-a8f7-87fa637a9220"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-79c902ab-4517-9c64-9845-b2284485f7ca"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-17dab352-d829-9cab-97d2-4e5d2d21f2bb"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-95111f33-6c4e-9cce-8e24-6e52ef938400"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-78059940-9819-9ef5-8fb5-346fed649b3c"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-8e13503e-b158-9892-beab-28aea043fd85"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-272fd665-14e1-9569-84dc-17b0c3e22ee9"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-96966819-cd6b-90a9-9834-323197fce27d","request_id":"96966819-cd6b-90a9-9834-323197fce27d"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-bb04fee6-d22b-95b7-8771-a21c922af945","request_id":"bb04fee6-d22b-95b7-8771-a21c922af945"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-31038694-b173-9593-99e0-775690b8d8f8","request_id":"31038694-b173-9593-99e0-775690b8d8f8"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5cff8b2c-c879-9bea-8787-71a7654c32c9","request_id":"5cff8b2c-c879-9bea-8787-71a7654c32c9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-30b963f6-8dba-9bdd-b620-0d02edec9263","request_id":"30b963f6-8dba-9bdd-b620-0d02edec9263"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-bcaf3627-4347-9b10-8c9d-ec65f86dd9cc","request_id":"bcaf3627-4347-9b10-8c9d-ec65f86dd9cc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-24194378-9cdc-9f76-a876-ae4738482a16","request_id":"24194378-9cdc-9f76-a876-ae4738482a16"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-fedf230f-da03-9756-a289-fd0ce6e750d2","request_id":"fedf230f-da03-9756-a289-fd0ce6e750d2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-00e35290-4c57-9c5e-aa2c-5bc8441775d5","request_id":"00e35290-4c57-9c5e-aa2c-5bc8441775d5"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-fef947a8-364c-9531-8aa1-db488cf04e6e","request_id":"fef947a8-364c-9531-8aa1-db488cf04e6e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git clean -f -X{ "command": "git clean -f -X" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -f -X -d
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdX
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdX
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-4.7-flash:medium__json_schema FAIL 0% 54 in → 578 out (552 reasoning)
git clean -fd
JSON Schema Structured Output
(raw) {"command": "git clean -fd"}
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'untracked.txt'}, {'type': 'file_not_exists', 'path': 'app.log'}, {'type': 'dir_not_exists', 'path': 'node_modules'}]