Combined cleanup of untracked and ignored with directories in specific path
Tests ability to perform combined cleanup of untracked and ignored files with directories in a specific path. Evaluates advanced multi-flag cleanup.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 printf '*.log tmp/ ' > .gitignore
  5. 05 mkdir -p src
  6. 06 echo 'main' > src/app.py
  7. 07 git add .gitignore src
  8. 08 git commit -m 'Initial commit'
  9. 09 echo 'debug' > src/debug.log
  10. 10 mkdir -p src/tmp
  11. 11 echo 'cache' > src/tmp/data
  12. 12 echo 'untracked' > src/new.py
  13. 13 echo 'root junk' > root.log
Prompt
Remove all untracked and ignored files and directories from the src/ directory only. Output ONLY the git command, nothing else.
Expected
git clean -f -x -d src/

Scoped model quality, cost, API time, and token usage for git_clean/f012.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 96 in → 22 out (29 reasoning)
git clean -fdx src/
anthropic/claude-fable-5:high__json_schema PASS 100% 306 in → 29 out (29 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-fable-5:low PASS 100% 96 in → 26 out (29 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-fable-5:low__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-fable-5:max PASS 100% 96 in → 262 out (74 reasoning)
git clean -fdx src/
anthropic/claude-fable-5:max__json_schema PASS 100% 306 in → 138 out (54 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-fable-5:medium PASS 100% 96 in → 26 out (29 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-fable-5:xhigh PASS 100% 96 in → 25 out (29 reasoning)
git clean -fdx src/
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 306 in → 29 out (29 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-haiku-4.5:high PASS 100% 108 in → 470 out (406 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 271 in → 627 out (503 reasoning)
git clean -fxd src/
JSON Schema Structured Output
(raw) {"command": "git clean -fxd src/"}
anthropic/claude-haiku-4.5:low PASS 100% 108 in → 373 out (316 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 271 in → 264 out (226 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-haiku-4.5:medium PASS 100% 108 in → 437 out (380 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 271 in → 321 out (267 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 108 in → 378 out (333 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 271 in → 317 out (287 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-opus-4.6:high PASS 100% 79 in → 124 out (93 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 242 in → 94 out (67 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.6:low PASS 100% 79 in → 14 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 242 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.6:max PASS 100% 79 in → 229 out (178 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 242 in → 124 out (97 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.6:medium PASS 100% 79 in → 14 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 242 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.6:none PASS 100% 79 in → 14 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 242 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.6:xhigh PASS 100% 79 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 242 in → 94 out (67 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.7:high PASS 100% 101 in → 15 out (0 reasoning)
`git clean -fdx src/`
anthropic/claude-opus-4.7:high__json_schema PASS 100% 311 in → 20 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.7:low PASS 100% 101 in → 13 out (0 reasoning)
git clean -fdx src/
anthropic/claude-opus-4.7:low__json_schema PASS 100% 311 in → 20 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.7:max PASS 100% 101 in → 292 out (59 reasoning)
`git clean -fdx src/`
anthropic/claude-opus-4.7:max__json_schema PASS 100% 311 in → 20 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.7:medium PASS 100% 101 in → 13 out (0 reasoning)
git clean -fdx src/
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 311 in → 20 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.7:none PASS 100% 101 in → 15 out (0 reasoning)
`git clean -fdx src/`
anthropic/claude-opus-4.7:none__json_schema PASS 100% 311 in → 20 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.7:xhigh PASS 100% 101 in → 15 out (0 reasoning)
`git clean -fdx src/`
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 311 in → 20 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.8:high PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.8:low PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-opus-4.8:max PASS 100% 96 in → 383 out (76 reasoning)
`git clean -fdx src/`
anthropic/claude-opus-4.8:max__json_schema PASS 100% 306 in → 155 out (65 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-opus-4.8:medium PASS 100% 96 in → 11 out (0 reasoning)
git clean -fdx src/
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.8:none PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 306 in → 17 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-opus-4.8:xhigh PASS 100% 96 in → 11 out (0 reasoning)
git clean -fdx src/
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-4.6:high PASS 100% 79 in → 125 out (95 reasoning)
``` git clean -fxd src/ ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 242 in → 122 out (93 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-4.6:low PASS 100% 79 in → 14 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 242 in → 15 out (0 reasoning)
git clean -fxd src/
JSON Schema Structured Output
(raw) {"command":"git clean -fxd src/"}
anthropic/claude-sonnet-4.6:max PASS 100% 79 in → 282 out (218 reasoning)
``` git clean -fxd src/ ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 242 in → 242 out (196 reasoning)
git clean -fxd src/
JSON Schema Structured Output
(raw) {"command":"git clean -fxd src/"}
anthropic/claude-sonnet-4.6:medium PASS 100% 79 in → 111 out (82 reasoning)
``` git clean -fxd src/ ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 242 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-4.6:none PASS 100% 79 in → 14 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 242 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 79 in → 207 out (175 reasoning)
``` git clean -fxd src/ ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 242 in → 120 out (92 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-5:high PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-5:low PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-5:low__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-sonnet-5:medium PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
anthropic/claude-sonnet-5:none PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 306 in → 17 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
anthropic/claude-sonnet-5:xhigh PASS 100% 96 in → 15 out (0 reasoning)
``` git clean -fdx src/ ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 306 in → 18 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
arcee-ai/trinity-large-thinking:high PASS 100% 64 in → 861 out (862 reasoning)
git clean -fdx src/
arcee-ai/trinity-large-thinking:low PASS 100% 64 in → 979 out (979 reasoning)
git clean -fdx src/
arcee-ai/trinity-large-thinking:medium PASS 100% 64 in → 751 out (746 reasoning)
git clean -xdf src/
arcee-ai/trinity-large-thinking:xhigh PASS 100% 64 in → 699 out (696 reasoning)
git clean -fdx -- src/
deepseek/deepseek-v4-flash-0731:high PASS 100% 63 in → 317 out (312 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 63 in → 224 out (210 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 63 in → 222 out (218 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash-0731:max PASS 100% 155 in → 301 out (288 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 142 in → 295 out (290 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 63 in → 137 out (129 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 63 in → 448 out (431 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 63 in → 12 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 63 in → 324 out (307 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 63 in → 356 out (340 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/" }
deepseek/deepseek-v4-flash:high PASS 100% 63 in → 697 out (688 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 63 in → 499 out (484 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
deepseek/deepseek-v4-flash:low PASS 100% 63 in → 525 out (488 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 63 in → 131 out (121 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
deepseek/deepseek-v4-flash:medium PASS 100% 63 in → 438 out (447 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-flash:xhigh PASS 100% 142 in → 536 out (527 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-pro:high PASS 100% 63 in → 287 out (277 reasoning)
git clean -fd -x src/
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 250 in → 289 out (275 reasoning)
git clean -xdf src/
JSON Schema Structured Output
(raw) {"command": "git clean -xdf src/"}
deepseek/deepseek-v4-pro:low PASS 100% 63 in → 8 out (0 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-pro:medium PASS 100% 63 in → 394 out (381 reasoning)
```bash git clean -fdx src/ ```
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 250 in → 481 out (467 reasoning)
git clean -xdf src/
JSON Schema Structured Output
(raw) {"command": "git clean -xdf src/"}
deepseek/deepseek-v4-pro:xhigh PASS 100% 142 in → 1,234 out (1,225 reasoning)
git clean -fdx src/
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 328 in → 1,566 out (1,552 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3-flash-preview:high PASS 100% 69 in → 337 out (330 reasoning)
git clean -fdx src/
google/gemini-3-flash-preview:high__json_schema PASS 100% 126 in → 278 out (263 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemini-3-flash-preview:low PASS 100% 69 in → 232 out (225 reasoning)
git clean -fdx src/
google/gemini-3-flash-preview:low__json_schema PASS 100% 126 in → 407 out (396 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3-flash-preview:medium PASS 100% 69 in → 218 out (211 reasoning)
git clean -fdx src/
google/gemini-3-flash-preview:medium__json_schema PASS 100% 126 in → 332 out (321 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3-flash-preview:xhigh PASS 100% 69 in → 394 out (387 reasoning)
git clean -fdx src/
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 126 in → 210 out (199 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 69 in → 321 out (314 reasoning)
git clean -fdx src/
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 70 in → 431 out (419 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 69 in → 138 out (131 reasoning)
git clean -fdx src/
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 126 in → 109 out (97 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 69 in → 247 out (240 reasoning)
git clean -fdx src/
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 126 in → 252 out (239 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx -- src/"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 70 in → 631 out (624 reasoning)
git clean -fdx src/
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 70 in → 450 out (438 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.1-pro-preview:high PASS 100% 69 in → 351 out (344 reasoning)
git clean -fdx src/
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 126 in → 350 out (338 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.1-pro-preview:low PASS 100% 69 in → 374 out (367 reasoning)
git clean -fdx src/
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 126 in → 293 out (282 reasoning)
git clean -fxd src/
JSON Schema Structured Output
(raw) {"command":"git clean -fxd src/"}
google/gemini-3.1-pro-preview:medium PASS 100% 69 in → 455 out (448 reasoning)
git clean -fdx src/
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 126 in → 366 out (355 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 69 in → 376 out (369 reasoning)
git clean -fdx src/
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 126 in → 439 out (428 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.5-flash-lite:high PASS 100% 70 in → 361 out (354 reasoning)
git clean -fdx src/
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 70 in → 533 out (521 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.5-flash-lite:max PASS 100% 69 in → 360 out (353 reasoning)
git clean -fdx src/
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 70 in → 711 out (700 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.5-flash-lite:medium PASS 100% 69 in → 415 out (408 reasoning)
git clean -fdx src/
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 70 in → 378 out (366 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 69 in → 637 out (630 reasoning)
git clean -fdx src/
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 70 in → 539 out (527 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.5-flash:high PASS 100% 69 in → 619 out (612 reasoning)
git clean -xdf src/
google/gemini-3.5-flash:high__json_schema PASS 100% 126 in → 587 out (575 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.5-flash:low PASS 100% 69 in → 474 out (467 reasoning)
git clean -xdf src/
google/gemini-3.5-flash:low__json_schema PASS 100% 126 in → 264 out (252 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.5-flash:medium PASS 100% 69 in → 476 out (469 reasoning)
git clean -fdx src/
google/gemini-3.5-flash:medium__json_schema PASS 100% 126 in → 391 out (379 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.5-flash:xhigh PASS 100% 69 in → 535 out (528 reasoning)
git clean -fdx src/
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 126 in → 480 out (468 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemini-3.6-flash:high PASS 100% 69 in → 449 out (442 reasoning)
git clean -fdx src/
google/gemini-3.6-flash:high__json_schema PASS 100% 70 in → 329 out (318 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.6-flash:low PASS 100% 70 in → 188 out (181 reasoning)
git clean -fdx src/
google/gemini-3.6-flash:low__json_schema PASS 100% 70 in → 11 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.6-flash:max PASS 100% 70 in → 547 out (540 reasoning)
git clean -fdx src/
google/gemini-3.6-flash:max__json_schema PASS 100% 70 in → 289 out (278 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.6-flash:medium PASS 100% 70 in → 497 out (490 reasoning)
git clean -fdx src/
google/gemini-3.6-flash:medium__json_schema PASS 100% 126 in → 311 out (300 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemini-3.6-flash:xhigh PASS 100% 70 in → 319 out (312 reasoning)
git clean -fdx src/
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 126 in → 349 out (333 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemma-4-26b-a4b-it:high PASS 100% 85 in → 271 out (263 reasoning)
git clean -fdx src/
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 85 in → 446 out (338 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemma-4-26b-a4b-it:low PASS 100% 85 in → 500 out (492 reasoning)
git clean -fdx src/
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 84 in → 291 out (228 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemma-4-26b-a4b-it:medium PASS 100% 85 in → 637 out (629 reasoning)
git clean -fdx src/
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 85 in → 352 out (278 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 85 in → 421 out (332 reasoning)
git clean -fdx src/
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 85 in → 280 out (221 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemma-4-31b-it:high PASS 100% 84 in → 231 out (185 reasoning)
git clean -fdx src/
google/gemma-4-31b-it:high__json_schema PASS 100% 85 in → 231 out (174 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
google/gemma-4-31b-it:low PASS 100% 85 in → 314 out (1 reasoning)
git clean -fdx src/
google/gemma-4-31b-it:low__json_schema PASS 100% 84 in → 214 out (175 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
google/gemma-4-31b-it:medium PASS 100% 88 in → 320 out (312 reasoning)
git clean -fdx src/
google/gemma-4-31b-it:medium__json_schema PASS 100% 88 in → 207 out (194 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
google/gemma-4-31b-it:xhigh PASS 100% 84 in → 348 out (287 reasoning)
git clean -fdx src/
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 84 in → 258 out (205 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
minimax/minimax-m2.5:high PASS 100% 92 in → 679 out (669 reasoning)
git clean -fdx -- src/
minimax/minimax-m2.5:high__json_schema PASS 100% 92 in → 836 out (907 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
minimax/minimax-m2.5:low PASS 100% 92 in → 362 out (353 reasoning)
git clean -fdx src/
minimax/minimax-m2.5:low__json_schema PASS 100% 92 in → 1,175 out (1,250 reasoning)
git clean -fd -x -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fd -x -- src/" }
minimax/minimax-m2.5:medium PASS 100% 94 in → 449 out (440 reasoning)
git clean -fdx -- src/
minimax/minimax-m2.5:medium__json_schema PASS 100% 109 in → 1,315 out (1,398 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
minimax/minimax-m2.5:xhigh PASS 100% 92 in → 743 out (795 reasoning)
git clean -fdx src/
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 92 in → 1,686 out (1,817 reasoning)
git -C src clean -fdx
JSON Schema Structured Output
(raw) { "command": "git -C src clean -fdx" }
minimax/minimax-m2.7:high PASS 100% 92 in → 1,258 out (1,248 reasoning)
git clean -fdx src/
minimax/minimax-m2.7:high__json_schema PASS 100% 224 in → 933 out (916 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
minimax/minimax-m2.7:low PASS 100% 94 in → 914 out (905 reasoning)
git clean -fdx -- src/
minimax/minimax-m2.7:low__json_schema PASS 100% 224 in → 783 out (770 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
minimax/minimax-m2.7:medium PASS 100% 92 in → 1,395 out (1,385 reasoning)
git clean -fdx src/
minimax/minimax-m2.7:medium__json_schema PASS 100% 224 in → 1,034 out (1,021 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
minimax/minimax-m2.7:xhigh PASS 100% 92 in → 1,261 out (1,251 reasoning)
git clean -fdx src/
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 91 in → 669 out (707 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
minimax/minimax-m3:high PASS 100% 230 in → 139 out (139 reasoning)
git clean -fdx src/
minimax/minimax-m3:high__json_schema PASS 100% 217 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
minimax/minimax-m3:low PASS 100% 230 in → 203 out (202 reasoning)
git clean -fdx src/
minimax/minimax-m3:medium PASS 100% 227 in → 460 out (469 reasoning)
git clean -fdx src/
minimax/minimax-m3:xhigh PASS 100% 230 in → 149 out (154 reasoning)
`git clean -fdx src/`
minimax/minimax-m3:xhigh__json_schema PASS 100% 217 in → 20 out (0 reasoning)
git clean -fX src/ && git clean -fd src/
JSON Schema Structured Output
(raw) {"command": "git clean -fX src/ && git clean -fd src/"}
mistralai/mistral-medium-3-5:high PASS 100% 74 in → 3,511 out (3,725 reasoning)
git clean -fdx -- src/
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 62 in → 393 out (405 reasoning)
git clean -fd -x -- src/
JSON Schema Structured Output
(raw) {"command": "git clean -fd -x -- src/"}
mistralai/mistral-medium-3-5:low PASS 100% 74 in → 528 out (553 reasoning)
git clean -fdx src/
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 62 in → 327 out (319 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
mistralai/mistral-medium-3-5:medium PASS 100% 74 in → 568 out (587 reasoning)
git clean -dfx src/
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 62 in → 386 out (390 reasoning)
git clean -fxd -- src/
JSON Schema Structured Output
(raw) {"command": "git clean -fxd -- src/"}
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 62 in → 17 out (0 reasoning)
git clean -f -d -x -- src/
JSON Schema Structured Output
(raw) {"command": "git clean -f -d -x -- src/"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 74 in → 1,093 out (1,145 reasoning)
git clean -dfx src/
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 62 in → 336 out (334 reasoning)
git clean -fx -- src/
JSON Schema Structured Output
(raw) {"command": "git clean -fx -- src/"}
moonshotai/kimi-k2.6:high PASS 100% 64 in → 3,078 out (3,306 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.6:high__json_schema PASS 100% 64 in → 1,400 out (1,386 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
moonshotai/kimi-k2.6:low PASS 100% 64 in → 2,011 out (2,002 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.6:low__json_schema PASS 100% 63 in → 1,058 out (834 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
moonshotai/kimi-k2.6:medium PASS 100% 64 in → 626 out (641 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 64 in → 1,051 out (1,067 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
moonshotai/kimi-k2.6:xhigh PASS 100% 64 in → 2,146 out (2,137 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 64 in → 1,650 out (1,753 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
moonshotai/kimi-k2.7-code:high PASS 100% 64 in → 214 out (211 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.7-code:low PASS 100% 64 in → 214 out (226 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 64 in → 142 out (127 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
moonshotai/kimi-k2.7-code:medium PASS 100% 64 in → 371 out (379 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 64 in → 314 out (298 reasoning)
git clean -fdx src
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 64 in → 345 out (336 reasoning)
git clean -fdx src/
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 64 in → 279 out (291 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
moonshotai/kimi-k3:high PASS 100% 141 in → 266 out (243 reasoning)
git clean -fdx -- src/
moonshotai/kimi-k3:high__json_schema PASS 100% 141 in → 197 out (175 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:low PASS 100% 141 in → 136 out (114 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:low__json_schema PASS 100% 141 in → 166 out (142 reasoning)
``` git clean -fdx src/ ```
moonshotai/kimi-k3:max PASS 100% 141 in → 157 out (135 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:max__json_schema PASS 100% 141 in → 171 out (149 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:medium PASS 100% 141 in → 181 out (159 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:medium__json_schema PASS 100% 141 in → 239 out (217 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:xhigh PASS 100% 141 in → 134 out (112 reasoning)
git clean -fdx src/
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 141 in → 182 out (160 reasoning)
git clean -fdx src/
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 75 in → 567 out (594 reasoning)
git clean -xfd src/
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 75 in → 581 out (612 reasoning)
git clean -xfd src/
JSON Schema Structured Output
(raw) { "command": "git clean -xfd src/" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 75 in → 632 out (660 reasoning)
git clean -fdx src/
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 75 in → 320 out (338 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 75 in → 265 out (269 reasoning)
git clean -xfd src/
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 75 in → 657 out (683 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 75 in → 839 out (806 reasoning)
git clean -f -x -- src/
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 75 in → 330 out (323 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 75 in → 386 out (390 reasoning)
git clean -f -d -x src/
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 75 in → 132 out (113 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 75 in → 147 out (135 reasoning)
git clean -fdx src/
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 75 in → 255 out (227 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 75 in → 166 out (150 reasoning)
git clean -fdx src/
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 75 in → 229 out (211 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 75 in → 15 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 75 in → 165 out (153 reasoning)
git clean -fdx src/
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 75 in → 306 out (278 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
openai/gpt-5.4-mini:high PASS 100% 60 in → 189 out (176 reasoning)
git clean -fdx src/
openai/gpt-5.4-mini:high__json_schema PASS 100% 95 in → 202 out (180 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4-mini:low PASS 100% 60 in → 80 out (66 reasoning)
git clean -fdx -- src/
openai/gpt-5.4-mini:low__json_schema PASS 100% 95 in → 106 out (84 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4-mini:medium PASS 100% 60 in → 130 out (117 reasoning)
git clean -fdx src/
openai/gpt-5.4-mini:medium__json_schema PASS 100% 95 in → 211 out (189 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4-mini:none PASS 100% 60 in → 12 out (0 reasoning)
git clean -fdx -- src/
openai/gpt-5.4-mini:none__json_schema PASS 100% 95 in → 20 out (0 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4-mini:xhigh PASS 100% 60 in → 431 out (417 reasoning)
git clean -fdx -- src/
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 95 in → 538 out (516 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4-nano:high__json_schema PASS 100% 95 in → 109 out (87 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4-nano:medium PASS 100% 60 in → 11 out (0 reasoning)
git clean -fdx src/
openai/gpt-5.4-nano:xhigh PASS 100% 60 in → 424 out (410 reasoning)
git clean -fdx -- src/
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 95 in → 327 out (305 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4:high PASS 100% 60 in → 155 out (141 reasoning)
git clean -fdx -- src/
openai/gpt-5.4:high__json_schema PASS 100% 95 in → 219 out (197 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4:low PASS 100% 60 in → 67 out (53 reasoning)
git clean -fdx -- src/
openai/gpt-5.4:low__json_schema PASS 100% 95 in → 67 out (45 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4:medium PASS 100% 60 in → 105 out (91 reasoning)
git clean -fdx -- src/
openai/gpt-5.4:medium__json_schema PASS 100% 95 in → 133 out (111 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4:none PASS 100% 60 in → 20 out (0 reasoning)
git clean -fdX -f src && git clean -fd src/new.py
openai/gpt-5.4:none__json_schema PASS 100% 95 in → 20 out (0 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.4:xhigh PASS 100% 60 in → 406 out (392 reasoning)
git clean -fdx -- src/
openai/gpt-5.4:xhigh__json_schema PASS 100% 95 in → 179 out (157 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.5:high PASS 100% 60 in → 130 out (116 reasoning)
git clean -fdx -- src/
openai/gpt-5.5:high__json_schema PASS 100% 95 in → 115 out (93 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.5:low PASS 100% 60 in → 96 out (82 reasoning)
git clean -fdx -- src/
openai/gpt-5.5:low__json_schema PASS 100% 95 in → 79 out (57 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.5:medium PASS 100% 60 in → 84 out (70 reasoning)
git clean -fdx -- src/
openai/gpt-5.5:medium__json_schema PASS 100% 95 in → 108 out (86 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.5:none PASS 100% 60 in → 18 out (0 reasoning)
git clean -fdX src/ && git clean -fd src/
openai/gpt-5.5:none__json_schema PASS 100% 95 in → 20 out (0 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.5:xhigh PASS 100% 60 in → 174 out (160 reasoning)
git clean -fdx -- src/
openai/gpt-5.5:xhigh__json_schema PASS 100% 95 in → 195 out (173 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-luna:high PASS 100% 60 in → 81 out (67 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-luna:high__json_schema PASS 100% 95 in → 131 out (109 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-luna:low PASS 100% 60 in → 49 out (35 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-luna:low__json_schema PASS 100% 95 in → 57 out (35 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-luna:max PASS 100% 60 in → 206 out (192 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-luna:max__json_schema PASS 100% 95 in → 473 out (451 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-luna:medium PASS 100% 60 in → 103 out (89 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-luna:medium__json_schema PASS 100% 95 in → 205 out (183 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-luna:none PASS 100% 60 in → 18 out (0 reasoning)
git clean -fdX src/ && git clean -fd src/
openai/gpt-5.6-luna:none__json_schema PASS 100% 95 in → 26 out (0 reasoning)
git clean -fdX src/ && git clean -fd src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdX src/ && git clean -fd src/"}
openai/gpt-5.6-luna:xhigh PASS 100% 60 in → 105 out (91 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 95 in → 150 out (128 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-sol:high PASS 100% 60 in → 55 out (41 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-sol:high__json_schema PASS 100% 95 in → 104 out (82 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-sol:low PASS 100% 60 in → 12 out (0 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-sol:low__json_schema PASS 100% 95 in → 103 out (81 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-sol:max PASS 100% 60 in → 72 out (58 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-sol:max__json_schema PASS 100% 95 in → 224 out (202 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-sol:medium PASS 100% 60 in → 12 out (0 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-sol:medium__json_schema PASS 100% 95 in → 121 out (99 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-sol:none PASS 100% 60 in → 12 out (0 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-sol:none__json_schema PASS 100% 95 in → 20 out (0 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-sol:xhigh PASS 100% 60 in → 92 out (78 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 95 in → 141 out (119 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-terra:high PASS 100% 60 in → 74 out (60 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-terra:high__json_schema PASS 100% 95 in → 51 out (29 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-terra:low PASS 100% 60 in → 44 out (30 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-terra:low__json_schema PASS 100% 95 in → 20 out (0 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-terra:max PASS 100% 60 in → 113 out (99 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-terra:max__json_schema PASS 100% 95 in → 151 out (129 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-terra:medium PASS 100% 60 in → 49 out (35 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-terra:medium__json_schema PASS 100% 95 in → 50 out (28 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-terra:none PASS 100% 60 in → 11 out (0 reasoning)
git clean -fdx src/
openai/gpt-5.6-terra:none__json_schema PASS 100% 95 in → 20 out (0 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-5.6-terra:xhigh PASS 100% 60 in → 91 out (77 reasoning)
git clean -fdx -- src/
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 95 in → 143 out (121 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx -- src/"}
openai/gpt-oss-120b:high PASS 100% 121 in → 147 out (130 reasoning)
git clean -fdx src/
openai/gpt-oss-120b:high__json_schema PASS 100% 119 in → 571 out (573 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
openai/gpt-oss-120b:low PASS 100% 119 in → 102 out (85 reasoning)
git clean -fdx src/
openai/gpt-oss-120b:low__json_schema PASS 100% 108 in → 61 out (40 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
openai/gpt-oss-120b:medium PASS 100% 108 in → 204 out (185 reasoning)
git clean -fdx src/
openai/gpt-oss-120b:medium__json_schema PASS 100% 119 in → 159 out (155 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
openai/gpt-oss-120b:xhigh PASS 100% 119 in → 599 out (636 reasoning)
git clean -fdx -- src/
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 121 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
openai/gpt-oss-20b:high PASS 100% 119 in → 9,339 out (9,991 reasoning)
git clean -fdx src
openai/gpt-oss-20b:low__json_schema PASS 100% 108 in → 67 out (37 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
openai/gpt-oss-20b:medium PASS 100% 123 in → 508 out (526 reasoning)
git clean -fdx src/
openai/gpt-oss-20b:xhigh PASS 100% 119 in → 842 out (852 reasoning)
git clean -fdx src/
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 119 in → 752 out (808 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
poolside/laguna-m.1:high PASS 100% 72 in → 233 out (223 reasoning)
git clean -fdx src/
poolside/laguna-m.1:low__json_schema PASS 100% 72 in → 445 out (431 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
poolside/laguna-m.1:medium PASS 100% 72 in → 352 out (342 reasoning)
git clean -fdx src/
poolside/laguna-m.1:medium__json_schema PASS 100% 72 in → 785 out (768 reasoning)
git clean -xdf src/
JSON Schema Structured Output
(raw) { "command": "git clean -xdf src/" }
poolside/laguna-m.1:xhigh__json_schema PASS 100% 72 in → 651 out (637 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
poolside/laguna-xs-2.1:high PASS 100% 72 in → 1,115 out (1,107 reasoning)
git clean -xf -- src/
poolside/laguna-xs-2.1:high__json_schema PASS 100% 72 in → 543 out (529 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
poolside/laguna-xs-2.1:low PASS 100% 72 in → 482 out (473 reasoning)
git clean -f -x src/
poolside/laguna-xs-2.1:low__json_schema PASS 100% 72 in → 1,210 out (1,196 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx src/"}
poolside/laguna-xs-2.1:medium PASS 100% 72 in → 1,079 out (1,071 reasoning)
git clean -fdx src/
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 72 in → 427 out (409 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
poolside/laguna-xs-2.1:none PASS 100% 72 in → 14 out (0 reasoning)
git clean -fdX src/ && git clean -fd src/
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 72 in → 467 out (449 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
poolside/laguna-xs.2:high PASS 100% 109 in → 274 out (264 reasoning)
git clean -fdx src/
poolside/laguna-xs.2:high__json_schema PASS 100% 109 in → 375 out (360 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdx -- src/"}
poolside/laguna-xs.2:low PASS 100% 109 in → 211 out (201 reasoning)
git clean -fdx src/
poolside/laguna-xs.2:low__json_schema PASS 100% 109 in → 235 out (216 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
poolside/laguna-xs.2:medium PASS 100% 109 in → 281 out (271 reasoning)
git clean -fdx src/
poolside/laguna-xs.2:medium__json_schema PASS 100% 109 in → 261 out (243 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
poolside/laguna-xs.2:xhigh PASS 100% 109 in → 289 out (278 reasoning)
git clean -fdx -- src/
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 109 in → 276 out (263 reasoning)
git clean -fx src/
JSON Schema Structured Output
(raw) {"command": "git clean -fx src/"}
qwen/qwen3.6-27b:high PASS 100% 68 in → 1,521 out (1,383 reasoning)
git clean -fdx src/
qwen/qwen3.6-27b:high__json_schema PASS 100% 68 in → 1,645 out (1,492 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
qwen/qwen3.6-27b:low PASS 100% 68 in → 1,707 out (1,526 reasoning)
git clean -fdx src/
qwen/qwen3.6-27b:medium PASS 100% 68 in → 1,694 out (1,579 reasoning)
git clean -fdx src/
qwen/qwen3.6-27b:medium__json_schema PASS 100% 68 in → 1,731 out (1,562 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
qwen/qwen3.6-27b:xhigh PASS 100% 68 in → 1,707 out (1,581 reasoning)
git clean -fdx src/
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 68 in → 1,355 out (1,228 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
qwen/qwen3.6-35b-a3b:high PASS 100% 68 in → 2,197 out (2,130 reasoning)
git clean -xfd src/
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 68 in → 3,146 out (2,808 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
qwen/qwen3.6-35b-a3b:low PASS 100% 68 in → 1,408 out (1,285 reasoning)
git clean -xfd src/
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 68 in → 1,180 out (1,073 reasoning)
git clean -xfd src/
JSON Schema Structured Output
(raw) { "command": "git clean -xfd src/" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 68 in → 778 out (727 reasoning)
git clean -dfx src/
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 68 in → 1,388 out (1,317 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 68 in → 1,591 out (1,414 reasoning)
git clean -dfx src/
JSON Schema Structured Output
(raw) { "command": "git clean -dfx src/" }
qwen/qwen3.6-flash:high PASS 100% 68 in → 1,942 out (1,929 reasoning)
git clean -fdx -- src/
qwen/qwen3.6-flash:low PASS 100% 68 in → 1,950 out (1,938 reasoning)
git clean -fdx src/
qwen/qwen3.6-flash:medium PASS 100% 68 in → 1,455 out (1,444 reasoning)
git clean -xfd src/
qwen/qwen3.6-flash:none PASS 100% 70 in → 8 out (0 reasoning)
git clean -fdx -- src/
qwen/qwen3.6-flash:xhigh PASS 100% 68 in → 1,945 out (1,934 reasoning)
git clean -xfd src/
qwen/qwen3.7-flash:high PASS 100% 68 in → 570 out (557 reasoning)
git clean -fdx -- src/
qwen/qwen3.7-flash:low PASS 100% 68 in → 1,174 out (1,162 reasoning)
git clean -fdx src/
qwen/qwen3.7-flash:max PASS 100% 68 in → 2,320 out (2,309 reasoning)
git clean -xfd src/
qwen/qwen3.7-flash:medium PASS 100% 68 in → 1,596 out (1,585 reasoning)
git clean -xfd src/
qwen/qwen3.7-flash:xhigh PASS 100% 68 in → 1,327 out (1,314 reasoning)
git clean -fdx -- src/
qwen/qwen3.7-max:high PASS 100% 68 in → 989 out (977 reasoning)
git clean -fdx src/
qwen/qwen3.7-max:low PASS 100% 68 in → 221 out (209 reasoning)
git clean -fdx src/
qwen/qwen3.7-max:medium PASS 100% 68 in → 398 out (386 reasoning)
git clean -fdx src/
qwen/qwen3.7-max:none PASS 100% 70 in → 7 out (0 reasoning)
git clean -fdx src/
qwen/qwen3.7-max:xhigh PASS 100% 68 in → 685 out (673 reasoning)
git clean -fdx src/
qwen/qwen3.7-plus:high PASS 100% 68 in → 594 out (582 reasoning)
git clean -fdx src/
qwen/qwen3.7-plus:low PASS 100% 68 in → 826 out (814 reasoning)
git clean -fdx src/
qwen/qwen3.7-plus:medium PASS 100% 68 in → 953 out (941 reasoning)
git clean -fdx src/
qwen/qwen3.7-plus:none PASS 100% 70 in → 7 out (0 reasoning)
git clean -fdx src/
qwen/qwen3.7-plus:xhigh PASS 100% 68 in → 706 out (694 reasoning)
git clean -fdx src/
tencent/hy3:high PASS 100% 69 in → 1,619 out (1,610 reasoning)
git clean -fdx src/
tencent/hy3:high__json_schema PASS 100% 69 in → 1,266 out (1,160 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
tencent/hy3:low PASS 100% 69 in → 761 out (753 reasoning)
git clean -fdx src
tencent/hy3:low__json_schema PASS 100% 69 in → 631 out (600 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
tencent/hy3:medium PASS 100% 69 in → 821 out (812 reasoning)
git clean -fdx src/
tencent/hy3:medium__json_schema PASS 100% 69 in → 1,347 out (1,192 reasoning)
git clean -fdx src
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src" }
tencent/hy3:none PASS 100% 72 in → 8 out (0 reasoning)
git clean -fdx src/
tencent/hy3:none__json_schema PASS 100% 72 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
tencent/hy3:xhigh PASS 100% 69 in → 912 out (903 reasoning)
git clean -fdx src/
tencent/hy3:xhigh__json_schema PASS 100% 69 in → 2,538 out (2,433 reasoning)
git clean -fdx src
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src" }
thinkingmachines/inkling-small:high PASS 100% 69 in → 198 out (200 reasoning)
git clean -fdx -- src/
thinkingmachines/inkling-small:high__json_schema PASS 100% 69 in → 242 out (222 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
thinkingmachines/inkling-small:low PASS 100% 69 in → 85 out (70 reasoning)
git clean -fdx -- src/
thinkingmachines/inkling-small:low__json_schema PASS 100% 69 in → 86 out (66 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
thinkingmachines/inkling-small:max PASS 100% 69 in → 336 out (323 reasoning)
git clean -fdx src/
thinkingmachines/inkling-small:max__json_schema PASS 100% 69 in → 419 out (400 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
thinkingmachines/inkling-small:medium PASS 100% 69 in → 210 out (196 reasoning)
git clean -fdx -- src/
thinkingmachines/inkling-small:medium__json_schema PASS 100% 69 in → 256 out (237 reasoning)
git clean -fdx -- src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx -- src/" }
thinkingmachines/inkling-small:none PASS 100% 67 in → 12 out (0 reasoning)
git clean -fd -x -- src/
thinkingmachines/inkling-small:xhigh PASS 100% 69 in → 253 out (246 reasoning)
git clean -fdx -- src/
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 69 in → 203 out (184 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
thinkingmachines/inkling:high PASS 100% 69 in → 297 out (284 reasoning)
git clean -fdx src/
thinkingmachines/inkling:max PASS 100% 69 in → 418 out (409 reasoning)
git clean -fdx src/
thinkingmachines/inkling:medium PASS 100% 69 in → 183 out (168 reasoning)
git clean -fdx src/
thinkingmachines/inkling:xhigh PASS 100% 69 in → 300 out (312 reasoning)
git clean -fdx src/
thinkingmachines/inkling:xhigh__json_schema PASS 100% 69 in → 229 out (213 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.3:high PASS 100% 239 in → 732 out (722 reasoning)
```bash git clean -xfd src/ ```
x-ai/grok-4.3:high__json_schema PASS 100% 295 in → 938 out (928 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.3:low PASS 100% 245 in → 301 out (294 reasoning)
git clean -fxd src/
x-ai/grok-4.3:low__json_schema PASS 100% 301 in → 420 out (410 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.3:max PASS 100% 239 in → 567 out (560 reasoning)
git clean -fdx src/
x-ai/grok-4.3:max__json_schema PASS 100% 295 in → 874 out (864 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.3:medium PASS 100% 245 in → 458 out (451 reasoning)
git clean -fdx src/
x-ai/grok-4.3:medium__json_schema PASS 100% 301 in → 748 out (734 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
x-ai/grok-4.3:xhigh PASS 100% 239 in → 732 out (725 reasoning)
git clean -fdx src/
x-ai/grok-4.3:xhigh__json_schema PASS 100% 295 in → 998 out (988 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.5:high PASS 100% 265 in → 346 out (339 reasoning)
git clean -fdx src/
x-ai/grok-4.5:high__json_schema PASS 100% 331 in → 530 out (518 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.5:low PASS 100% 265 in → 200 out (190 reasoning)
``` git clean -fdx src/ ```
x-ai/grok-4.5:low__json_schema PASS 100% 331 in → 258 out (246 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.5:max PASS 100% 265 in → 354 out (343 reasoning)
``` git clean -fdx -- src/ ```
x-ai/grok-4.5:max__json_schema PASS 100% 331 in → 674 out (662 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.5:medium PASS 100% 265 in → 372 out (361 reasoning)
```git git clean -fdx src/ ```
x-ai/grok-4.5:medium__json_schema PASS 100% 331 in → 739 out (727 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
x-ai/grok-4.5:xhigh PASS 100% 265 in → 327 out (320 reasoning)
git clean -fdx src/
x-ai/grok-4.5:xhigh__json_schema PASS 100% 331 in → 434 out (422 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdx src/"}
z-ai/glm-4.7-flash:low PASS 100% 60 in → 1,001 out (990 reasoning)
git clean -f -d -x -- src/
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 60 in → 1,024 out (1,009 reasoning)
git clean -dfx src/
JSON Schema Structured Output
(raw) {"command": "git clean -dfx src/"}
z-ai/glm-4.7-flash:none__json_schema PASS 100% 60 in → 13 out (0 reasoning)
git clean -dfx -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -dfx -- src/"}
z-ai/glm-4.7-flash:xhigh PASS 100% 60 in → 1,191 out (1,182 reasoning)
git clean -fdx src/
z-ai/glm-5.2:high PASS 100% 67 in → 378 out (345 reasoning)
git clean -xfd src/
z-ai/glm-5.2:low PASS 100% 67 in → 320 out (311 reasoning)
git clean -fxd src/
z-ai/glm-5.2:low__json_schema PASS 100% 67 in → 138 out (142 reasoning)
git clean -fdx src
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src" }
z-ai/glm-5.2:medium PASS 100% 67 in → 339 out (330 reasoning)
git clean -fxd src/
z-ai/glm-5.2:medium__json_schema PASS 100% 67 in → 260 out (245 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
z-ai/glm-5.2:none PASS 100% 61 in → 8 out (0 reasoning)
git clean -fdx src/
z-ai/glm-5.2:none__json_schema PASS 100% 61 in → 14 out (0 reasoning)
git clean -fdx src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdx src/" }
z-ai/glm-5.2:xhigh PASS 100% 67 in → 350 out (341 reasoning)
git clean -fdx src/
z-ai/glm-5.2:xhigh__json_schema PASS 100% 67 in → 640 out (575 reasoning)
git clean -dxf src/
JSON Schema Structured Output
(raw) { "command": "git clean -dxf src/" }
arcee-ai/trinity-mini:low__json_schema FAIL 83.3% 64 in → 696 out (740 reasoning)
git clean -f -X src/
JSON Schema Structured Output
(raw) { "command": "git clean -f -X src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-flash-0731:low__json_schema FAIL 83.3% 63 in → 15 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-flash-0731:none FAIL 83.3% 63 in → 8 out (0 reasoning)
git clean -fX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-flash:none FAIL 83.3% 63 in → 8 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-flash:none__json_schema FAIL 83.3% 63 in → 14 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-flash:xhigh__json_schema FAIL 83.3% 144 in → 820 out (804 reasoning)
git clean -fX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-pro:low__json_schema FAIL 83.3% 65 in → 632 out (613 reasoning)
cd src && git clean -fdX
JSON Schema Structured Output
(raw) { "command": "cd src && git clean -fdX" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-pro:none FAIL 83.3% 63 in → 7 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
deepseek/deepseek-v4-pro:none__json_schema FAIL 83.3% 63 in → 14 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
google/gemini-3.5-flash-lite:low__json_schema FAIL 83.3% 70 in → 12 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdX src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
google/gemma-4-31b-it:none FAIL 83.3% 82 in → 8 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
google/gemma-4-31b-it:none__json_schema FAIL 83.3% 82 in → 14 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
ibm-granite/granite-4.1-8b FAIL 83.3% 79 in → 8 out
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
minimax/minimax-m3:low__json_schema FAIL 83.3% 217 in → 14 out (0 reasoning)
git clean -fX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
minimax/minimax-m3:medium__json_schema FAIL 83.3% 217 in → 14 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
mistralai/mistral-medium-3-5:none FAIL 83.3% 74 in → 12 out (0 reasoning)
```sh git clean -fdX src/ ```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
nvidia/nemotron-3-nano-30b-a3b:none FAIL 83.3% 75 in → 8 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
nvidia/nemotron-3-nano-30b-a3b:none__json_schema FAIL 83.3% 75 in → 15 out (0 reasoning)
git clean -Xf src/
JSON Schema Structured Output
(raw) { "command": "git clean -Xf src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-5.4-nano:high FAIL 83.3% 60 in → 11 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-5.4-nano:low FAIL 83.3% 60 in → 10 out (0 reasoning)
git clean -fdX src
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-5.4-nano:low__json_schema FAIL 83.3% 95 in → 20 out (0 reasoning)
git clean -fdX -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdX -- src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-5.4-nano:medium__json_schema FAIL 83.3% 95 in → 18 out (0 reasoning)
git clean -fdX src
JSON Schema Structured Output
(raw) {"command":"git clean -fdX src"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-5.4-nano:none FAIL 83.3% 60 in → 11 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-5.4-nano:none__json_schema FAIL 83.3% 95 in → 20 out (0 reasoning)
git clean -fdX -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdX -- src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-oss-20b:high__json_schema FAIL 83.3% 121 in → 15 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-oss-20b:low FAIL 83.3% 108 in → 207 out (184 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
openai/gpt-oss-20b:medium__json_schema FAIL 83.3% 108 in → 708 out (720 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdX src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
poolside/laguna-m.1:high__json_schema FAIL 83.3% 72 in → 379 out (365 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) {"command": "git clean -fdX src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
poolside/laguna-m.1:none FAIL 83.3% 72 in → 9 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
poolside/laguna-m.1:none__json_schema FAIL 83.3% 72 in → 17 out (0 reasoning)
git clean -fdX src/
JSON Schema Structured Output
(raw) { "command": "git clean -fdX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
qwen/qwen3.6-27b:none FAIL 83.3% 70 in → 7 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
qwen/qwen3.6-27b:none__json_schema FAIL 83.3% 70 in → 17 out (0 reasoning)
git clean -dfX src/
JSON Schema Structured Output
(raw) { "command": "git clean -dfX src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
thinkingmachines/inkling:none FAIL 83.3% 67 in → 10 out (0 reasoning)
git clean -fdX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
x-ai/grok-4.3:none__json_schema FAIL 83.3% 293 in → 11 out (0 reasoning)
git clean -fdX -- src/
JSON Schema Structured Output
(raw) {"command":"git clean -fdX -- src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
z-ai/glm-4.7-flash:high FAIL 83.3% 60 in → 481 out (436 reasoning)
git clean -dfX -- src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
z-ai/glm-4.7-flash:medium FAIL 83.3% 60 in → 721 out (712 reasoning)
git clean -dfX src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/new.py'}]
anthropic/claude-haiku-4.5:none FAIL 66.7% 79 in → 14 out (0 reasoning)
```bash git clean -fd src/ ```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
anthropic/claude-haiku-4.5:none__json_schema FAIL 66.7% 241 in → 14 out (0 reasoning)
git clean -fd src/
JSON Schema Structured Output
(raw) {"command": "git clean -fd src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
arcee-ai/trinity-mini:medium__json_schema FAIL 66.7% 64 in → 1,774 out (2,042 reasoning)
git clean -f
JSON Schema Structured Output
(raw) { "command": "git clean -f" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
arcee-ai/trinity-mini:xhigh__json_schema FAIL 66.7% 64 in → 998 out (1,104 reasoning)
git clean -f -d src/
JSON Schema Structured Output
(raw) { "command": "git clean -f -d src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
google/gemini-3.5-flash-lite:low FAIL 66.7% 69 in → 8 out (0 reasoning)
`git clean -fd src/`
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
google/gemma-4-26b-a4b-it:none FAIL 66.7% 82 in → 7 out (0 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
google/gemma-4-26b-a4b-it:none__json_schema FAIL 66.7% 82 in → 16 out (0 reasoning)
git clean -fd src/
JSON Schema Structured Output
(raw) { "command": "git clean -fd src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
inclusionai/ling-2.6-flash FAIL 66.7% 80 in → 12 out
```sh git clean -fd src/ ```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
mistralai/devstral-2512 FAIL 66.7% 62 in → 11 out
```bash git clean -fd src/ ```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
mistralai/devstral-2512__json_schema FAIL 66.7% 62 in → 12 out
git clean -fd src/
JSON Schema Structured Output
(raw) {"command": "git clean -fd src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
nvidia/nemotron-3-super-120b-a12b:none FAIL 66.7% 75 in → 7 out (0 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
poolside/laguna-xs-2.1:none__json_schema FAIL 66.7% 72 in → 16 out (0 reasoning)
git clean -fd src/
JSON Schema Structured Output
(raw) { "command": "git clean -fd src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
poolside/laguna-xs.2:none FAIL 66.7% 109 in → 8 out (0 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
poolside/laguna-xs.2:none__json_schema FAIL 66.7% 109 in → 12 out (0 reasoning)
git clean -fd src/
JSON Schema Structured Output
(raw) {"command": "git clean -fd src/"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
qwen/qwen3.6-35b-a3b:none FAIL 66.7% 70 in → 7 out (0 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 66.7% 70 in → 16 out (0 reasoning)
git clean -fd src/
JSON Schema Structured Output
(raw) { "command": "git clean -fd src/" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
qwen/qwen3.7-flash:none FAIL 66.7% 70 in → 6 out (0 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
thinkingmachines/inkling:low FAIL 66.7% 69 in → 23 out (10 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
x-ai/grok-4.3:none FAIL 66.7% 237 in → 6 out (0 reasoning)
git clean -fd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}]
arcee-ai/trinity-mini:high FAIL 50% 64 in → 1,456 out (1,710 reasoning)
git clean-f src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
arcee-ai/trinity-mini:high__json_schema FAIL 50% 64 in → 859 out (941 reasoning)
gitclean -f -x
JSON Schema Structured Output
(raw) { "command": "gitclean -f -x" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
arcee-ai/trinity-mini:low FAIL 50% 64 in → 980 out (1,081 reasoning)
git clean-f -d -x src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
arcee-ai/trinity-mini:medium FAIL 50% 64 in → 1,538 out (1,693 reasoning)
git clean -f-d -x src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
arcee-ai/trinity-mini:xhigh FAIL 50% 64 in → 1,134 out (1,270 reasoning)
git clean -f -dsrc/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
git clean -fd src/ --exclude-standard
JSON Schema Structured Output
(raw) {"command":"git clean -fd src/ --exclude-standard"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
liquid/lfm-2-24b-a2b FAIL 50% 73 in → 22 out
git rm --ignore-unmatch -r src/new.py src/new.py src/new.py
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
poolside/laguna-m.1:low FAIL 50% 72 in → 196 out (187 reasoning)
git clean -xd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
poolside/laguna-m.1:xhigh FAIL 50% 72 in → 341 out (332 reasoning)
git clean -xd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
poolside/laguna-xs-2.1:xhigh FAIL 50% 72 in → 495 out (485 reasoning)
git clean -f -x -u src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
qwen/qwen3.6-35b-a3b:xhigh FAIL 50% 68 in → 2,006 out (1,854 reasoning)
git clean -xd src/
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_not_exists', 'path': 'src/new.py'}]
z-ai/glm-4.7-flash:none FAIL 50% 60 in → 15 out (0 reasoning)
git clean -fdx -e src/debug.log -e src/tmp/data
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'src/debug.log'}, {'type': 'dir_not_exists', 'path': 'src/tmp'}, {'type': 'file_exists', 'path': 'root.log'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -f -d -x src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git rm --ignore-unmatch -r src/new.py src/new.py src/new.py
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdx src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1cfeaa7f-8323-9b59-8283-a8c328b6a680","request_id":"1cfeaa7f-8323-9b59-8283-a8c328b6a680"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8cdc6c81-fc42-94a6-8c2c-cea3fbe760cf","request_id":"8cdc6c81-fc42-94a6-8c2c-cea3fbe760cf"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-fbdf55d8-db7c-916e-b162-2c08a870d214","request_id":"fbdf55d8-db7c-916e-b162-2c08a870d214"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-88d00282-72fa-9939-87d3-d11aeae71295","request_id":"88d00282-72fa-9939-87d3-d11aeae71295"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f7e1ace8-a3e9-975d-85db-b2a449603642","request_id":"f7e1ace8-a3e9-975d-85db-b2a449603642"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-27140cc7-980d-933e-b7b2-0337974c8b14"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-f6c9dfaa-e583-988e-b24d-5db69411170f"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-706536dd-b7ed-9e71-98ef-96e77a625c4b"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-2aad46e5-ea84-9b61-a5e8-0489e04851aa"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-8ca9f387-1e38-955b-a5e0-bcc1e13af763"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-3bc7cc7e-afba-96e4-b285-766c831b52b6"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-30300578-b28c-9798-8138-c2a7d9390cd9","request_id":"30300578-b28c-9798-8138-c2a7d9390cd9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9b0febed-e698-99ce-b85d-9b75edc25bcd","request_id":"9b0febed-e698-99ce-b85d-9b75edc25bcd"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4839031a-930b-9b01-92d9-949f57c2bbea","request_id":"4839031a-930b-9b01-92d9-949f57c2bbea"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-163fd8ae-6c3e-90d4-a567-b33fc0488025","request_id":"163fd8ae-6c3e-90d4-a567-b33fc0488025"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-11466a4a-5fd5-9133-938f-d58d6a37c073","request_id":"11466a4a-5fd5-9133-938f-d58d6a37c073"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d36a8c68-40ab-97fb-9a25-984c639dc38c","request_id":"d36a8c68-40ab-97fb-9a25-984c639dc38c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a84e8523-f1de-9890-a6cf-35ee33bd80b5","request_id":"a84e8523-f1de-9890-a6cf-35ee33bd80b5"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d20113c4-8cdc-9d5a-9a29-275380cd3530","request_id":"d20113c4-8cdc-9d5a-9a29-275380cd3530"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e32c79dc-bf51-913a-8adb-d87b123de995","request_id":"e32c79dc-bf51-913a-8adb-d87b123de995"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5fa7166a-897b-9f76-a91e-af0f11baa2e2","request_id":"5fa7166a-897b-9f76-a91e-af0f11baa2e2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git clean -fdx src/{ "command": "git clean -fdx src/" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdx src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdx src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fxd src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdx src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fdX src/{ "command": "git clean -fdX src/" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fxd src/
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)