Exclude specific files from cleaning
Tests ability to exclude specific files from cleaning with git clean -e. Evaluates exclusion-pattern usage.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'tracked' > tracked.txt
  5. 05 git add tracked.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 echo 'important' > important.tmp
  8. 08 echo 'junk1' > junk1.tmp
  9. 09 echo 'junk2' > junk2.txt
Prompt
Remove all untracked files except important.tmp. Output ONLY the git command, nothing else.
Expected
git clean -f -e important.tmp

Scoped model quality, cost, API time, and token usage for git_clean/f007.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5.1:high PASS 100% 108 in → 14 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5.1:high__json_schema PASS 100% 318 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5.1:low PASS 100% 108 in → 14 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5.1:low__json_schema PASS 100% 318 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5.1:max PASS 100% 108 in → 251 out (237 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5.1:max__json_schema PASS 100% 318 in → 247 out (226 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5.1:medium PASS 100% 108 in → 14 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5.1:medium__json_schema PASS 100% 318 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5.1:xhigh PASS 100% 108 in → 227 out (213 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5.1:xhigh__json_schema PASS 100% 318 in → 199 out (178 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5:high PASS 100% 106 in → 54 out (35 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5:high__json_schema PASS 100% 316 in → 45 out (35 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5:low PASS 100% 106 in → 43 out (39 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5:low__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5:max PASS 100% 106 in → 313 out (45 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-fable-5:max__json_schema PASS 100% 316 in → 310 out (91 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5:medium PASS 100% 106 in → 38 out (35 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5:medium__json_schema PASS 100% 316 in → 45 out (35 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-fable-5:xhigh PASS 100% 106 in → 117 out (49 reasoning)
git clean -f -e important.tmp
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 316 in → 45 out (30 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-haiku-4.5:high PASS 100% 114 in → 240 out (207 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 277 in → 217 out (186 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-haiku-4.5:low PASS 100% 114 in → 264 out (217 reasoning)
``` git clean -f junk1.tmp junk2.txt ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 277 in → 255 out (209 reasoning)
git clean -f -d -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -d -e important.tmp"}
anthropic/claude-haiku-4.5:medium PASS 100% 114 in → 309 out (270 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 277 in → 223 out (184 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
anthropic/claude-haiku-4.5:none PASS 100% 85 in → 18 out (0 reasoning)
```bash git clean -fd --exclude=important.tmp ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 247 in → 18 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f --exclude=important.tmp"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 114 in → 293 out (251 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 277 in → 384 out (335 reasoning)
git clean -f -d --exclude='important.tmp'
JSON Schema Structured Output
(raw) {"command": "git clean -f -d --exclude='important.tmp'"}
anthropic/claude-opus-4.6:high PASS 100% 85 in → 144 out (101 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 248 in → 146 out (102 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.6:low PASS 100% 85 in → 17 out (0 reasoning)
``` git clean -f --exclude=important.tmp ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 248 in → 17 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
anthropic/claude-opus-4.6:max PASS 100% 85 in → 127 out (87 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 248 in → 159 out (116 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.6:medium PASS 100% 85 in → 18 out (0 reasoning)
``` git clean -f --exclude=important.tmp ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 248 in → 49 out (20 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.6:none PASS 100% 85 in → 17 out (0 reasoning)
``` git clean -f --exclude=important.tmp ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 248 in → 17 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
anthropic/claude-opus-4.6:xhigh PASS 100% 85 in → 134 out (91 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 248 in → 134 out (91 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.7:high PASS 100% 111 in → 18 out (0 reasoning)
`git clean -f -e important.tmp`
anthropic/claude-opus-4.7:high__json_schema PASS 100% 321 in → 23 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.7:low PASS 100% 111 in → 16 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-4.7:low__json_schema PASS 100% 321 in → 23 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.7:max PASS 100% 111 in → 21 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 321 in → 23 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.7:medium PASS 100% 111 in → 16 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 321 in → 23 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.7:none PASS 100% 111 in → 18 out (0 reasoning)
`git clean -f -e important.tmp`
anthropic/claude-opus-4.7:none__json_schema PASS 100% 321 in → 23 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.7:xhigh PASS 100% 111 in → 18 out (0 reasoning)
`git clean -f -e important.tmp`
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 321 in → 23 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.8:high PASS 100% 106 in → 14 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-4.8:high__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.8:low PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.8:max PASS 100% 106 in → 285 out (46 reasoning)
`git clean -f -e important.tmp`
anthropic/claude-opus-4.8:max__json_schema PASS 100% 316 in → 210 out (40 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-opus-4.8:medium PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.8:none PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 316 in → 20 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-4.8:xhigh PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-5.5:high PASS 100% 108 in → 83 out (65 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-5.5:high__json_schema PASS 100% 318 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-5.5:low PASS 100% 108 in → 14 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-5.5:low__json_schema PASS 100% 318 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-5.5:max PASS 100% 108 in → 355 out (341 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-5.5:max__json_schema PASS 100% 318 in → 414 out (393 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-opus-5.5:medium PASS 100% 108 in → 80 out (66 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-5.5:medium__json_schema PASS 100% 318 in → 107 out (86 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-opus-5.5:xhigh PASS 100% 108 in → 157 out (143 reasoning)
git clean -f -e important.tmp
anthropic/claude-opus-5.5:xhigh__json_schema PASS 100% 318 in → 92 out (71 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-opus-5:high PASS 100% 106 in → 74 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-5:high__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-opus-5:low PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-5:low__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-5:max PASS 100% 106 in → 83 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-opus-5:max__json_schema PASS 100% 316 in → 93 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-opus-5:medium PASS 100% 106 in → 16 out (0 reasoning)
`git clean -f -e important.tmp`
anthropic/claude-opus-5:medium__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-opus-5:xhigh PASS 100% 106 in → 16 out (0 reasoning)
`git clean -f -e important.tmp`
anthropic/claude-opus-5:xhigh__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
anthropic/claude-sonnet-4.6:high PASS 100% 85 in → 97 out (71 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 248 in → 122 out (86 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-sonnet-4.6:low PASS 100% 85 in → 17 out (0 reasoning)
``` git clean -f --exclude=important.tmp ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 248 in → 17 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
anthropic/claude-sonnet-4.6:max PASS 100% 85 in → 136 out (102 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 248 in → 276 out (163 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-sonnet-4.6:medium PASS 100% 85 in → 17 out (0 reasoning)
``` git clean -f --exclude=important.tmp ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 248 in → 17 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
anthropic/claude-sonnet-4.6:none PASS 100% 85 in → 17 out (0 reasoning)
``` git clean -f --exclude=important.tmp ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 248 in → 17 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 85 in → 111 out (79 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 248 in → 109 out (77 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-sonnet-5:high PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
anthropic/claude-sonnet-5:low PASS 100% 106 in → 14 out (0 reasoning)
git clean -f -e important.tmp
anthropic/claude-sonnet-5:low__json_schema PASS 100% 316 in → 22 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
anthropic/claude-sonnet-5:medium PASS 100% 106 in → 19 out (0 reasoning)
``` git clean -fd -e important.tmp ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 316 in → 22 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
anthropic/claude-sonnet-5:none PASS 100% 106 in → 19 out (0 reasoning)
``` git clean -fd -e important.tmp ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
anthropic/claude-sonnet-5:xhigh PASS 100% 106 in → 18 out (0 reasoning)
``` git clean -f -e important.tmp ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 316 in → 22 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
arcee-ai/trinity-large-thinking:high PASS 100% 65 in → 622 out (612 reasoning)
git clean -f -e important.tmp
arcee-ai/trinity-large-thinking:low PASS 100% 65 in → 526 out (516 reasoning)
git clean -f -e important.tmp
arcee-ai/trinity-large-thinking:medium PASS 100% 65 in → 3,939 out (3,969 reasoning)
git clean -f -e important.tmp
arcee-ai/trinity-large-thinking:xhigh PASS 100% 65 in → 648 out (637 reasoning)
git clean -f -e important.tmp
arcee-ai/trinity-mini:high PASS 100% 65 in → 480 out (530 reasoning)
git clean -f -e important.tmp
arcee-ai/trinity-mini:low PASS 100% 65 in → 341 out (373 reasoning)
git clean -f -e important.tmp
arcee-ai/trinity-mini:low__json_schema PASS 100% 65 in → 364 out (400 reasoning)
git clean -f -eimportant.tmp
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 65 in → 391 out (428 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:high PASS 100% 148 in → 211 out (206 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 69 in → 376 out (366 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 69 in → 341 out (325 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 69 in → 22 out (0 reasoning)
git clean -f -e important.tmp -x
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp -x" }
deepseek/deepseek-v4-flash-0731:max PASS 100% 161 in → 4,235 out (4,198 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 347 in → 660 out (645 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
deepseek/deepseek-v4-flash-0731:medium PASS 100% 69 in → 352 out (341 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 69 in → 252 out (239 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 69 in → 10 out (0 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 69 in → 18 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 69 in → 389 out (380 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 69 in → 301 out (274 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-flash:high PASS 100% 69 in → 230 out (208 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 161 in → 14 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
deepseek/deepseek-v4-flash:low PASS 100% 69 in → 321 out (298 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 161 in → 14 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
deepseek/deepseek-v4-flash:medium PASS 100% 69 in → 300 out (288 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash:none PASS 100% 69 in → 14 out (0 reasoning)
```bash git clean -f -e important.tmp ```
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 69 in → 18 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 148 in → 919 out (908 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 150 in → 940 out (922 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-pro:high PASS 100% 69 in → 317 out (306 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 71 in → 290 out (271 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-pro:low PASS 100% 69 in → 181 out (170 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 256 in → 429 out (413 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
deepseek/deepseek-v4-pro:medium PASS 100% 69 in → 239 out (229 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 71 in → 349 out (330 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-pro:none PASS 100% 69 in → 9 out (0 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 69 in → 18 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 148 in → 651 out (641 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 334 in → 594 out (578 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
deepseek/deepseek-v4.1-flash:high PASS 100% 95 in → 832 out (821 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4.1-flash:high__json_schema PASS 100% 95 in → 423 out (408 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
deepseek/deepseek-v4.1-flash:low PASS 100% 95 in → 201 out (190 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4.1-flash:low__json_schema PASS 100% 95 in → 247 out (229 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4.1-flash:max PASS 100% 95 in → 315 out (304 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4.1-flash:max__json_schema PASS 100% 186 in → 126 out (111 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
deepseek/deepseek-v4.1-flash:medium PASS 100% 95 in → 148 out (146 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4.1-flash:medium__json_schema PASS 100% 95 in → 229 out (211 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4.1-flash:none PASS 100% 69 in → 10 out (0 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4.1-flash:none__json_schema PASS 100% 69 in → 17 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
deepseek/deepseek-v4.1-flash:xhigh PASS 100% 95 in → 188 out (177 reasoning)
git clean -f -e important.tmp
deepseek/deepseek-v4.1-flash:xhigh__json_schema PASS 100% 95 in → 326 out (308 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemini-3-flash-preview:high PASS 100% 69 in → 298 out (289 reasoning)
git clean -f -e important.tmp
google/gemini-3-flash-preview:high__json_schema PASS 100% 126 in → 247 out (229 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemini-3-flash-preview:low PASS 100% 69 in → 148 out (139 reasoning)
git clean -f -e important.tmp
google/gemini-3-flash-preview:low__json_schema PASS 100% 126 in → 281 out (268 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3-flash-preview:medium PASS 100% 69 in → 232 out (223 reasoning)
git clean -f -e important.tmp
google/gemini-3-flash-preview:medium__json_schema PASS 100% 126 in → 236 out (222 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3-flash-preview:xhigh PASS 100% 69 in → 325 out (316 reasoning)
git clean -f -e important.tmp
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 126 in → 243 out (230 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 69 in → 280 out (271 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 70 in → 308 out (294 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 70 in → 127 out (118 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 126 in → 122 out (108 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 69 in → 336 out (327 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 70 in → 217 out (203 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 69 in → 359 out (350 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 126 in → 500 out (486 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
google/gemini-3.1-pro-preview:high PASS 100% 69 in → 524 out (515 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 126 in → 386 out (373 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.1-pro-preview:low PASS 100% 69 in → 285 out (276 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 126 in → 215 out (201 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.1-pro-preview:medium PASS 100% 69 in → 225 out (216 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 126 in → 104 out (91 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 69 in → 409 out (400 reasoning)
git clean -f -e important.tmp
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 126 in → 316 out (303 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.5-flash-lite:high PASS 100% 69 in → 288 out (279 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 70 in → 423 out (409 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.5-flash-lite:low PASS 100% 69 in → 9 out (0 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 126 in → 19 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemini-3.5-flash-lite:max PASS 100% 70 in → 548 out (539 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 126 in → 425 out (406 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemini-3.5-flash-lite:medium PASS 100% 70 in → 369 out (360 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 70 in → 290 out (276 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 69 in → 338 out (329 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 126 in → 319 out (305 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
google/gemini-3.5-flash:high PASS 100% 69 in → 606 out (597 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash:high__json_schema PASS 100% 126 in → 270 out (256 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.5-flash:low PASS 100% 69 in → 259 out (250 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash:low__json_schema PASS 100% 126 in → 159 out (145 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.5-flash:medium PASS 100% 69 in → 382 out (373 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash:medium__json_schema PASS 100% 126 in → 569 out (556 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.5-flash:xhigh PASS 100% 69 in → 509 out (500 reasoning)
git clean -f -e important.tmp
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 126 in → 424 out (410 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.6-flash:high PASS 100% 69 in → 454 out (445 reasoning)
git clean -f -e important.tmp
google/gemini-3.6-flash:high__json_schema PASS 100% 70 in → 374 out (361 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.6-flash:low PASS 100% 70 in → 224 out (215 reasoning)
git clean -f -e important.tmp
google/gemini-3.6-flash:low__json_schema PASS 100% 70 in → 19 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemini-3.6-flash:max PASS 100% 69 in → 328 out (319 reasoning)
git clean -f -e important.tmp
google/gemini-3.6-flash:max__json_schema PASS 100% 70 in → 486 out (473 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.6-flash:medium PASS 100% 70 in → 290 out (281 reasoning)
git clean -f -e important.tmp
google/gemini-3.6-flash:medium__json_schema PASS 100% 70 in → 412 out (399 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.6-flash:xhigh PASS 100% 69 in → 604 out (595 reasoning)
git clean -f -e important.tmp
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 126 in → 306 out (293 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.8-flash:high PASS 100% 69 in → 337 out (328 reasoning)
git clean -f -e important.tmp
google/gemini-3.8-flash:high__json_schema PASS 100% 126 in → 321 out (308 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.8-flash:low PASS 100% 69 in → 9 out (0 reasoning)
git clean -f -e important.tmp
google/gemini-3.8-flash:low__json_schema PASS 100% 126 in → 13 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.8-flash:max PASS 100% 69 in → 275 out (266 reasoning)
git clean -f -e important.tmp
google/gemini-3.8-flash:max__json_schema PASS 100% 126 in → 469 out (456 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
google/gemini-3.8-flash:medium PASS 100% 69 in → 253 out (244 reasoning)
git clean -f -e important.tmp
google/gemini-3.8-flash:medium__json_schema PASS 100% 126 in → 183 out (169 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemini-3.8-flash:xhigh PASS 100% 69 in → 349 out (340 reasoning)
git clean -f -e important.tmp
google/gemini-3.8-flash:xhigh__json_schema PASS 100% 126 in → 348 out (334 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemma-4-26b-a4b-it:high PASS 100% 85 in → 285 out (234 reasoning)
git clean -f -e important.tmp
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 85 in → 290 out (224 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemma-4-26b-a4b-it:low PASS 100% 85 in → 429 out (370 reasoning)
git clean -f -e important.tmp
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 84 in → 267 out (205 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemma-4-26b-a4b-it:medium PASS 100% 84 in → 313 out (271 reasoning)
git clean -f -e important.tmp
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 84 in → 271 out (223 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemma-4-26b-a4b-it:none PASS 100% 82 in → 10 out (0 reasoning)
git clean -f -e important.tmp
google/gemma-4-26b-a4b-it:xhigh PASS 100% 85 in → 370 out (318 reasoning)
git clean -f -e important.tmp
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 85 in → 294 out (236 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
google/gemma-4-31b-it:high PASS 100% 85 in → 200 out (1 reasoning)
git clean -f -e important.tmp
google/gemma-4-31b-it:high__json_schema PASS 100% 85 in → 186 out (148 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemma-4-31b-it:low PASS 100% 85 in → 214 out (183 reasoning)
git clean -f -e important.tmp
google/gemma-4-31b-it:low__json_schema PASS 100% 85 in → 236 out (1 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemma-4-31b-it:medium PASS 100% 85 in → 234 out (188 reasoning)
git clean -f -e important.tmp
google/gemma-4-31b-it:medium__json_schema PASS 100% 84 in → 241 out (188 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
google/gemma-4-31b-it:none PASS 100% 82 in → 10 out (0 reasoning)
git clean -fd -e important.tmp
google/gemma-4-31b-it:none__json_schema PASS 100% 82 in → 20 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
google/gemma-4-31b-it:xhigh PASS 100% 84 in → 186 out (155 reasoning)
git clean -f -e important.tmp
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 88 in → 219 out (204 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
ibm-granite/granite-4.2-8b:high PASS 100% 87 in → 5,184 out (4,946 reasoning)
git clean -f junk1.tmp junk2.txt
ibm-granite/granite-4.2-8b:max PASS 100% 87 in → 2,418 out (2,405 reasoning)
git clean -fd --exclude='important.tmp'
ibm-granite/granite-4.2-8b:xhigh__json_schema PASS 100% 87 in → 20 out (0 reasoning)
rm junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) { "command": "rm junk1.tmp junk2.txt" }
inception/mercury-2.5:high PASS 100% 57 in → 1,612 out (1,601 reasoning)
git clean -f junk1.tmp junk2.txt
inception/mercury-2.5:high__json_schema PASS 100% 251 in → 1,285 out (1,235 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
inception/mercury-2.5:low PASS 100% 55 in → 250 out (235 reasoning)
git clean -f -e important.tmp
inception/mercury-2.5:low__json_schema PASS 100% 258 in → 467 out (255 reasoning)
git clean -f junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) { "command": "git clean -f junk1.tmp junk2.txt" }
inception/mercury-2.5:max PASS 100% 57 in → 1,107 out (1,097 reasoning)
git clean -fd -e important.tmp
inception/mercury-2.5:max__json_schema PASS 100% 255 in → 1,988 out (1,941 reasoning)
git clean -f junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command": "git clean -f junk1.tmp junk2.txt"}
inception/mercury-2.5:medium PASS 100% 55 in → 482 out (472 reasoning)
git clean -f -e important.tmp
inception/mercury-2.5:medium__json_schema PASS 100% 260 in → 2,075 out (1,986 reasoning)
git add important.tmp && git clean -f -x
JSON Schema Structured Output
(raw) { "command": "git add important.tmp && git clean -f -x" }
inception/mercury-2.5:none__json_schema PASS 100% 262 in → 57 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
inception/mercury-2.5:xhigh PASS 100% 57 in → 970 out (953 reasoning)
git clean -fd junk1.tmp junk2.txt
inception/mercury-2.5:xhigh__json_schema PASS 100% 254 in → 1,298 out (1,230 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f --exclude=important.tmp"}
inclusionai/ling-2.6-flash PASS 100% 83 in → 14 out
```sh git clean -f --exclude important.tmp ```
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
meta/muse-spark-1.1:high PASS 100% 63 in → 857 out (838 reasoning)
git clean -f --exclude=important.tmp
meta/muse-spark-1.1:high__json_schema PASS 100% 63 in → 709 out (683 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
meta/muse-spark-1.1:low PASS 100% 63 in → 373 out (354 reasoning)
git clean -f --exclude=important.tmp
meta/muse-spark-1.1:low__json_schema PASS 100% 63 in → 301 out (276 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
meta/muse-spark-1.1:max PASS 100% 63 in → 841 out (823 reasoning)
git clean -f -e important.tmp
meta/muse-spark-1.1:max__json_schema PASS 100% 63 in → 575 out (549 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
meta/muse-spark-1.1:medium PASS 100% 63 in → 397 out (379 reasoning)
git clean -f -e important.tmp
meta/muse-spark-1.1:medium__json_schema PASS 100% 63 in → 537 out (511 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
meta/muse-spark-1.1:xhigh PASS 100% 63 in → 846 out (827 reasoning)
git clean -f --exclude=important.tmp
meta/muse-spark-1.1:xhigh__json_schema PASS 100% 63 in → 633 out (607 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
meta/muse-spark-1.3-contributor:high PASS 100% 63 in → 717 out (681 reasoning)
git clean -f -e important.tmp
meta/muse-spark-1.3-contributor:high__json_schema PASS 100% 63 in → 1,016 out (952 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
meta/muse-spark-1.3-contributor:low PASS 100% 63 in → 345 out (326 reasoning)
git clean -f --exclude=important.tmp
meta/muse-spark-1.3-contributor:low__json_schema PASS 100% 63 in → 775 out (713 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
meta/muse-spark-1.3-contributor:medium PASS 100% 63 in → 661 out (643 reasoning)
git clean -f -e important.tmp
meta/muse-spark-1.3-contributor:medium__json_schema PASS 100% 63 in → 788 out (726 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
meta/muse-spark-1.3-contributor:xhigh PASS 100% 63 in → 996 out (940 reasoning)
git clean -f -e important.tmp
meta/muse-spark-1.3-contributor:xhigh__json_schema PASS 100% 63 in → 1,112 out (1,050 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
minimax/minimax-m2.5:high PASS 100% 100 in → 201 out (192 reasoning)
git clean -fe important.tmp
minimax/minimax-m2.5:high__json_schema PASS 100% 100 in → 1,944 out (1,971 reasoning)
git clean -fd -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) { "command": "git clean -fd -f -- junk1.tmp junk2.txt" }
minimax/minimax-m2.5:low PASS 100% 100 in → 297 out (275 reasoning)
git clean -f junk1.tmp junk2.txt
minimax/minimax-m2.5:low__json_schema PASS 100% 100 in → 905 out (989 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
minimax/minimax-m2.5:medium PASS 100% 100 in → 203 out (212 reasoning)
git clean -f -e important.tmp
minimax/minimax-m2.5:xhigh PASS 100% 110 in → 163 out (152 reasoning)
git clean -f -e important.tmp
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 100 in → 365 out (338 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
minimax/minimax-m2.7:high PASS 100% 103 in → 697 out (687 reasoning)
git clean -fd -e important.tmp
minimax/minimax-m2.7:low PASS 100% 100 in → 812 out (801 reasoning)
git clean -fd -e important.tmp
minimax/minimax-m2.7:medium PASS 100% 100 in → 617 out (595 reasoning)
git clean -f -e important.tmp
minimax/minimax-m2.7:medium__json_schema PASS 100% 232 in → 412 out (393 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
minimax/minimax-m2.7:xhigh PASS 100% 100 in → 564 out (552 reasoning)
git clean -f -e important.tmp
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 232 in → 4,827 out (4,811 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
minimax/minimax-m3:high PASS 100% 238 in → 159 out (151 reasoning)
git clean -f -e important.tmp
minimax/minimax-m3:low PASS 100% 238 in → 236 out (239 reasoning)
git clean -f -e important.tmp
minimax/minimax-m3:low__json_schema PASS 100% 225 in → 17 out (0 reasoning)
git clean -f -- ':!important.tmp'
JSON Schema Structured Output
(raw) {"command": "git clean -f -- ':!important.tmp'"}
minimax/minimax-m3:medium PASS 100% 238 in → 82 out (78 reasoning)
git clean -f -e important.tmp
minimax/minimax-m3:xhigh PASS 100% 238 in → 103 out (106 reasoning)
git clean -f -e important.tmp
mistralai/devstral-2512 PASS 100% 68 in → 15 out
```bash git clean -f -e important.tmp ```
mistralai/devstral-2512__json_schema PASS 100% 68 in → 15 out
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
mistralai/mistral-medium-3-5:high PASS 100% 80 in → 909 out (960 reasoning)
git clean -f --exclude=important.tmp
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 68 in → 1,161 out (1,217 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
mistralai/mistral-medium-3-5:low PASS 100% 80 in → 4,474 out (4,822 reasoning)
git clean -fd -e important.tmp
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 68 in → 328 out (315 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
mistralai/mistral-medium-3-5:medium PASS 100% 80 in → 2,665 out (2,911 reasoning)
git clean -f -e important.tmp
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 68 in → 521 out (535 reasoning)
git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd --exclude=important.tmp"}
mistralai/mistral-medium-3-5:none PASS 100% 80 in → 13 out (0 reasoning)
git clean -f -x --exclude=important.tmp
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 68 in → 16 out (0 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f --exclude=important.tmp"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 80 in → 775 out (796 reasoning)
git clean -f -e important.tmp
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 68 in → 651 out (673 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f --exclude=important.tmp"}
moonshotai/kimi-k2.6:high PASS 100% 65 in → 1,294 out (1,284 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.6:high__json_schema PASS 100% 247 in → 877 out (863 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
moonshotai/kimi-k2.6:low PASS 100% 65 in → 1,244 out (1,234 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.6:low__json_schema PASS 100% 64 in → 1,044 out (931 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
moonshotai/kimi-k2.6:medium PASS 100% 65 in → 1,409 out (1,400 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 64 in → 944 out (669 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
moonshotai/kimi-k2.6:xhigh PASS 100% 64 in → 937 out (927 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 65 in → 478 out (465 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
moonshotai/kimi-k2.7-code:high PASS 100% 64 in → 219 out (209 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 65 in → 120 out (114 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
moonshotai/kimi-k2.7-code:low PASS 100% 64 in → 132 out (121 reasoning)
git clean -fd --exclude=important.tmp
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 148 in → 93 out (78 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
moonshotai/kimi-k2.7-code:medium PASS 100% 65 in → 474 out (499 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 64 in → 382 out (106 reasoning)
git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd --exclude=important.tmp" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 64 in → 159 out (149 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 148 in → 142 out (127 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
moonshotai/kimi-k3:high PASS 100% 142 in → 357 out (334 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:high__json_schema PASS 100% 142 in → 332 out (306 reasoning)
``` git clean -fd -e important.tmp ```
moonshotai/kimi-k3:low PASS 100% 142 in → 201 out (178 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:low__json_schema PASS 100% 142 in → 143 out (120 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:max PASS 100% 142 in → 141 out (118 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:medium PASS 100% 142 in → 109 out (86 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:medium__json_schema PASS 100% 142 in → 333 out (310 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:xhigh PASS 100% 142 in → 251 out (228 reasoning)
git clean -f -e important.tmp
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 142 in → 554 out (531 reasoning)
git clean -f -e important.tmp
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 81 in → 714 out (768 reasoning)
git clean -fd --exclude=important.tmp
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 81 in → 252 out (240 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 81 in → 307 out (306 reasoning)
git clean -fdx --exclude=important.tmp .
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 81 in → 350 out (371 reasoning)
git clean -f --exclude=important.tmp
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 81 in → 414 out (402 reasoning)
git clean -fd --exclude=important.tmp .
JSON Schema Structured Output
(raw) { "command": "git clean -fd --exclude=important.tmp ." }
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 81 in → 19 out (0 reasoning)
git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd --exclude=important.tmp" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 81 in → 3,510 out (3,848 reasoning)
git clean -f -e important.tmp
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 81 in → 403 out (393 reasoning)
git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd --exclude=important.tmp" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 81 in → 206 out (189 reasoning)
git clean -f -e important.tmp
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 81 in → 247 out (213 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 81 in → 221 out (191 reasoning)
git clean -f -e important.tmp
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 81 in → 229 out (201 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 81 in → 269 out (239 reasoning)
git clean -f -e important.tmp
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 81 in → 404 out (360 reasoning)
git clean -f -e "important.tmp"
JSON Schema Structured Output
(raw) {"command": "git clean -f -e \"important.tmp\""}
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 81 in → 12 out (0 reasoning)
git clean -f -x -e important.tmp
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 81 in → 25 out (0 reasoning)
git clean -fX && git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fX && git clean -fd --exclude=important.tmp" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 81 in → 267 out (254 reasoning)
git clean -f -e important.tmp
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 81 in → 174 out (144 reasoning)
git clean -f -e 'important.tmp'
JSON Schema Structured Output
(raw) { "command": "git clean -f -e 'important.tmp'" }
openai/gpt-5.4-mini:high PASS 100% 61 in → 156 out (142 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4-mini:high__json_schema PASS 100% 96 in → 154 out (132 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.4-mini:low PASS 100% 61 in → 45 out (31 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4-mini:low__json_schema PASS 100% 96 in → 86 out (64 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.4-mini:medium PASS 100% 61 in → 101 out (87 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4-mini:medium__json_schema PASS 100% 96 in → 150 out (128 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-5.4-mini:none PASS 100% 61 in → 12 out (0 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4-mini:none__json_schema PASS 100% 96 in → 20 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.4-mini:xhigh PASS 100% 61 in → 202 out (188 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 96 in → 538 out (516 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.4-nano:high PASS 100% 61 in → 130 out (115 reasoning)
git clean -fd --exclude=important.tmp
openai/gpt-5.4-nano:high__json_schema PASS 100% 96 in → 20 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-5.4-nano:low__json_schema PASS 100% 96 in → 20 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-5.4-nano:medium PASS 100% 61 in → 95 out (81 reasoning)
git clean -fd -e important.tmp
openai/gpt-5.4-nano:medium__json_schema PASS 100% 96 in → 105 out (83 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-5.4-nano:none PASS 100% 61 in → 13 out (0 reasoning)
git clean -fd --exclude=important.tmp
openai/gpt-5.4-nano:none__json_schema PASS 100% 96 in → 20 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-5.4-nano:xhigh PASS 100% 61 in → 250 out (236 reasoning)
git clean -fd -e important.tmp
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 96 in → 185 out (163 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-5.4:high PASS 100% 61 in → 215 out (201 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4:high__json_schema PASS 100% 96 in → 123 out (101 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.4:low PASS 100% 61 in → 77 out (63 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4:low__json_schema PASS 100% 96 in → 87 out (62 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-5.4:medium PASS 100% 61 in → 126 out (112 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4:medium__json_schema PASS 100% 96 in → 106 out (84 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.4:none PASS 100% 61 in → 17 out (0 reasoning)
git clean -f -- . ':(exclude)important.tmp'
openai/gpt-5.4:none__json_schema PASS 100% 96 in → 25 out (0 reasoning)
git clean -f -- . ':(exclude)important.tmp'
JSON Schema Structured Output
(raw) {"command":"git clean -f -- . ':(exclude)important.tmp'"}
openai/gpt-5.4:xhigh PASS 100% 61 in → 446 out (432 reasoning)
git clean -f -e important.tmp
openai/gpt-5.4:xhigh__json_schema PASS 100% 96 in → 538 out (516 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.5:high PASS 100% 61 in → 158 out (144 reasoning)
git clean -f -e important.tmp
openai/gpt-5.5:high__json_schema PASS 100% 96 in → 89 out (67 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.5:low PASS 100% 61 in → 60 out (46 reasoning)
git clean -f -e important.tmp
openai/gpt-5.5:low__json_schema PASS 100% 96 in → 96 out (74 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.5:medium PASS 100% 61 in → 94 out (80 reasoning)
git clean -f -e important.tmp
openai/gpt-5.5:medium__json_schema PASS 100% 96 in → 139 out (117 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.5:none PASS 100% 61 in → 14 out (0 reasoning)
git clean -f -- ':!important.tmp'
openai/gpt-5.5:none__json_schema PASS 100% 96 in → 22 out (0 reasoning)
git clean -f -- ':!important.tmp'
JSON Schema Structured Output
(raw) {"command":"git clean -f -- ':!important.tmp'"}
openai/gpt-5.5:xhigh PASS 100% 61 in → 530 out (516 reasoning)
git clean -f -e important.tmp
openai/gpt-5.5:xhigh__json_schema PASS 100% 96 in → 381 out (357 reasoning)
git clean -f -- ':!important.tmp'
JSON Schema Structured Output
(raw) {"command":"git clean -f -- ':!important.tmp'"}
openai/gpt-5.6-luna:high PASS 100% 61 in → 99 out (85 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-luna:high__json_schema PASS 100% 96 in → 60 out (35 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-5.6-luna:low PASS 100% 61 in → 82 out (68 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-luna:low__json_schema PASS 100% 96 in → 51 out (28 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
openai/gpt-5.6-luna:max PASS 100% 61 in → 127 out (113 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-luna:max__json_schema PASS 100% 96 in → 538 out (516 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-luna:medium PASS 100% 61 in → 75 out (61 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-luna:medium__json_schema PASS 100% 96 in → 68 out (46 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-luna:none PASS 100% 61 in → 13 out (0 reasoning)
git clean -f --exclude=important.tmp
openai/gpt-5.6-luna:none__json_schema PASS 100% 96 in → 23 out (0 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-5.6-luna:xhigh PASS 100% 61 in → 182 out (168 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 96 in → 262 out (240 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-sol:high PASS 100% 61 in → 67 out (53 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-sol:high__json_schema PASS 100% 96 in → 102 out (80 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-sol:low PASS 100% 61 in → 50 out (36 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-sol:low__json_schema PASS 100% 96 in → 119 out (97 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-sol:max PASS 100% 61 in → 145 out (131 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-sol:max__json_schema PASS 100% 96 in → 214 out (192 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-sol:medium PASS 100% 61 in → 60 out (46 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-sol:medium__json_schema PASS 100% 96 in → 92 out (70 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-sol:none PASS 100% 61 in → 12 out (0 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-sol:none__json_schema PASS 100% 96 in → 20 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-sol:xhigh PASS 100% 61 in → 68 out (54 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 96 in → 136 out (114 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-terra:high PASS 100% 61 in → 62 out (48 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-terra:high__json_schema PASS 100% 96 in → 86 out (61 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-5.6-terra:low PASS 100% 61 in → 41 out (27 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-terra:low__json_schema PASS 100% 96 in → 23 out (0 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-5.6-terra:max PASS 100% 61 in → 123 out (108 reasoning)
git clean -f --exclude=important.tmp
openai/gpt-5.6-terra:max__json_schema PASS 100% 96 in → 126 out (104 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-5.6-terra:medium PASS 100% 61 in → 42 out (28 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-terra:none PASS 100% 61 in → 12 out (0 reasoning)
git clean -f -e important.tmp
openai/gpt-5.6-terra:none__json_schema PASS 100% 96 in → 23 out (0 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-5.6-terra:xhigh PASS 100% 61 in → 78 out (61 reasoning)
git clean -f -- junk1.tmp junk2.txt
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 96 in → 83 out (60 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
openai/gpt-6-astra:high PASS 100% 61 in → 87 out (73 reasoning)
git clean -f -e important.tmp
openai/gpt-6-astra:high__json_schema PASS 100% 96 in → 160 out (138 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-6-astra:low PASS 100% 61 in → 12 out (0 reasoning)
git clean -f -e important.tmp
openai/gpt-6-astra:low__json_schema PASS 100% 96 in → 20 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-6-astra:max PASS 100% 61 in → 463 out (449 reasoning)
git clean -f -e important.tmp
openai/gpt-6-astra:max__json_schema PASS 100% 96 in → 239 out (217 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-6-astra:medium PASS 100% 61 in → 58 out (44 reasoning)
git clean -fd -e important.tmp
openai/gpt-6-astra:medium__json_schema PASS 100% 96 in → 69 out (47 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-6-astra:xhigh PASS 100% 61 in → 132 out (118 reasoning)
git clean -fd -e important.tmp
openai/gpt-6-astra:xhigh__json_schema PASS 100% 96 in → 158 out (136 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-6-luna:high PASS 100% 61 in → 111 out (97 reasoning)
git clean -f -e important.tmp
openai/gpt-6-luna:high__json_schema PASS 100% 96 in → 147 out (125 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-6-luna:low PASS 100% 61 in → 68 out (54 reasoning)
git clean -f -e important.tmp
openai/gpt-6-luna:low__json_schema PASS 100% 96 in → 73 out (48 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-6-luna:max PASS 100% 61 in → 190 out (176 reasoning)
git clean -f -e important.tmp
openai/gpt-6-luna:max__json_schema PASS 100% 96 in → 364 out (339 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-6-luna:medium PASS 100% 61 in → 88 out (74 reasoning)
git clean -f -e important.tmp
openai/gpt-6-luna:medium__json_schema PASS 100% 96 in → 172 out (147 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-6-luna:xhigh PASS 100% 61 in → 118 out (104 reasoning)
git clean -f -e important.tmp
openai/gpt-6-luna:xhigh__json_schema PASS 100% 96 in → 113 out (88 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
openai/gpt-6-sol:high PASS 100% 61 in → 145 out (131 reasoning)
git clean -fd -e important.tmp
openai/gpt-6-sol:high__json_schema PASS 100% 96 in → 242 out (220 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-6-sol:low PASS 100% 61 in → 60 out (46 reasoning)
git clean -f -e important.tmp
openai/gpt-6-sol:low__json_schema PASS 100% 96 in → 48 out (26 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-6-sol:max PASS 100% 61 in → 461 out (447 reasoning)
git clean -fd -e important.tmp
openai/gpt-6-sol:max__json_schema PASS 100% 96 in → 975 out (953 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-6-sol:medium PASS 100% 61 in → 146 out (132 reasoning)
git clean -fd -e important.tmp
openai/gpt-6-sol:medium__json_schema PASS 100% 96 in → 128 out (106 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-6-sol:xhigh PASS 100% 61 in → 122 out (108 reasoning)
git clean -fd -e important.tmp
openai/gpt-6-sol:xhigh__json_schema PASS 100% 96 in → 182 out (160 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-oss-120b:high PASS 100% 109 in → 483 out (518 reasoning)
git clean -f -e important.tmp
openai/gpt-oss-120b:low PASS 100% 124 in → 218 out (215 reasoning)
git clean -fd -e important.tmp
openai/gpt-oss-120b:low__json_schema PASS 100% 122 in → 122 out (100 reasoning)
git clean -f -e important.tmp .
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp ."}
openai/gpt-oss-120b:medium PASS 100% 124 in → 329 out (334 reasoning)
git clean -f -e important.tmp .
openai/gpt-oss-120b:medium__json_schema PASS 100% 109 in → 232 out (231 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-oss-120b:xhigh PASS 100% 120 in → 676 out (710 reasoning)
git clean -f -e important.tmp
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 109 in → 792 out (886 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
openai/gpt-oss-20b:high PASS 100% 120 in → 681 out (750 reasoning)
git clean -f -e important.tmp
openai/gpt-oss-20b:low PASS 100% 120 in → 49 out (32 reasoning)
git clean -f -e important.tmp
openai/gpt-oss-20b:low__json_schema PASS 100% 120 in → 61 out (45 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
openai/gpt-oss-20b:medium PASS 100% 122 in → 305 out (297 reasoning)
git clean -fd -e important.tmp
openai/gpt-oss-20b:medium__json_schema PASS 100% 120 in → 404 out (401 reasoning)
git clean -f -e important.tmp -d
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp -d"}
openai/gpt-oss-20b:xhigh PASS 100% 122 in → 219 out (211 reasoning)
git clean -fd -e important.tmp
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 120 in → 570 out (600 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
poolside/laguna-m.1:high PASS 100% 74 in → 1,446 out (1,436 reasoning)
git clean -f -e important.tmp
poolside/laguna-m.1:high__json_schema PASS 100% 74 in → 1,039 out (1,019 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
poolside/laguna-m.1:low PASS 100% 74 in → 1,361 out (1,350 reasoning)
git clean -f -e important.tmp
poolside/laguna-m.1:low__json_schema PASS 100% 74 in → 586 out (567 reasoning)
git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd --exclude=important.tmp" }
poolside/laguna-m.1:medium PASS 100% 74 in → 263 out (252 reasoning)
git clean -fd -e important.tmp
poolside/laguna-m.1:medium__json_schema PASS 100% 74 in → 444 out (429 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
poolside/laguna-m.1:xhigh PASS 100% 74 in → 383 out (370 reasoning)
git clean -fd -e 'important.tmp'
poolside/laguna-m.1:xhigh__json_schema PASS 100% 74 in → 2,752 out (2,736 reasoning)
git clean -fd --exclude important.tmp -f
JSON Schema Structured Output
(raw) {"command": "git clean -fd --exclude important.tmp -f"}
poolside/laguna-xs-2.1:high PASS 100% 74 in → 514 out (503 reasoning)
git clean -fd --exclude important.tmp --force
poolside/laguna-xs-2.1:high__json_schema PASS 100% 74 in → 235 out (220 reasoning)
git clean -df -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -df -e important.tmp"}
poolside/laguna-xs-2.1:low PASS 100% 74 in → 405 out (395 reasoning)
git clean -f --exclude=important.tmp
poolside/laguna-xs-2.1:low__json_schema PASS 100% 74 in → 314 out (299 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 74 in → 2,215 out (2,200 reasoning)
git clean -f -x -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -x -e important.tmp"}
poolside/laguna-xs-2.1:none PASS 100% 74 in → 15 out (0 reasoning)
git clean -fdX && git clean -fd --exclude=important.tmp
poolside/laguna-xs-2.1:xhigh PASS 100% 74 in → 1,353 out (1,341 reasoning)
git clean -f -- junk1.tmp junk2.txt
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 74 in → 673 out (657 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f --exclude=important.tmp"}
poolside/laguna-xs.2:high PASS 100% 111 in → 597 out (584 reasoning)
git clean -fd --exclude='important.tmp'
poolside/laguna-xs.2:high__json_schema PASS 100% 111 in → 190 out (175 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
poolside/laguna-xs.2:low PASS 100% 111 in → 630 out (619 reasoning)
git clean -fd -e important.tmp
poolside/laguna-xs.2:low__json_schema PASS 100% 111 in → 697 out (679 reasoning)
git clean -fd ':(exclude)important.tmp'
JSON Schema Structured Output
(raw) {"command": "git clean -fd ':(exclude)important.tmp'"}
poolside/laguna-xs.2:medium PASS 100% 111 in → 150 out (139 reasoning)
git clean -f -e important.tmp
poolside/laguna-xs.2:medium__json_schema PASS 100% 111 in → 686 out (671 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
poolside/laguna-xs.2:xhigh PASS 100% 111 in → 344 out (333 reasoning)
git clean -f -e important.tmp
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 111 in → 270 out (250 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
prism-ml/ternary-bonsai-2-27b:high PASS 100% 70 in → 742 out (729 reasoning)
git clean -f junk1.tmp junk2.txt
prism-ml/ternary-bonsai-2-27b:high__json_schema PASS 100% 164 in → 509 out (488 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
prism-ml/ternary-bonsai-2-27b:low PASS 100% 70 in → 108 out (91 reasoning)
``` git clean -f junk1.tmp junk2.txt ```
prism-ml/ternary-bonsai-2-27b:low__json_schema PASS 100% 164 in → 208 out (193 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
prism-ml/ternary-bonsai-2-27b:max PASS 100% 112 in → 461 out (450 reasoning)
git clean -fd -e important.tmp
prism-ml/ternary-bonsai-2-27b:max__json_schema PASS 100% 202 in → 219 out (198 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
prism-ml/ternary-bonsai-2-27b:medium PASS 100% 70 in → 299 out (286 reasoning)
git clean -f junk1.tmp junk2.txt
prism-ml/ternary-bonsai-2-27b:medium__json_schema PASS 100% 164 in → 491 out (476 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
prism-ml/ternary-bonsai-2-27b:none PASS 100% 72 in → 12 out (0 reasoning)
`git clean -f -e "important.tmp"`
prism-ml/ternary-bonsai-2-27b:none__json_schema PASS 100% 166 in → 18 out (0 reasoning)
git clean -fd -x -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -fd -x -- junk1.tmp junk2.txt"}
prism-ml/ternary-bonsai-2-27b:xhigh PASS 100% 112 in → 295 out (283 reasoning)
git clean -f --exclude=important.tmp
prism-ml/ternary-bonsai-2-27b:xhigh__json_schema PASS 100% 202 in → 424 out (407 reasoning)
git clean -f --exclude=important.tmp .
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp ."}
qwen/qwen3.6-27b:high PASS 100% 70 in → 1,879 out (1,742 reasoning)
git clean -fd --exclude=important.tmp
qwen/qwen3.6-27b:high__json_schema PASS 100% 70 in → 2,242 out (2,141 reasoning)
git clean -f --exclude important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude important.tmp" }
qwen/qwen3.6-27b:low PASS 100% 70 in → 1,660 out (1,582 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-27b:low__json_schema PASS 100% 70 in → 1,287 out (1,220 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
qwen/qwen3.6-27b:medium PASS 100% 70 in → 1,454 out (1,356 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-27b:none PASS 100% 72 in → 9 out (0 reasoning)
git clean -xf -e important.tmp
qwen/qwen3.6-27b:none__json_schema PASS 100% 72 in → 25 out (0 reasoning)
git clean -fdX && git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fdX && git clean -fd -e important.tmp" }
qwen/qwen3.6-27b:xhigh PASS 100% 70 in → 1,454 out (1 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 70 in → 2,019 out (1,858 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
qwen/qwen3.6-35b-a3b:high PASS 100% 70 in → 1,376 out (1,282 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 70 in → 2,189 out (2,059 reasoning)
git clean -fd --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd --exclude=important.tmp" }
qwen/qwen3.6-35b-a3b:low PASS 100% 70 in → 1,826 out (1,813 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 70 in → 1,961 out (1,813 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 70 in → 1,930 out (1,793 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 70 in → 1,729 out (1,622 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
qwen/qwen3.6-35b-a3b:none PASS 100% 72 in → 10 out (0 reasoning)
git clean -fdx -e important.tmp
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 70 in → 741 out (709 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 70 in → 1,647 out (1,492 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
qwen/qwen3.6-flash:high PASS 100% 70 in → 1,427 out (1,413 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.6-flash:low PASS 100% 70 in → 1,637 out (1,624 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-flash:medium PASS 100% 70 in → 1,377 out (1,364 reasoning)
git clean -f -e important.tmp
qwen/qwen3.6-flash:none PASS 100% 72 in → 9 out (0 reasoning)
git clean -fd --exclude=important.tmp
qwen/qwen3.6-flash:xhigh PASS 100% 70 in → 1,183 out (1,169 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.7-flash:high PASS 100% 70 in → 1,582 out (1,568 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.7-flash:low PASS 100% 70 in → 1,299 out (1,285 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.7-flash:max PASS 100% 70 in → 1,262 out (1,248 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.7-flash:medium PASS 100% 70 in → 816 out (803 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-flash:none PASS 100% 72 in → 9 out (0 reasoning)
git clean -fd --exclude=important.tmp
qwen/qwen3.7-flash:xhigh PASS 100% 70 in → 593 out (579 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.7-max:high PASS 100% 70 in → 1,056 out (1,043 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-max:low PASS 100% 70 in → 385 out (372 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-max:medium PASS 100% 70 in → 879 out (866 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-max:none PASS 100% 72 in → 9 out (0 reasoning)
git clean -f --exclude=important.tmp
qwen/qwen3.7-max:xhigh PASS 100% 70 in → 807 out (794 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-plus:high PASS 100% 70 in → 708 out (695 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-plus:low PASS 100% 70 in → 708 out (695 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-plus:medium PASS 100% 70 in → 741 out (728 reasoning)
git clean -f -e important.tmp
qwen/qwen3.7-plus:none PASS 100% 72 in → 8 out (0 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.7-plus:xhigh PASS 100% 70 in → 431 out (418 reasoning)
git clean -f -e important.tmp
qwen/qwen3.8-flash:high PASS 100% 121 in → 269 out (258 reasoning)
git clean -f -e important.tmp
qwen/qwen3.8-flash:high__json_schema PASS 100% 123 in → 377 out (361 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
qwen/qwen3.8-flash:low PASS 100% 121 in → 294 out (283 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.8-flash:max PASS 100% 121 in → 493 out (482 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.8-flash:max__json_schema PASS 100% 123 in → 330 out (309 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
qwen/qwen3.8-flash:medium PASS 100% 121 in → 219 out (208 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.8-flash:xhigh PASS 100% 121 in → 519 out (508 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.8-flash:xhigh__json_schema PASS 100% 123 in → 467 out (451 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
qwen/qwen3.8-max:high PASS 100% 108 in → 258 out (245 reasoning)
git clean -f -e important.tmp
qwen/qwen3.8-max:high__json_schema PASS 100% 110 in → 286 out (264 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
qwen/qwen3.8-max:low PASS 100% 96 in → 86 out (73 reasoning)
git clean -f -e important.tmp
qwen/qwen3.8-max:low__json_schema PASS 100% 98 in → 90 out (71 reasoning)
git clean -fd --exclude="important.tmp"
JSON Schema Structured Output
(raw) {"command": "git clean -fd --exclude=\"important.tmp\""}
qwen/qwen3.8-max:max PASS 100% 108 in → 419 out (406 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.8-max:max__json_schema PASS 100% 110 in → 432 out (409 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f --exclude=important.tmp" }
qwen/qwen3.8-max:medium PASS 100% 70 in → 128 out (115 reasoning)
git clean -f -e important.tmp
qwen/qwen3.8-max:medium__json_schema PASS 100% 72 in → 147 out (125 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
qwen/qwen3.8-max:xhigh PASS 100% 108 in → 295 out (282 reasoning)
git clean -fd -e important.tmp
qwen/qwen3.8-max:xhigh__json_schema PASS 100% 110 in → 443 out (421 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
sakana/fugu-max:high PASS 100% 138 in → 57 out (43 reasoning)
git clean -f -e important.tmp
sakana/fugu-max:high__json_schema PASS 100% 173 in → 132 out (109 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
sakana/fugu-max:low PASS 100% 138 in → 59 out (42 reasoning)
git clean -f -- junk1.tmp junk2.txt
sakana/fugu-max:low__json_schema PASS 100% 173 in → 73 out (50 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f --exclude=important.tmp"}
sakana/fugu-max:max PASS 100% 138 in → 224 out (209 reasoning)
git clean -f --exclude=important.tmp
sakana/fugu-max:max__json_schema PASS 100% 173 in → 493 out (471 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
sakana/fugu-max:medium PASS 100% 138 in → 72 out (58 reasoning)
git clean -f -e important.tmp
sakana/fugu-max:medium__json_schema PASS 100% 173 in → 101 out (76 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
sakana/fugu-max:xhigh PASS 100% 138 in → 129 out (115 reasoning)
git clean -f -e important.tmp
sakana/fugu-max:xhigh__json_schema PASS 100% 173 in → 476 out (451 reasoning)
git clean -f -- junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git clean -f -- junk1.tmp junk2.txt"}
stealth/ox-alpha:high PASS 100% 145 in → 36 out (0 reasoning)
git clean -fd -e important.tmp
stealth/ox-alpha:low PASS 100% 145 in → 10 out (0 reasoning)
git clean -e important.tmp -f
stealth/ox-alpha:max PASS 100% 145 in → 238 out (0 reasoning)
```bash git clean -f -e important.tmp ```
tencent/hy3:high PASS 100% 75 in → 516 out (505 reasoning)
git clean -f -e important.tmp
tencent/hy3:high__json_schema PASS 100% 75 in → 927 out (836 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
tencent/hy3:low PASS 100% 75 in → 542 out (531 reasoning)
git clean -f -e important.tmp
tencent/hy3:low__json_schema PASS 100% 75 in → 557 out (507 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
tencent/hy3:medium PASS 100% 75 in → 433 out (422 reasoning)
git clean -f -e important.tmp
tencent/hy3:medium__json_schema PASS 100% 75 in → 359 out (321 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
tencent/hy3:none PASS 100% 78 in → 10 out (0 reasoning)
git clean -f -e important.tmp
tencent/hy3:none__json_schema PASS 100% 78 in → 20 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
tencent/hy3:xhigh PASS 100% 75 in → 554 out (543 reasoning)
git clean -f -e important.tmp
tencent/hy3:xhigh__json_schema PASS 100% 75 in → 1,293 out (1,200 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
thinkingmachines/inkling-small:high PASS 100% 70 in → 175 out (161 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling-small:high__json_schema PASS 100% 70 in → 183 out (163 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
thinkingmachines/inkling-small:low PASS 100% 70 in → 59 out (50 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling-small:low__json_schema PASS 100% 70 in → 55 out (35 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
thinkingmachines/inkling-small:max PASS 100% 70 in → 260 out (270 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling-small:max__json_schema PASS 100% 70 in → 270 out (252 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd -e important.tmp"}
thinkingmachines/inkling-small:medium PASS 100% 70 in → 114 out (100 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling-small:medium__json_schema PASS 100% 70 in → 170 out (150 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
thinkingmachines/inkling-small:none PASS 100% 68 in → 12 out (0 reasoning)
git clean -fd --exclude=important.tmp
thinkingmachines/inkling-small:xhigh PASS 100% 70 in → 247 out (233 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 70 in → 276 out (259 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -fd -e important.tmp"}
thinkingmachines/inkling:high PASS 100% 70 in → 415 out (401 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling:high__json_schema PASS 100% 70 in → 182 out (161 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
thinkingmachines/inkling:low PASS 100% 70 in → 78 out (61 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling:max PASS 100% 70 in → 201 out (187 reasoning)
git clean -fd -e important.tmp
thinkingmachines/inkling:medium PASS 100% 70 in → 255 out (256 reasoning)
git clean -f -e important.tmp
thinkingmachines/inkling:medium__json_schema PASS 100% 70 in → 118 out (98 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
thinkingmachines/inkling:none PASS 100% 68 in → 12 out (0 reasoning)
git clean -f -e important.tmp
thinkingmachines/inkling:xhigh PASS 100% 70 in → 249 out (235 reasoning)
git clean -fd -e important.tmp
x-ai/grok-4.3:high PASS 100% 240 in → 473 out (463 reasoning)
`git clean -f -e important.tmp`
x-ai/grok-4.3:high__json_schema PASS 100% 296 in → 691 out (679 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.3:low PASS 100% 246 in → 333 out (325 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.3:low__json_schema PASS 100% 302 in → 356 out (340 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
x-ai/grok-4.3:max PASS 100% 240 in → 774 out (766 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.3:max__json_schema PASS 100% 296 in → 454 out (438 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
x-ai/grok-4.3:medium PASS 100% 246 in → 870 out (862 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.3:medium__json_schema PASS 100% 302 in → 541 out (525 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
x-ai/grok-4.3:none PASS 100% 238 in → 9 out (0 reasoning)
git clean -fd --exclude=important.tmp
x-ai/grok-4.3:xhigh PASS 100% 240 in → 846 out (836 reasoning)
git clean -f -- ":!important.tmp"
x-ai/grok-4.3:xhigh__json_schema PASS 100% 296 in → 1,096 out (1,084 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.5:high PASS 100% 261 in → 336 out (328 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.5:high__json_schema PASS 100% 327 in → 463 out (450 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.5:low PASS 100% 261 in → 184 out (171 reasoning)
```git git clean -f -e important.tmp ```
x-ai/grok-4.5:low__json_schema PASS 100% 327 in → 188 out (175 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.5:max PASS 100% 261 in → 434 out (422 reasoning)
``` git clean -f -e important.tmp ```
x-ai/grok-4.5:max__json_schema PASS 100% 327 in → 499 out (486 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.5:medium PASS 100% 261 in → 560 out (552 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.5:medium__json_schema PASS 100% 327 in → 440 out (427 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.5:xhigh PASS 100% 261 in → 430 out (422 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.5:xhigh__json_schema PASS 100% 327 in → 617 out (604 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.7:high PASS 100% 1,297 in → 295 out (287 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.7:high__json_schema PASS 100% 1,363 in → 532 out (519 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.7:low PASS 100% 1,297 in → 193 out (185 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.7:low__json_schema PASS 100% 1,363 in → 265 out (252 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.7:max PASS 100% 1,298 in → 659 out (651 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.7:max__json_schema PASS 100% 1,364 in → 355 out (342 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.7:medium PASS 100% 1,297 in → 257 out (249 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.7:medium__json_schema PASS 100% 1,363 in → 346 out (333 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
x-ai/grok-4.7:xhigh PASS 100% 1,298 in → 269 out (261 reasoning)
git clean -f -e important.tmp
x-ai/grok-4.7:xhigh__json_schema PASS 100% 1,364 in → 618 out (605 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
xiaomi/mimo-v2.6-flash:high PASS 100% 65 in → 136 out (126 reasoning)
git clean -f -e important.tmp
xiaomi/mimo-v2.6-flash:high__json_schema PASS 100% 65 in → 93 out (78 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp"}
xiaomi/mimo-v2.6-flash:low PASS 100% 65 in → 80 out (70 reasoning)
git clean -f -e important.tmp
xiaomi/mimo-v2.6-flash:low__json_schema PASS 100% 65 in → 58 out (41 reasoning)
git clean -f --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f --exclude=important.tmp"}
xiaomi/mimo-v2.6-flash:max PASS 100% 65 in → 108 out (97 reasoning)
git clean -f -e important.tmp
xiaomi/mimo-v2.6-flash:max__json_schema PASS 100% 65 in → 71 out (53 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
xiaomi/mimo-v2.6-flash:medium PASS 100% 65 in → 50 out (36 reasoning)
``` git clean -f -e important.tmp ```
xiaomi/mimo-v2.6-flash:medium__json_schema PASS 100% 65 in → 106 out (90 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
xiaomi/mimo-v2.6-flash:none PASS 100% 67 in → 9 out (0 reasoning)
git clean -f -e important.tmp
xiaomi/mimo-v2.6-flash:none__json_schema PASS 100% 67 in → 16 out (0 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
xiaomi/mimo-v2.6-flash:xhigh PASS 100% 65 in → 48 out (34 reasoning)
``` git clean -f -e important.tmp ```
xiaomi/mimo-v2.6-flash:xhigh__json_schema PASS 100% 65 in → 63 out (47 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
xiaomi/mimo-v2.6-pro:high PASS 100% 65 in → 21 out (10 reasoning)
git clean -f --exclude=important.tmp
xiaomi/mimo-v2.6-pro:high__json_schema PASS 100% 65 in → 59 out (38 reasoning)
git clean -f -x -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -x -e important.tmp" }
xiaomi/mimo-v2.6-pro:low PASS 100% 65 in → 23 out (13 reasoning)
git clean -f -e important.tmp
xiaomi/mimo-v2.6-pro:low__json_schema PASS 100% 65 in → 31 out (11 reasoning)
git clean -fd -x -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -x -e important.tmp" }
xiaomi/mimo-v2.6-pro:max PASS 100% 65 in → 61 out (51 reasoning)
git clean -fd -e important.tmp
xiaomi/mimo-v2.6-pro:max__json_schema PASS 100% 65 in → 56 out (37 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
xiaomi/mimo-v2.6-pro:medium PASS 100% 65 in → 19 out (9 reasoning)
git clean -f -e important.tmp
xiaomi/mimo-v2.6-pro:medium__json_schema PASS 100% 65 in → 28 out (9 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
xiaomi/mimo-v2.6-pro:xhigh PASS 100% 65 in → 15 out (5 reasoning)
git clean -fd -e important.tmp
xiaomi/mimo-v2.6-pro:xhigh__json_schema PASS 100% 65 in → 66 out (48 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
z-ai/glm-4.7-flash:high PASS 100% 63 in → 379 out (349 reasoning)
git clean -f -e important.tmp
z-ai/glm-4.7-flash:high__json_schema PASS 100% 63 in → 489 out (442 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -f -e important.tmp"}
z-ai/glm-4.7-flash:low PASS 100% 63 in → 1,287 out (1,285 reasoning)
git clean -f -e important.tmp
z-ai/glm-4.7-flash:medium PASS 100% 63 in → 1,560 out (1,579 reasoning)
git clean -f -e important.tmp
z-ai/glm-4.7-flash:xhigh PASS 100% 63 in → 716 out (706 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.2:high PASS 100% 70 in → 490 out (0 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.2:low PASS 100% 70 in → 492 out (475 reasoning)
git clean -fd -e important.tmp
z-ai/glm-5.2:low__json_schema PASS 100% 245 in → 216 out (200 reasoning)
git clean -f -d -e important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -d -e important.tmp"}
z-ai/glm-5.2:medium PASS 100% 70 in → 352 out (342 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.2:medium__json_schema PASS 100% 70 in → 403 out (361 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
z-ai/glm-5.2:none PASS 100% 64 in → 9 out (0 reasoning)
git clean -fd -e important.tmp
z-ai/glm-5.2:none__json_schema PASS 100% 66 in → 15 out (0 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
z-ai/glm-5.2:xhigh PASS 100% 70 in → 722 out (712 reasoning)
git clean -fd -e important.tmp
z-ai/glm-5.2:xhigh__json_schema PASS 100% 72 in → 494 out (477 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
z-ai/glm-5.3-flash:high PASS 100% 70 in → 73 out (63 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.3-flash:high__json_schema PASS 100% 70 in → 57 out (42 reasoning)
git clean -f -e important.tmp .
JSON Schema Structured Output
(raw) {"command":"git clean -f -e important.tmp ."}
z-ai/glm-5.3-flash:low PASS 100% 70 in → 15 out (0 reasoning)
``` git clean -f --exclude=important.tmp . ```
z-ai/glm-5.3-flash:low__json_schema PASS 100% 70 in → 17 out (0 reasoning)
git clean -e important.tmp -f
JSON Schema Structured Output
(raw) { "command": "git clean -e important.tmp -f" }
z-ai/glm-5.3-flash:max PASS 100% 70 in → 184 out (180 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.3-flash:max__json_schema PASS 100% 70 in → 437 out (420 reasoning)
git clean -f -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f -e important.tmp" }
z-ai/glm-5.3-flash:medium PASS 100% 70 in → 10 out (0 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.3-flash:medium__json_schema PASS 100% 70 in → 17 out (0 reasoning)
git clean -e important.tmp -f
JSON Schema Structured Output
(raw) { "command": "git clean -e important.tmp -f" }
z-ai/glm-5.3-flash:xhigh PASS 100% 70 in → 44 out (34 reasoning)
git clean -f -e important.tmp
z-ai/glm-5.3-flash:xhigh__json_schema PASS 100% 70 in → 78 out (56 reasoning)
git clean -fd -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fd -e important.tmp" }
arcee-ai/trinity-mini:high__json_schema FAIL 75% 65 in → 727 out (783 reasoning)
git clean -f&& git add important.tmp
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}]
arcee-ai/trinity-mini:medium FAIL 75% 65 in → 438 out (457 reasoning)
git clean -f -e "junk*.tmp junk2.txt"
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}]
minimax/minimax-m2.7:low__json_schema FAIL 75% 100 in → 29 out (29 reasoning)
git clean -f && git checkout -- important.tmp && git clean -f -e important.tmp -X
JSON Schema Structured Output
(raw) {"command": "git clean -f && git checkout -- important.tmp && git clean -f -e important.tmp -X"}
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}]
openai/gpt-5.6-terra:medium__json_schema FAIL 75% 96 in → 33 out (0 reasoning)
find . -type f -not -path './.git/*' ! -name important.tmp -delete
JSON Schema Structured Output
(raw) {"command":"find . -type f -not -path './.git/*' ! -name important.tmp -delete"}
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'tracked.txt'}]
poolside/laguna-xs-2.1:medium FAIL 75% 74 in → 699 out (689 reasoning)
git clean -fd -e '!important.tmp'
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}]
poolside/laguna-xs-2.1:none__json_schema FAIL 75% 74 in → 28 out (0 reasoning)
git clean -f -e !important.tmp -e *.tmp -e *.txt
JSON Schema Structured Output
(raw) { "command": "git clean -f -e !important.tmp -e *.tmp -e *.txt" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk2.txt'}]
arcee-ai/trinity-mini:xhigh FAIL 50% 65 in → 389 out (435 reasoning)
gitclean -f -e important.tmp
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
google/gemma-4-26b-a4b-it:none__json_schema FAIL 50% 82 in → 19 out (0 reasoning)
git clean -f !important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f !important.tmp" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.1-8b FAIL 50% 80 in → 11 out
git rm --cached junk1.tmp junk2.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
git reset HEAD important.tmp && git clean -dfx --except important.tmp
JSON Schema Structured Output
(raw) { "command": "git reset HEAD important.tmp && git clean -dfx --except important.tmp" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:high__json_schema FAIL 50% 87 in → 24 out (0 reasoning)
git rm --cached important.tmp && git add important.tmp
JSON Schema Structured Output
(raw) { "command": "git rm --cached important.tmp && git add important.tmp" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:low FAIL 50% 96 in → 233 out (209 reasoning)
git rm -r --cached junk1.tmp junk2.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:low__json_schema FAIL 50% 96 in → 34 out (0 reasoning)
git rm --cached important.tmp && git commit -m 'remove junk' && git clean -f junk1.tmp junk2.txt
JSON Schema Structured Output
(raw) {"command":"git rm --cached important.tmp && git commit -m 'remove junk' && git clean -f junk1.tmp junk2.txt"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:max__json_schema FAIL 50% 87 in → 18 out (0 reasoning)
git rm --cached important.tmp
JSON Schema Structured Output
(raw) { "command": "git rm --cached important.tmp" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:medium FAIL 50% 96 in → 77 out (59 reasoning)
git rm --force junk1.tmp junk2.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:medium__json_schema FAIL 50% 96 in → 23 out (0 reasoning)
git rm --cached --ignore-untracked important.tmp
JSON Schema Structured Output
(raw) { "command": "git rm --cached --ignore-untracked important.tmp" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:none FAIL 50% 87 in → 2,043 out (0 reasoning)
git rm --cached --ignore-untracked junk1.tmp junk2.txt Note: Since `important.tmp` is untracked and should be kept, we only remove the other untracked files (`junk1.tmp` and `junk2.txt`). However, the instruction says "Remove all untracked files except important.tmp". Since `important.tmp` is untracked (status `??`), and we want to keep it untracked (i.e., not commit it), we simply delete the other untracked files from the working directory (not staged or committed). But the user said "Output ONLY the git command, nothing else." and the context shows "Untracked files" list. Important: `git rm` without `--cached` removes from working tree and stages it (which would commit it if committed), but we don't want to commit `important.tmp` — we just want to delete the other untracked files from the working directory. Actually, "remove" in the context of untracked files likely means delete them from the filesystem (since they are untracked), not stage them. But the instruction says "git command", so the safest and most direct git command to delete untracked files is to use `git clean`, but `git clean` deletes by pattern and requires `-f` for force, and by default it only deletes ignored files or clean-files? Let's clarify: - `git rm` removes from index (if tracked) or from working tree if `--cached` is used; but for untracked files, `git rm` alone does nothing (it requires the file to be tracked or you need `--cached` to remove from working tree). - To delete untracked files from the working directory via git, you use `git clean`. `git clean -f -x` deletes all untracked files and ignored files, but we don't want to delete `important.tmp`. So we need a git command that deletes only the specified untracked files (junk1.tmp and junk2.txt), keeping important.tmp. `git clean` can take file arguments: `git clean -f <files>`. It will delete the listed untracked files (if they match ignored patterns? no, it deletes the listed files if they are untracked and not ignored? Actually, `git clean` deletes files that are untracked or ignored, but if you specify files on the command line, it deletes those files if they are untracked (and not ignored), otherwise it may not delete tracked files). According to git docs: `git clean [-f] [-i] [-n] [-x] [-X] [--] [<files>]` - `-f` : actually remove files (default is --dry-run without -f) - If `<files>` are given, only those files/directories are affected; otherwise all untracked and ignored files are affected. So `git clean -f junk1.tmp junk2.txt` will delete exactly those two untracked files, leaving `important.tmp` untouched. `important.tmp` is untracked but not listed in the clean command, so it won't be deleted. Is `important.tmp` ignored? No, "Ignored files: (none)", so it's not ignored. Thus, `git clean -f junk1.tmp junk2.txt` is the correct git command to delete the unwanted untracked files. But the user said "Remove all untracked files except important.tmp", and `important.tmp` is untracked, so we want to keep it untracked (i.e., not delete it), and delete the others. `git clean -f junk1.tmp junk2.txt` does exactly that. `git rm` would not be appropriate because `git rm` on untracked files requires `--cached` to remove from working tree, but `git rm --cached junk1.tmp` would stage it for removal (i.e., add to index for deletion), which is not what we want — we want to delete the file from disk, not stage it. The user said "remove", which in common git usage for untracked files means delete from disk. Also, the status shows `??` for untracked, so `git rm` without `--cached` would fail because the file isn't tracked. `git rm --cached` would stage a deletion (i.e., add to index to remove it), but then you'd have to commit to actually delete it from history, and it wouldn't delete the working copy immediately? Actually, `git rm --cached` removes the file from the index (if tracked) or from the working tree? Let's recall: - `git rm` without arguments removes from index and working tree (for tracked files). - `git rm --cached` removes from index only (working tree remains), which is used to drop local modifications without committing. But for untracked files, `git rm` does nothing (because they aren't in index), and `git rm --cached` also does nothing for untracked files (since they aren't in index). So `git rm` is not suitable for deleting untracked files from the working directory. Therefore, `git clean` is the correct git command. `git clean -f junk1.tmp junk2.txt` will delete those two files from the working directory. We should output ONLY the command, no explanation. Command: `git clean -f junk1.tmp junk2.txt` But let's confirm the status: - `important.tmp` is `??` (untracked, not ignored) - `junk1.tmp` and `junk2.txt` are `??` (untracked) - Ignored: none So `git clean -f junk1.tmp junk2.txt` will delete junk1.tmp and junk2.txt, keep important.tmp. Yes. Is there any risk that `git clean` deletes ignored files? No, because we are specifying the exact filenames, and `-x` would delete ignored files too, but we are not using `-x`, just `-f` with files, so only the listed files are deleted if they are untracked (and not ignored; but since ignored files are not tracked, `git clean` with file args would delete them if listed? The docs say: "If some of the arguments refer to untracked or ignored files, they will be ignored". Wait: from git-clean(1): > -x also delete ignored files (in addition to empty directories). > Without -x, ignore-patterns would prevent deletion of ignored files. > > <files> > > Files to exclude from removal. By default, all untracked files are removed. > > When specifying files, only those files (and their parent directories that become empty) are removed. > > If some of the arguments refer to untracked or ignored files, they will be ignored. Ah, important: "If some of the arguments refer to untracked or ignored files, they will be ignored." — no: > > "If some of the arguments refer to untracked or ignored files, they will be ignored." — actually the exact quote from man page (as of git 2.30): > > "If some of the arguments refer to untracked or ignored files, they will be ignored." Wait, let's recall precisely: from `man git-clean`: > -x Also delete ignored files. > > <files> > > Files to exclude from removal. By default, all untracked files are removed. > > When specifying files, only those files (and their parent directories that become empty) are removed. > > If some of the arguments refer to untracked or ignored files, they will be ignored. Yes, that seems to be the case: if you list an ignored file, `git clean` with file args will ignore it (i.e., not delete it) unless `-x` is used. But in our case, "Ignored files: (none)", so no ignored files, so junk1.tmp and junk2.txt are not ignored, just untracked. So `git clean -f junk1.tmp junk2.txt` will delete them. Also, `important.tmp` is untracked but not listed, so it won't be touched. Thus, `git clean -f junk1.tmp junk2.txt` is correct. But the user said "Output ONLY the git command, nothing else." So the answer is: `git clean -f junk1.tmp junk2.txt` However, sometimes people use `git rm` for untracked with `--cached`, but that stages the deletion, not deletes the file immediately from working tree? Let's see: `git rm --cached junk1.tmp` would fail because junk1.tmp is not tracked; `git rm junk1.tmp` would fail because it's not tracked. So `git rm` cannot delete untracked files; it can only remove tracked files from index/working tree. Therefore, `git clean` is the right command. Final command: `git clean -f junk1.tmp junk2.txt` Output only that string. git clean -f junk1.tmp junk2.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:none__json_schema FAIL 50% 87 in → 19 out (0 reasoning)
git rm --cached important.tmp
JSON Schema Structured Output
(raw) { "command": "git rm --cached important.tmp\n" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
ibm-granite/granite-4.2-8b:xhigh FAIL 50% 87 in → 6,429 out (6,401 reasoning)
git rm junk1.tmp junk2.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
inception/mercury-2.5:none FAIL 50% 57 in → 12 out (0 reasoning)
git clean -fx -X junk1.tmp junk2.txt
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
liquid/lfm-2-24b-a2b FAIL 50% 76 in → 13 out
git rm --ignore-unmatch --force important.tmp
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
minimax/minimax-m2.5:medium__json_schema FAIL 50% 100 in → 242 out (244 reasoning)
git clean -f !important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -f !important.tmp" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
minimax/minimax-m3:high__json_schema FAIL 50% 225 in → 13 out (0 reasoning)
git clean -fn
JSON Schema Structured Output
(raw) { "command": "git clean -fn" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
minimax/minimax-m3:medium__json_schema FAIL 50% 225 in → 18 out (0 reasoning)
git clean -f -- !(important.tmp)
JSON Schema Structured Output
(raw) { "command": "git clean -f -- !(important.tmp)" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
minimax/minimax-m3:xhigh__json_schema FAIL 50% 225 in → 17 out (0 reasoning)
git clean -f -- !(important.tmp)
JSON Schema Structured Output
(raw) { "command": "git clean -f -- !(important.tmp)" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
nvidia/nemotron-3-nano-30b-a3b:none FAIL 50% 81 in → 35 out (0 reasoning)
git clean -fdX --force --important.tmp ; git clean -nfd . > .gitignore; git clean -fdX --force --important.tmp
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
openai/gpt-oss-120b:high__json_schema FAIL 50% 122 in → 541 out (539 reasoning)
finalgit clean -fd -e important.tmp
JSON Schema Structured Output
(raw) {"command":"finalgit clean -fd -e important.tmp"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
openai/gpt-oss-20b:high__json_schema FAIL 50% 122 in → 17 out (0 reasoning)
git clean -f -X -X !important.tmp
JSON Schema Structured Output
(raw) {"command":"git clean -f -X -X !important.tmp"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
poolside/laguna-m.1:none FAIL 50% 74 in → 9 out (0 reasoning)
git clean -dX !important.tmp
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
poolside/laguna-xs.2:none FAIL 50% 111 in → 11 out (0 reasoning)
git clean -O -- 'important.tmp'
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
poolside/laguna-xs.2:none__json_schema FAIL 50% 111 in → 16 out (0 reasoning)
git clean -fdv --exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fdv --exclude=important.tmp"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
qwen/qwen3.8-flash:low__json_schema FAIL 50% 123 in → 641 out (623 reasoning)
git clean -fd --force exclude=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd --force exclude=important.tmp"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
qwen/qwen3.8-flash:medium__json_schema FAIL 50% 123 in → 364 out (347 reasoning)
git clean -fd --except=important.tmp
JSON Schema Structured Output
(raw) {"command": "git clean -fd --except=important.tmp"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
x-ai/grok-4.3:none__json_schema FAIL 50% 294 in → 15 out (0 reasoning)
git clean -f -x "!(important.tmp)"
JSON Schema Structured Output
(raw) {"command":"git clean -f -x \"!(important.tmp)\""}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
z-ai/glm-4.7-flash:none FAIL 50% 63 in → 9 out (0 reasoning)
git clean -f -X important.tmp
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
z-ai/glm-4.7-flash:none__json_schema FAIL 50% 63 in → 12 out (0 reasoning)
git clean -X -f
JSON Schema Structured Output
(raw) {"command": "git clean -X -f"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
arcee-ai/trinity-mini:medium__json_schema FAIL 25% 65 in → 622 out (629 reasoning)
git clean -f -e junk1.tmp -e junk2.txt
JSON Schema Structured Output
(raw) { "command": "git clean -f -e junk1.tmp -e junk2.txt" }
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}, {'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
openai/gpt-5.4-nano:low FAIL 25% 61 in → 14 out (0 reasoning)
git clean -fdX --exclude=important.tmp
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}, {'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
poolside/laguna-m.1:none__json_schema FAIL 25% 74 in → 15 out (0 reasoning)
git clean -fdx 'important.tmp'
JSON Schema Structured Output
(raw) {"command": "git clean -fdx 'important.tmp'"}
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}, {'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
qwen/qwen3.6-27b:medium__json_schema FAIL 25% 72 in → 20 out (0 reasoning)
git clean -fdX -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fdX -e important.tmp" }
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}, {'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 25% 72 in → 17 out (0 reasoning)
git clean -fdX -e important.tmp
JSON Schema Structured Output
(raw) { "command": "git clean -fdX -e important.tmp" }
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'important.tmp'}, {'type': 'file_not_exists', 'path': 'junk1.tmp'}, {'type': 'file_not_exists', 'path': 'junk2.txt'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -f -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git rm --ignore-unmatch --force important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: OpenAI-compatible response contained provider error: Provider returned error
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-768d91df-ef4f-943b-b890-5da269e6b361","request_id":"768d91df-ef4f-943b-b890-5da269e6b361"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-381ff324-9134-900a-b2e1-e37d00d8e735","request_id":"381ff324-9134-900a-b2e1-e37d00d8e735"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d9b3d969-eed0-900c-ac71-8568635bd3ca","request_id":"d9b3d969-eed0-900c-ac71-8568635bd3ca"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-69a888ea-81cd-90a6-b2d1-5ac336658bfe","request_id":"69a888ea-81cd-90a6-b2d1-5ac336658bfe"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4ec12378-8c9b-96d9-b08d-860dd7f36cbc","request_id":"4ec12378-8c9b-96d9-b08d-860dd7f36cbc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-5f3adb71-d38c-9a52-94b7-2060be39d590"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-29f74a4f-efe9-9883-a726-2aa8c2948ed0"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-9b889ab3-59be-9557-8af3-85fd4c22b164"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-9a584efe-3c6a-960b-8299-4faf2227d6ff"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-4a5bf91c-e93a-9b0a-b355-19512e759ff6"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-f76dd7da-81fa-9dfd-9815-bfa320312737"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-041e08ac-a43b-91e9-9c35-87c08fe40b9a","request_id":"041e08ac-a43b-91e9-9c35-87c08fe40b9a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4c764cf6-4000-94fe-8685-582e090fcb2f","request_id":"4c764cf6-4000-94fe-8685-582e090fcb2f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d421ab52-37cd-9cf8-89e9-9cf2e02af1af","request_id":"d421ab52-37cd-9cf8-89e9-9cf2e02af1af"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-708a0771-5ed5-9b47-8411-7747c2d2f61d","request_id":"708a0771-5ed5-9b47-8411-7747c2d2f61d"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e1743cee-f095-9554-8038-8829ccf8d13c","request_id":"e1743cee-f095-9554-8038-8829ccf8d13c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-bc5db4e5-08fb-9294-9767-26f56b4c6de2","request_id":"bc5db4e5-08fb-9294-9767-26f56b4c6de2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c7dc73b3-ef52-9ddb-9b64-ba40ca09362c","request_id":"c7dc73b3-ef52-9ddb-9b64-ba40ca09362c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-609c3ca6-0ab8-9e58-b2ff-82b130e601ce","request_id":"609c3ca6-0ab8-9e58-b2ff-82b130e601ce"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-47cc1881-5af4-9731-a74f-cdc13339eb57","request_id":"47cc1881-5af4-9731-a74f-cdc13339eb57"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c6adab0f-b3da-92ca-8676-cd7ff69ca17b","request_id":"c6adab0f-b3da-92ca-8676-cd7ff69ca17b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git clean -f -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -e important.tmp -f
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -f -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fd -e important.tmp{ "command": "git clean -fd -e important.tmp" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fd -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fd -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fd --exclude=important.tmp{ "command": "git clean -fd --exclude=important.tmp" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fd -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -f -e important.tmp
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)