Dry-run preview of untracked file removal
Tests ability to preview untracked file removal with git clean -n (dry-run). Evaluates safe-cleanup workflow.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'tracked' > tracked.txt
  5. 05 git add tracked.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 echo 'untracked' > untracked.txt
  8. 08 echo 'another' > another.txt
Prompt
Show what files would be removed by git clean without actually removing them. Output ONLY the git command, nothing else.
Expected
git clean -f -n

Scoped model quality, cost, API time, and token usage for git_clean/f002.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 90 in → 19 out (18 reasoning)
git clean -n
anthropic/claude-fable-5:high__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-fable-5:low PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-fable-5:low__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-fable-5:max PASS 100% 90 in → 170 out (39 reasoning)
git clean -n
anthropic/claude-fable-5:max__json_schema PASS 100% 300 in → 137 out (50 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-fable-5:medium PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-fable-5:medium__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-fable-5:xhigh PASS 100% 90 in → 30 out (28 reasoning)
git clean -n
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-haiku-4.5:high PASS 100% 103 in → 156 out (144 reasoning)
``` git clean -n ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 266 in → 248 out (233 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-haiku-4.5:low PASS 100% 103 in → 142 out (139 reasoning)
git clean -n
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 266 in → 174 out (171 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-haiku-4.5:medium PASS 100% 103 in → 125 out (109 reasoning)
``` git clean -n ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 266 in → 230 out (219 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-haiku-4.5:none PASS 100% 74 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 236 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 103 in → 120 out (104 reasoning)
``` git clean -n ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 266 in → 181 out (169 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.6:high PASS 100% 74 in → 12 out (0 reasoning)
``` git clean -n ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.6:low PASS 100% 74 in → 9 out (0 reasoning)
`git clean -n`
anthropic/claude-opus-4.6:low__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.6:max PASS 100% 74 in → 12 out (0 reasoning)
``` git clean -n ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 237 in → 74 out (48 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.6:medium PASS 100% 74 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.6:none PASS 100% 74 in → 12 out (0 reasoning)
``` git clean -dn ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.6:xhigh PASS 100% 74 in → 12 out (0 reasoning)
``` git clean -n ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.7:high PASS 100% 95 in → 9 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.7:high__json_schema PASS 100% 305 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.7:low PASS 100% 95 in → 9 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.7:low__json_schema PASS 100% 305 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.7:max PASS 100% 95 in → 14 out (0 reasoning)
``` git clean -n ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 305 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.7:medium PASS 100% 95 in → 9 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 305 in → 17 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.7:none PASS 100% 95 in → 9 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.7:none__json_schema PASS 100% 305 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.7:xhigh PASS 100% 95 in → 9 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 305 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.8:high PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:high__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.8:low PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:low__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.8:max PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:max__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.8:medium PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.8:none PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:none__json_schema PASS 100% 300 in → 13 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-opus-4.8:xhigh PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-4.6:high PASS 100% 74 in → 92 out (74 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-4.6:low PASS 100% 74 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-4.6:max PASS 100% 74 in → 100 out (78 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 237 in → 119 out (93 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-4.6:medium PASS 100% 74 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-4.6:none PASS 100% 74 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 74 in → 112 out (89 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 237 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-5:high PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-sonnet-5:high__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-5:low PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-sonnet-5:low__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-5:medium PASS 100% 90 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
anthropic/claude-sonnet-5:none PASS 100% 90 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-sonnet-5:none__json_schema PASS 100% 300 in → 13 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
anthropic/claude-sonnet-5:xhigh PASS 100% 90 in → 11 out (0 reasoning)
``` git clean -n ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 300 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
arcee-ai/trinity-large-thinking:high PASS 100% 61 in → 287 out (281 reasoning)
git clean -n
arcee-ai/trinity-large-thinking:low PASS 100% 61 in → 277 out (272 reasoning)
git clean -n
arcee-ai/trinity-large-thinking:medium PASS 100% 61 in → 254 out (248 reasoning)
git clean -n
arcee-ai/trinity-large-thinking:xhigh PASS 100% 59 in → 0 out (203 reasoning)
(empty output)
arcee-ai/trinity-mini:high PASS 100% 61 in → 348 out (380 reasoning)
git clean -n
arcee-ai/trinity-mini:high__json_schema PASS 100% 61 in → 378 out (417 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
arcee-ai/trinity-mini:low PASS 100% 61 in → 320 out (376 reasoning)
git clean -n
arcee-ai/trinity-mini:low__json_schema PASS 100% 61 in → 399 out (456 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command":"git clean -n" }
arcee-ai/trinity-mini:medium PASS 100% 61 in → 377 out (428 reasoning)
git clean -n
arcee-ai/trinity-mini:medium__json_schema PASS 100% 61 in → 358 out (401 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
arcee-ai/trinity-mini:xhigh PASS 100% 61 in → 605 out (682 reasoning)
gitclean -n
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 61 in → 344 out (377 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 144 in → 302 out (321 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 138 in → 90 out (84 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 59 in → 133 out (132 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 201 in → 115 out (105 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 151 in → 239 out (234 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 138 in → 109 out (99 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 59 in → 60 out (54 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 201 in → 1,809 out (1,789 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
deepseek/deepseek-v4-flash-0731:none PASS 100% 59 in → 5 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 59 in → 14 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 138 in → 67 out (62 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 144 in → 293 out (294 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash:high PASS 100% 59 in → 104 out (95 reasoning)
git clean -n
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 123 in → 122 out (120 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
deepseek/deepseek-v4-flash:low PASS 100% 59 in → 107 out (101 reasoning)
git clean -n
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 61 in → 117 out (103 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash:medium PASS 100% 59 in → 109 out (103 reasoning)
git clean -n
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 61 in → 116 out (103 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash:none PASS 100% 59 in → 4 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 61 in → 11 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 138 in → 303 out (297 reasoning)
git clean -n
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 138 in → 296 out (296 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-pro:high PASS 100% 59 in → 5 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 61 in → 93 out (79 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-pro:low PASS 100% 59 in → 119 out (114 reasoning)
git clean -n
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 246 in → 82 out (71 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
deepseek/deepseek-v4-pro:medium PASS 100% 59 in → 163 out (166 reasoning)
git clean -n
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 59 in → 9 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
deepseek/deepseek-v4-pro:none PASS 100% 59 in → 4 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 61 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 59 in → 5 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 324 in → 131 out (120 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3-flash-preview:high PASS 100% 62 in → 167 out (163 reasoning)
git clean -n
google/gemini-3-flash-preview:high__json_schema PASS 100% 119 in → 178 out (169 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3-flash-preview:low PASS 100% 62 in → 315 out (311 reasoning)
git clean -n
google/gemini-3-flash-preview:low__json_schema PASS 100% 119 in → 282 out (274 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3-flash-preview:medium PASS 100% 62 in → 173 out (169 reasoning)
git clean -n
google/gemini-3-flash-preview:medium__json_schema PASS 100% 119 in → 241 out (233 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3-flash-preview:xhigh PASS 100% 62 in → 144 out (140 reasoning)
git clean -n
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 119 in → 431 out (423 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 63 in → 345 out (341 reasoning)
git clean -n
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 63 in → 196 out (187 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 62 in → 112 out (108 reasoning)
git clean -nd
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 63 in → 122 out (113 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 63 in → 123 out (119 reasoning)
git clean -nd
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 119 in → 177 out (163 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 63 in → 346 out (342 reasoning)
git clean -n
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 63 in → 239 out (230 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.1-pro-preview:high PASS 100% 62 in → 182 out (178 reasoning)
git clean -n
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 119 in → 192 out (184 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.1-pro-preview:low PASS 100% 62 in → 162 out (158 reasoning)
git clean -n
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 119 in → 80 out (71 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.1-pro-preview:medium PASS 100% 62 in → 222 out (218 reasoning)
git clean -n
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 119 in → 225 out (216 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 62 in → 520 out (516 reasoning)
git clean -n
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 119 in → 143 out (134 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.5-flash-lite:high PASS 100% 62 in → 294 out (290 reasoning)
git clean -n
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 63 in → 265 out (256 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.5-flash-lite:low PASS 100% 62 in → 4 out (0 reasoning)
git clean -nd
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 63 in → 9 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash-lite:max PASS 100% 62 in → 311 out (307 reasoning)
git clean -n
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 63 in → 245 out (237 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.5-flash-lite:medium PASS 100% 63 in → 207 out (203 reasoning)
git clean -n
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 119 in → 279 out (271 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 62 in → 261 out (257 reasoning)
git clean -n
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 119 in → 258 out (250 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3.5-flash:high PASS 100% 62 in → 346 out (342 reasoning)
git clean -n
google/gemini-3.5-flash:high__json_schema PASS 100% 119 in → 309 out (300 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.5-flash:low PASS 100% 62 in → 244 out (240 reasoning)
git clean -n
google/gemini-3.5-flash:low__json_schema PASS 100% 119 in → 99 out (90 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.5-flash:medium PASS 100% 62 in → 498 out (494 reasoning)
git clean -n
google/gemini-3.5-flash:medium__json_schema PASS 100% 119 in → 267 out (253 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemini-3.5-flash:xhigh PASS 100% 62 in → 399 out (395 reasoning)
git clean -n
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 119 in → 298 out (289 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemini-3.6-flash:high PASS 100% 63 in → 277 out (273 reasoning)
git clean -n
google/gemini-3.6-flash:high__json_schema PASS 100% 119 in → 271 out (263 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.6-flash:low PASS 100% 63 in → 156 out (152 reasoning)
git clean -n
google/gemini-3.6-flash:low__json_schema PASS 100% 119 in → 8 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.6-flash:max PASS 100% 62 in → 271 out (267 reasoning)
git clean -n
google/gemini-3.6-flash:max__json_schema PASS 100% 63 in → 274 out (266 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.6-flash:medium PASS 100% 63 in → 208 out (204 reasoning)
git clean -n
google/gemini-3.6-flash:medium__json_schema PASS 100% 119 in → 281 out (273 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemini-3.6-flash:xhigh PASS 100% 63 in → 271 out (267 reasoning)
git clean -n
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 63 in → 165 out (157 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemma-4-26b-a4b-it:high PASS 100% 78 in → 200 out (195 reasoning)
git clean -n
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 77 in → 165 out (143 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-26b-a4b-it:low PASS 100% 78 in → 208 out (191 reasoning)
git clean -n
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 78 in → 441 out (382 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-26b-a4b-it:medium PASS 100% 78 in → 251 out (222 reasoning)
git clean -n
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 78 in → 353 out (312 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-26b-a4b-it:none PASS 100% 75 in → 5 out (0 reasoning)
git clean -n
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 75 in → 15 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 78 in → 232 out (205 reasoning)
git clean -n
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 77 in → 199 out (167 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-31b-it:high PASS 100% 78 in → 198 out (1 reasoning)
git clean -n
google/gemma-4-31b-it:high__json_schema PASS 100% 78 in → 165 out (138 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-31b-it:low PASS 100% 77 in → 122 out (102 reasoning)
git clean -n
google/gemma-4-31b-it:low__json_schema PASS 100% 78 in → 151 out (128 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
google/gemma-4-31b-it:medium PASS 100% 78 in → 153 out (136 reasoning)
git clean -n
google/gemma-4-31b-it:medium__json_schema PASS 100% 81 in → 121 out (111 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
google/gemma-4-31b-it:none PASS 100% 75 in → 5 out (0 reasoning)
git clean -n
google/gemma-4-31b-it:none__json_schema PASS 100% 75 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
google/gemma-4-31b-it:xhigh PASS 100% 77 in → 139 out (124 reasoning)
git clean -n
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 78 in → 166 out (144 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
ibm-granite/granite-4.1-8b PASS 100% 71 in → 5 out
git clean -n
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
inclusionai/ling-2.6-flash PASS 100% 77 in → 10 out
```sh git clean -n ```
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
liquid/lfm-2-24b-a2b PASS 100% 66 in → 12 out
git clean -dn --exclude="*.txt"
minimax/minimax-m2.5:high PASS 100% 89 in → 155 out (150 reasoning)
git clean -n
minimax/minimax-m2.5:high__json_schema PASS 100% 89 in → 851 out (839 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m2.5:low PASS 100% 89 in → 116 out (141 reasoning)
git clean -n
minimax/minimax-m2.5:low__json_schema PASS 100% 89 in → 120 out (119 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m2.5:medium PASS 100% 89 in → 85 out (90 reasoning)
git clean -n
minimax/minimax-m2.5:medium__json_schema PASS 100% 89 in → 215 out (221 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m2.5:xhigh PASS 100% 89 in → 229 out (223 reasoning)
git clean -n
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 89 in → 157 out (145 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m2.7:high PASS 100% 89 in → 255 out (277 reasoning)
git clean -n
minimax/minimax-m2.7:high__json_schema PASS 100% 89 in → 13 out (13 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m2.7:low PASS 100% 89 in → 349 out (343 reasoning)
git clean -n
minimax/minimax-m2.7:low__json_schema PASS 100% 221 in → 314 out (304 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
minimax/minimax-m2.7:medium PASS 100% 92 in → 213 out (208 reasoning)
git clean -n
minimax/minimax-m2.7:medium__json_schema PASS 100% 221 in → 314 out (304 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
minimax/minimax-m2.7:xhigh PASS 100% 89 in → 296 out (289 reasoning)
git clean -n
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 221 in → 204 out (194 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
minimax/minimax-m3:high PASS 100% 227 in → 42 out (39 reasoning)
git clean -n
minimax/minimax-m3:high__json_schema PASS 100% 214 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
minimax/minimax-m3:low PASS 100% 227 in → 42 out (39 reasoning)
git clean -nd
minimax/minimax-m3:low__json_schema PASS 100% 214 in → 13 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m3:medium PASS 100% 227 in → 50 out (45 reasoning)
git clean -n
minimax/minimax-m3:medium__json_schema PASS 100% 214 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
minimax/minimax-m3:xhigh PASS 100% 227 in → 50 out (45 reasoning)
git clean -n
minimax/minimax-m3:xhigh__json_schema PASS 100% 214 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
mistralai/devstral-2512 PASS 100% 58 in → 10 out
```bash git clean -n ```
mistralai/devstral-2512__json_schema PASS 100% 58 in → 10 out
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:high PASS 100% 70 in → 272 out (266 reasoning)
git clean -n
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 58 in → 264 out (265 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:low PASS 100% 70 in → 257 out (273 reasoning)
git clean -n
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 58 in → 177 out (190 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:medium PASS 100% 70 in → 5,921 out (6,555 reasoning)
git clean -n
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 58 in → 243 out (243 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:none PASS 100% 70 in → 5 out (0 reasoning)
git clean -n
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 58 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 70 in → 627 out (693 reasoning)
git clean -n
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 58 in → 168 out (175 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
moonshotai/kimi-k2.6:high PASS 100% 61 in → 1,296 out (1,292 reasoning)
git clean -n
moonshotai/kimi-k2.6:high__json_schema PASS 100% 61 in → 570 out (557 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
moonshotai/kimi-k2.6:low PASS 100% 61 in → 819 out (813 reasoning)
git clean -n
moonshotai/kimi-k2.6:low__json_schema PASS 100% 61 in → 871 out (862 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
moonshotai/kimi-k2.6:medium PASS 100% 61 in → 449 out (443 reasoning)
git clean -n
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 60 in → 1,864 out (1,721 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
moonshotai/kimi-k2.6:xhigh PASS 100% 61 in → 732 out (726 reasoning)
git clean -n
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 61 in → 726 out (717 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
moonshotai/kimi-k2.7-code:high PASS 100% 61 in → 150 out (144 reasoning)
git clean -n
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 61 in → 653 out (669 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
moonshotai/kimi-k2.7-code:low PASS 100% 60 in → 183 out (177 reasoning)
git clean -n
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 144 in → 59 out (48 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
moonshotai/kimi-k2.7-code:medium PASS 100% 61 in → 170 out (164 reasoning)
git clean -n
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 61 in → 55 out (42 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 61 in → 225 out (217 reasoning)
git clean -n -d
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 61 in → 272 out (258 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
moonshotai/kimi-k3:high PASS 100% 138 in → 371 out (352 reasoning)
git clean -n
moonshotai/kimi-k3:high__json_schema PASS 100% 138 in → 149 out (130 reasoning)
git clean -n
moonshotai/kimi-k3:low PASS 100% 138 in → 210 out (188 reasoning)
``` git clean -n ```
moonshotai/kimi-k3:low__json_schema PASS 100% 138 in → 172 out (153 reasoning)
git clean -n
moonshotai/kimi-k3:max PASS 100% 138 in → 81 out (62 reasoning)
git clean -n
moonshotai/kimi-k3:max__json_schema PASS 100% 138 in → 169 out (147 reasoning)
``` git clean -n ```
moonshotai/kimi-k3:medium PASS 100% 138 in → 185 out (166 reasoning)
git clean -n
moonshotai/kimi-k3:medium__json_schema PASS 100% 138 in → 99 out (80 reasoning)
git clean -n
moonshotai/kimi-k3:xhigh PASS 100% 138 in → 251 out (232 reasoning)
git clean -n
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 138 in → 133 out (114 reasoning)
git clean -n
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 71 in → 217 out (224 reasoning)
git clean -n
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 71 in → 313 out (319 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 71 in → 130 out (136 reasoning)
git clean -n
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 71 in → 223 out (233 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 71 in → 157 out (167 reasoning)
git clean -n
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 71 in → 234 out (254 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 71 in → 14 out (0 reasoning)
git clean -n -d -x -e .gitignore
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 71 in → 14 out (0 reasoning)
git clean -nxd
JSON Schema Structured Output
(raw) { "command": "git clean -nxd" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 71 in → 212 out (219 reasoning)
git clean -n
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 71 in → 170 out (174 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 71 in → 116 out (105 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 71 in → 85 out (69 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 71 in → 78 out (71 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 71 in → 145 out (127 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 71 in → 160 out (149 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 71 in → 153 out (139 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 71 in → 5 out (0 reasoning)
git clean -nd
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 71 in → 13 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 71 in → 89 out (77 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 71 in → 187 out (171 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
openai/gpt-5.4-mini:high PASS 100% 57 in → 131 out (121 reasoning)
git clean -n
openai/gpt-5.4-mini:high__json_schema PASS 100% 92 in → 109 out (91 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4-mini:low PASS 100% 57 in → 78 out (68 reasoning)
git clean -n
openai/gpt-5.4-mini:low__json_schema PASS 100% 92 in → 73 out (55 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4-mini:medium PASS 100% 57 in → 68 out (58 reasoning)
git clean -n
openai/gpt-5.4-mini:medium__json_schema PASS 100% 92 in → 88 out (70 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4-mini:none PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.4-mini:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4-mini:xhigh PASS 100% 57 in → 226 out (216 reasoning)
git clean -n
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 92 in → 304 out (286 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4-nano:high PASS 100% 57 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.4-nano:high__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:low PASS 100% 57 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.4-nano:low__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:medium PASS 100% 57 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.4-nano:medium__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:none PASS 100% 57 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.4-nano:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:xhigh PASS 100% 57 in → 196 out (186 reasoning)
git clean -n
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 92 in → 267 out (249 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:high PASS 100% 57 in → 95 out (85 reasoning)
git clean -n
openai/gpt-5.4:high__json_schema PASS 100% 92 in → 177 out (159 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4:low PASS 100% 57 in → 31 out (21 reasoning)
git clean -nd
openai/gpt-5.4:low__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:medium PASS 100% 57 in → 86 out (76 reasoning)
git clean -n
openai/gpt-5.4:medium__json_schema PASS 100% 92 in → 79 out (61 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.4:none PASS 100% 57 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.4:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:xhigh PASS 100% 57 in → 92 out (82 reasoning)
git clean -n
openai/gpt-5.4:xhigh__json_schema PASS 100% 92 in → 534 out (516 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.5:high PASS 100% 57 in → 162 out (152 reasoning)
git clean -n
openai/gpt-5.5:high__json_schema PASS 100% 92 in → 69 out (51 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.5:low PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.5:low__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.5:medium PASS 100% 57 in → 63 out (53 reasoning)
git clean -n
openai/gpt-5.5:medium__json_schema PASS 100% 92 in → 17 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
openai/gpt-5.5:none PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.5:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.5:xhigh PASS 100% 57 in → 225 out (215 reasoning)
git clean -n
openai/gpt-5.5:xhigh__json_schema PASS 100% 92 in → 176 out (158 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-luna:high PASS 100% 57 in → 66 out (56 reasoning)
git clean -n
openai/gpt-5.6-luna:high__json_schema PASS 100% 92 in → 77 out (59 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-luna:low PASS 100% 57 in → 56 out (46 reasoning)
git clean -n
openai/gpt-5.6-luna:low__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-luna:max PASS 100% 57 in → 75 out (65 reasoning)
git clean -n
openai/gpt-5.6-luna:max__json_schema PASS 100% 92 in → 534 out (516 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-luna:medium PASS 100% 57 in → 48 out (38 reasoning)
git clean -n
openai/gpt-5.6-luna:medium__json_schema PASS 100% 92 in → 68 out (50 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-luna:none PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-luna:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-luna:xhigh PASS 100% 57 in → 79 out (69 reasoning)
git clean -n
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 92 in → 71 out (53 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-sol:high PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-sol:high__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-sol:low PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-sol:low__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-sol:max PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-sol:max__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-sol:medium PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-sol:medium__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-sol:none PASS 100% 57 in → 10 out (0 reasoning)
git clean -n -f
openai/gpt-5.6-sol:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-sol:xhigh PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-terra:high PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-terra:high__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-terra:low PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-terra:low__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:max PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-terra:max__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-terra:medium PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-terra:medium__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-5.6-terra:none PASS 100% 57 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-terra:none__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:xhigh PASS 100% 57 in → 8 out (0 reasoning)
git clean -n
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 92 in → 16 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-oss-120b:high PASS 100% 118 in → 372 out (408 reasoning)
git clean -n
openai/gpt-oss-120b:high__json_schema PASS 100% 118 in → 541 out (526 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-oss-120b:low PASS 100% 105 in → 75 out (51 reasoning)
git clean -n -d
openai/gpt-oss-120b:medium PASS 100% 116 in → 154 out (148 reasoning)
git clean -n
openai/gpt-oss-120b:medium__json_schema PASS 100% 105 in → 130 out (128 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-oss-120b:xhigh PASS 100% 118 in → 477 out (459 reasoning)
git clean -n
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 105 in → 232 out (242 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-oss-20b:high PASS 100% 116 in → 268 out (282 reasoning)
git clean -n
openai/gpt-oss-20b:high__json_schema PASS 100% 105 in → 333 out (352 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
openai/gpt-oss-20b:low PASS 100% 123 in → 32 out (16 reasoning)
git clean -n -d
openai/gpt-oss-20b:low__json_schema PASS 100% 118 in → 51 out (31 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
openai/gpt-oss-20b:medium PASS 100% 118 in → 202 out (198 reasoning)
git clean -n
openai/gpt-oss-20b:medium__json_schema PASS 100% 218 in → 228 out (207 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
openai/gpt-oss-20b:xhigh PASS 100% 105 in → 326 out (334 reasoning)
git clean -n
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 118 in → 12 out (0 reasoning)
git clean -n -d -f
JSON Schema Structured Output
(raw) {"command":"git clean -n -d -f"}
poolside/laguna-m.1:high PASS 100% 70 in → 260 out (253 reasoning)
git clean -n
poolside/laguna-m.1:high__json_schema PASS 100% 70 in → 396 out (385 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
poolside/laguna-m.1:low PASS 100% 70 in → 243 out (236 reasoning)
git clean -n
poolside/laguna-m.1:low__json_schema PASS 100% 70 in → 629 out (619 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
poolside/laguna-m.1:medium PASS 100% 70 in → 502 out (495 reasoning)
git clean -nd
poolside/laguna-m.1:medium__json_schema PASS 100% 70 in → 408 out (393 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-m.1:none PASS 100% 70 in → 8 out (0 reasoning)
git clean -d -n
poolside/laguna-m.1:none__json_schema PASS 100% 70 in → 10 out (0 reasoning)
git clean -nX
JSON Schema Structured Output
(raw) {"command": "git clean -nX"}
poolside/laguna-m.1:xhigh PASS 100% 70 in → 455 out (448 reasoning)
git clean -n
poolside/laguna-m.1:xhigh__json_schema PASS 100% 70 in → 357 out (341 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-xs-2.1:high PASS 100% 70 in → 852 out (847 reasoning)
git clean -n
poolside/laguna-xs-2.1:high__json_schema PASS 100% 70 in → 295 out (279 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-xs-2.1:low PASS 100% 70 in → 204 out (199 reasoning)
git clean -n
poolside/laguna-xs-2.1:low__json_schema PASS 100% 70 in → 444 out (433 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
poolside/laguna-xs-2.1:medium PASS 100% 70 in → 149 out (144 reasoning)
git clean -n
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 70 in → 438 out (422 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-xs-2.1:none PASS 100% 70 in → 4 out (0 reasoning)
git clean -n
poolside/laguna-xs-2.1:none__json_schema PASS 100% 70 in → 12 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
poolside/laguna-xs-2.1:xhigh PASS 100% 70 in → 209 out (204 reasoning)
git clean -n
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 70 in → 1,283 out (1,270 reasoning)
git clean -n -f
JSON Schema Structured Output
(raw) {"command": "git clean -n -f"}
poolside/laguna-xs.2:high PASS 100% 107 in → 170 out (163 reasoning)
git clean -n
poolside/laguna-xs.2:high__json_schema PASS 100% 107 in → 115 out (99 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-xs.2:low PASS 100% 107 in → 310 out (303 reasoning)
git clean -n
poolside/laguna-xs.2:low__json_schema PASS 100% 107 in → 96 out (85 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
poolside/laguna-xs.2:medium PASS 100% 107 in → 84 out (77 reasoning)
git clean -n
poolside/laguna-xs.2:medium__json_schema PASS 100% 107 in → 326 out (315 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
poolside/laguna-xs.2:none PASS 100% 107 in → 6 out (0 reasoning)
git clean -n
poolside/laguna-xs.2:none__json_schema PASS 100% 107 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
poolside/laguna-xs.2:xhigh PASS 100% 107 in → 278 out (271 reasoning)
git clean -n
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 107 in → 116 out (100 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-27b:high PASS 100% 66 in → 996 out (1 reasoning)
git clean -n
qwen/qwen3.6-27b:high__json_schema PASS 100% 66 in → 913 out (1 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-27b:low PASS 100% 66 in → 497 out (466 reasoning)
git clean -n
qwen/qwen3.6-27b:low__json_schema PASS 100% 66 in → 698 out (652 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-27b:medium PASS 100% 66 in → 1,254 out (1,209 reasoning)
git clean -n
qwen/qwen3.6-27b:medium__json_schema PASS 100% 68 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
qwen/qwen3.6-27b:none PASS 100% 68 in → 5 out (0 reasoning)
git clean -n
qwen/qwen3.6-27b:none__json_schema PASS 100% 68 in → 17 out (0 reasoning)
git clean -n -f
JSON Schema Structured Output
(raw) { "command": "git clean -n -f" }
qwen/qwen3.6-27b:xhigh PASS 100% 66 in → 1,031 out (1,022 reasoning)
git clean -n
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 66 in → 933 out (898 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-35b-a3b:high PASS 100% 66 in → 1,085 out (1,072 reasoning)
git clean -n
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 66 in → 749 out (708 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-35b-a3b:low PASS 100% 66 in → 385 out (377 reasoning)
git clean -n
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 66 in → 791 out (737 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 66 in → 848 out (820 reasoning)
git clean -n
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 66 in → 1,000 out (909 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-35b-a3b:none PASS 100% 68 in → 6 out (0 reasoning)
git clean --dry-run
qwen/qwen3.6-35b-a3b:none__json_schema PASS 100% 68 in → 20 out (0 reasoning)
git clean -n -d --dry-run
JSON Schema Structured Output
(raw) { "command": "git clean -n -d --dry-run" }
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 66 in → 778 out (753 reasoning)
git clean -n
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 66 in → 676 out (660 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-flash:high PASS 100% 66 in → 837 out (828 reasoning)
git clean -n
qwen/qwen3.6-flash:low PASS 100% 66 in → 965 out (956 reasoning)
git clean -n
qwen/qwen3.6-flash:medium PASS 100% 66 in → 1,427 out (1,417 reasoning)
git clean --dry-run
qwen/qwen3.6-flash:none PASS 100% 68 in → 4 out (0 reasoning)
git clean -n
qwen/qwen3.6-flash:xhigh PASS 100% 66 in → 830 out (821 reasoning)
git clean -n
qwen/qwen3.7-flash:high PASS 100% 66 in → 803 out (794 reasoning)
git clean -n
qwen/qwen3.7-flash:low PASS 100% 66 in → 675 out (666 reasoning)
git clean -n
qwen/qwen3.7-flash:max PASS 100% 66 in → 334 out (325 reasoning)
git clean -n
qwen/qwen3.7-flash:medium PASS 100% 66 in → 883 out (874 reasoning)
git clean -n
qwen/qwen3.7-flash:none PASS 100% 68 in → 5 out (0 reasoning)
git clean --dry-run
qwen/qwen3.7-flash:xhigh PASS 100% 66 in → 982 out (973 reasoning)
git clean -n
qwen/qwen3.7-max:high PASS 100% 66 in → 177 out (168 reasoning)
git clean -n
qwen/qwen3.7-max:low PASS 100% 66 in → 176 out (167 reasoning)
git clean -n
qwen/qwen3.7-max:medium PASS 100% 66 in → 253 out (244 reasoning)
git clean -n
qwen/qwen3.7-max:none PASS 100% 68 in → 4 out (0 reasoning)
git clean -n
qwen/qwen3.7-max:xhigh PASS 100% 66 in → 155 out (146 reasoning)
git clean -n
qwen/qwen3.7-plus:high PASS 100% 66 in → 635 out (626 reasoning)
git clean -n
qwen/qwen3.7-plus:low PASS 100% 66 in → 591 out (582 reasoning)
git clean -n
qwen/qwen3.7-plus:medium PASS 100% 66 in → 457 out (448 reasoning)
git clean -n
qwen/qwen3.7-plus:none PASS 100% 68 in → 4 out (0 reasoning)
git clean -n
qwen/qwen3.7-plus:xhigh PASS 100% 66 in → 382 out (373 reasoning)
git clean -n
tencent/hy3:high PASS 100% 64 in → 341 out (335 reasoning)
git clean -n
tencent/hy3:high__json_schema PASS 100% 64 in → 658 out (667 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
tencent/hy3:low PASS 100% 64 in → 378 out (372 reasoning)
git clean -n
tencent/hy3:low__json_schema PASS 100% 64 in → 221 out (213 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
tencent/hy3:medium PASS 100% 64 in → 203 out (197 reasoning)
git clean -n
tencent/hy3:medium__json_schema PASS 100% 64 in → 209 out (198 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
tencent/hy3:none PASS 100% 67 in → 5 out (0 reasoning)
git clean -nd
tencent/hy3:xhigh PASS 100% 64 in → 346 out (340 reasoning)
git clean -n
tencent/hy3:xhigh__json_schema PASS 100% 64 in → 490 out (456 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
thinkingmachines/inkling-small:high PASS 100% 66 in → 152 out (142 reasoning)
git clean -n
thinkingmachines/inkling-small:high__json_schema PASS 100% 66 in → 96 out (79 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
thinkingmachines/inkling-small:low PASS 100% 66 in → 59 out (46 reasoning)
git clean -n
thinkingmachines/inkling-small:low__json_schema PASS 100% 66 in → 54 out (40 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
thinkingmachines/inkling-small:max PASS 100% 66 in → 110 out (100 reasoning)
git clean -n
thinkingmachines/inkling-small:max__json_schema PASS 100% 66 in → 207 out (193 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
thinkingmachines/inkling-small:medium PASS 100% 66 in → 29 out (19 reasoning)
git clean -n
thinkingmachines/inkling-small:medium__json_schema PASS 100% 66 in → 104 out (90 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
thinkingmachines/inkling-small:none PASS 100% 64 in → 7 out (0 reasoning)
git clean -n
thinkingmachines/inkling-small:xhigh PASS 100% 66 in → 143 out (133 reasoning)
git clean -n
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 66 in → 110 out (96 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
thinkingmachines/inkling:high PASS 100% 66 in → 76 out (65 reasoning)
git clean -n
thinkingmachines/inkling:high__json_schema PASS 100% 66 in → 109 out (96 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
thinkingmachines/inkling:low PASS 100% 66 in → 27 out (17 reasoning)
git clean -n
thinkingmachines/inkling:max PASS 100% 66 in → 216 out (206 reasoning)
git clean -n
thinkingmachines/inkling:medium PASS 100% 66 in → 73 out (63 reasoning)
git clean -n
thinkingmachines/inkling:medium__json_schema PASS 100% 66 in → 98 out (82 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
thinkingmachines/inkling:none PASS 100% 64 in → 8 out (0 reasoning)
git clean -n
thinkingmachines/inkling:none__json_schema PASS 100% 64 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n"}
thinkingmachines/inkling:xhigh PASS 100% 66 in → 130 out (120 reasoning)
git clean -n
thinkingmachines/inkling:xhigh__json_schema PASS 100% 66 in → 61 out (45 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
x-ai/grok-4.3:high PASS 100% 236 in → 197 out (193 reasoning)
git clean -n
x-ai/grok-4.3:high__json_schema PASS 100% 292 in → 385 out (377 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
x-ai/grok-4.3:low PASS 100% 242 in → 394 out (290 reasoning)
git clean -n The command `git clean -n` (or equivalently `--dry-run`) performs a dry-run of `git clean`, listing only the untracked files that would be deleted while leaving the working tree untouched. Because the supplied status shows exactly the two untracked files `another.txt` and `untracked.txt` and no ignored files, the output of the dry-run will be those two paths. No other flags (e.g., `-d` or `-x`) are required for this scenario.
x-ai/grok-4.3:low__json_schema PASS 100% 298 in → 308 out (299 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
x-ai/grok-4.3:max PASS 100% 236 in → 210 out (206 reasoning)
git clean -n
x-ai/grok-4.3:max__json_schema PASS 100% 292 in → 349 out (337 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
x-ai/grok-4.3:medium PASS 100% 242 in → 422 out (418 reasoning)
git clean -n
x-ai/grok-4.3:medium__json_schema PASS 100% 298 in → 447 out (435 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
x-ai/grok-4.3:none PASS 100% 234 in → 6 out (0 reasoning)
git clean -n -d
x-ai/grok-4.3:none__json_schema PASS 100% 290 in → 8 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
x-ai/grok-4.3:xhigh PASS 100% 236 in → 167 out (160 reasoning)
``` git clean -n ```
x-ai/grok-4.3:xhigh__json_schema PASS 100% 292 in → 439 out (431 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
x-ai/grok-4.5:high PASS 100% 257 in → 315 out (311 reasoning)
git clean -n
x-ai/grok-4.5:high__json_schema PASS 100% 323 in → 302 out (293 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
x-ai/grok-4.5:low PASS 100% 257 in → 134 out (126 reasoning)
``` git clean -n ```
x-ai/grok-4.5:low__json_schema PASS 100% 323 in → 253 out (244 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
x-ai/grok-4.5:max PASS 100% 257 in → 313 out (309 reasoning)
git clean -n
x-ai/grok-4.5:max__json_schema PASS 100% 323 in → 471 out (462 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
x-ai/grok-4.5:medium PASS 100% 257 in → 322 out (318 reasoning)
git clean -n
x-ai/grok-4.5:medium__json_schema PASS 100% 323 in → 317 out (308 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
x-ai/grok-4.5:xhigh PASS 100% 257 in → 173 out (169 reasoning)
git clean -n
x-ai/grok-4.5:xhigh__json_schema PASS 100% 323 in → 419 out (410 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
z-ai/glm-4.7-flash:high PASS 100% 57 in → 360 out (358 reasoning)
git clean -n
z-ai/glm-4.7-flash:low PASS 100% 57 in → 414 out (407 reasoning)
git clean -n
z-ai/glm-4.7-flash:medium PASS 100% 57 in → 408 out (395 reasoning)
git clean -n
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 57 in → 291 out (271 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
z-ai/glm-4.7-flash:none PASS 100% 57 in → 5 out (0 reasoning)
git clean -n
z-ai/glm-4.7-flash:none__json_schema PASS 100% 57 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
z-ai/glm-4.7-flash:xhigh PASS 100% 57 in → 341 out (337 reasoning)
git clean -n
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 57 in → 260 out (232 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
z-ai/glm-5.2:high PASS 100% 64 in → 209 out (204 reasoning)
git clean -n
z-ai/glm-5.2:high__json_schema PASS 100% 57 in → 447 out (394 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
z-ai/glm-5.2:low PASS 100% 64 in → 261 out (247 reasoning)
git clean -n
z-ai/glm-5.2:low__json_schema PASS 100% 66 in → 219 out (206 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
z-ai/glm-5.2:medium PASS 100% 64 in → 281 out (276 reasoning)
git clean -n
z-ai/glm-5.2:medium__json_schema PASS 100% 239 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command":"git clean -n"}
z-ai/glm-5.2:none PASS 100% 58 in → 4 out (0 reasoning)
git clean -n
z-ai/glm-5.2:none__json_schema PASS 100% 60 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
z-ai/glm-5.2:xhigh PASS 100% 64 in → 386 out (0 reasoning)
git clean -n
z-ai/glm-5.2:xhigh__json_schema PASS 100% 64 in → 281 out (253 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -fd
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: {"cmd":["bash","-lc","git clean -n"]
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting ',' delimiter: line 1 column 37 (char 36)
Failure: Failed to parse structured JSON response: Expecting ',' delimiter: line 1 column 37 (char 36)
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-2e820d9f-562a-962b-8654-33ec60185d74","request_id":"2e820d9f-562a-962b-8654-33ec60185d74"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9b8bdd4c-6e64-9068-bcc4-dbc266ead9c0","request_id":"9b8bdd4c-6e64-9068-bcc4-dbc266ead9c0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-93bee89c-ee03-9f2e-9674-b809e292af54","request_id":"93bee89c-ee03-9f2e-9674-b809e292af54"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f0ed976d-99b2-9d4b-b426-022a061c94dc","request_id":"f0ed976d-99b2-9d4b-b426-022a061c94dc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a1e156f1-a746-9e9f-9be6-cdd14d8f8e53","request_id":"a1e156f1-a746-9e9f-9be6-cdd14d8f8e53"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-65c49cb1-df59-9e96-947a-a47fc22c6a62"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-adedaacd-0d92-968f-b1d5-fe1596cacdfe"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-9a688032-59a0-9cf0-a402-f9041a8ed709"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-0113f4af-e486-99ed-b181-86b851cf9aac"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-d13e0e5e-85d0-97c2-ac31-fcbca5325993"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ccb95563-d7bf-9010-9e54-611573ab67c1"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7db79f12-63f2-97cb-8b2a-0ed509241394","request_id":"7db79f12-63f2-97cb-8b2a-0ed509241394"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d6b7667d-da49-9d37-89a8-f1848e369000","request_id":"d6b7667d-da49-9d37-89a8-f1848e369000"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9bf226ff-9a3c-9e83-b342-3f31228fdeb2","request_id":"9bf226ff-9a3c-9e83-b342-3f31228fdeb2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e105237c-890c-981f-8e6c-75c6eab8f121","request_id":"e105237c-890c-981f-8e6c-75c6eab8f121"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-97f4ec9a-3627-940e-bc19-7295fa704125","request_id":"97f4ec9a-3627-940e-bc19-7295fa704125"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-54ee2921-d859-9d33-b71d-4db40b84d46e","request_id":"54ee2921-d859-9d33-b71d-4db40b84d46e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d6a25371-1947-92a3-9c11-8ea01bfc2fa0","request_id":"d6a25371-1947-92a3-9c11-8ea01bfc2fa0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c0b40ffb-68fa-9025-b772-6f510661a8d5","request_id":"c0b40ffb-68fa-9025-b772-6f510661a8d5"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4d9e083a-4bb3-94bc-b8db-35f756217024","request_id":"4d9e083a-4bb3-94bc-b8db-35f756217024"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c684ce64-c69f-9093-a518-dd974a982254","request_id":"c684ce64-c69f-9093-a518-dd974a982254"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Model 'tencent/hy3:none' violated the reasoning_level=none invariant: provider reported 413 reasoning token(s)
Invalid JSON. Output: git clean -f -n{"command": "git clean -f -n"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -n
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -n
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)