Dry-run with directory removal preview
Tests ability to preview directory removal with dry-run. Evaluates combined flag usage (-nd).

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'tracked' > tracked.txt
  5. 05 git add tracked.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 mkdir -p artifacts/logs
  8. 08 echo 'log' > artifacts/logs/run.log
  9. 09 echo 'untracked' > scratch.txt
Prompt
Preview what untracked files and directories would be removed without actually removing them. Output ONLY the git command, nothing else.
Expected
git clean -f -d -n

Scoped model quality, cost, API time, and token usage for git_clean/f010.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 101 in → 18 out (23 reasoning)
git clean -n -d
anthropic/claude-fable-5:high__json_schema PASS 100% 311 in → 16 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
anthropic/claude-fable-5:low PASS 100% 101 in → 8 out (0 reasoning)
git clean -nd
anthropic/claude-fable-5:low__json_schema PASS 100% 311 in → 16 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
anthropic/claude-fable-5:max PASS 100% 101 in → 217 out (67 reasoning)
``` git clean -nd ```
anthropic/claude-fable-5:max__json_schema PASS 100% 311 in → 126 out (56 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
anthropic/claude-fable-5:medium PASS 100% 101 in → 16 out (25 reasoning)
git clean -nd
anthropic/claude-fable-5:medium__json_schema PASS 100% 311 in → 16 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
anthropic/claude-fable-5:xhigh PASS 100% 101 in → 26 out (31 reasoning)
git clean -nd
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 311 in → 25 out (23 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
anthropic/claude-haiku-4.5:high PASS 100% 104 in → 202 out (191 reasoning)
``` git clean -nd ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 267 in → 330 out (333 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
anthropic/claude-haiku-4.5:low PASS 100% 104 in → 524 out (518 reasoning)
``` git clean -nd ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 267 in → 218 out (204 reasoning)
git clean -d -n
JSON Schema Structured Output
(raw) {"command": "git clean -d -n"}
anthropic/claude-haiku-4.5:medium PASS 100% 104 in → 401 out (386 reasoning)
``` git clean -n -d ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 267 in → 156 out (152 reasoning)
git clean -d -n
JSON Schema Structured Output
(raw) {"command": "git clean -d -n"}
anthropic/claude-haiku-4.5:none PASS 100% 75 in → 12 out (0 reasoning)
``` git clean -dn ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 237 in → 13 out (0 reasoning)
git clean -dn
JSON Schema Structured Output
(raw) {"command": "git clean -dn"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 104 in → 165 out (148 reasoning)
``` git clean -dn ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 267 in → 190 out (176 reasoning)
git clean -d -n
JSON Schema Structured Output
(raw) {"command": "git clean -d -n"}
anthropic/claude-opus-4.6:high PASS 100% 75 in → 14 out (0 reasoning)
``` git clean -d -n ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 238 in → 73 out (45 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.6:low PASS 100% 75 in → 9 out (0 reasoning)
git clean -d -n
anthropic/claude-opus-4.6:low__json_schema PASS 100% 238 in → 13 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.6:max PASS 100% 75 in → 91 out (70 reasoning)
``` git clean -dn ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 238 in → 115 out (85 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.6:medium PASS 100% 75 in → 14 out (0 reasoning)
``` git clean -d -n ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 238 in → 13 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.6:none PASS 100% 75 in → 13 out (0 reasoning)
``` git clean -d -n ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 238 in → 13 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.6:xhigh PASS 100% 75 in → 14 out (0 reasoning)
``` git clean -d -n ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 238 in → 73 out (45 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.7:high PASS 100% 106 in → 11 out (0 reasoning)
git clean -n -d
anthropic/claude-opus-4.7:high__json_schema PASS 100% 316 in → 17 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-opus-4.7:low PASS 100% 106 in → 10 out (0 reasoning)
git clean -nd
anthropic/claude-opus-4.7:low__json_schema PASS 100% 316 in → 18 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.7:max PASS 100% 106 in → 16 out (0 reasoning)
``` git clean -n -d ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 316 in → 18 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.7:medium PASS 100% 106 in → 11 out (0 reasoning)
git clean -n -d
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 316 in → 17 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-opus-4.7:none PASS 100% 106 in → 15 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-opus-4.7:none__json_schema PASS 100% 316 in → 18 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.7:xhigh PASS 100% 106 in → 13 out (0 reasoning)
`git clean -n -d`
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 316 in → 17 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-opus-4.8:high PASS 100% 101 in → 7 out (0 reasoning)
git clean -n
anthropic/claude-opus-4.8:high__json_schema PASS 100% 311 in → 16 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.8:low PASS 100% 101 in → 9 out (0 reasoning)
git clean -n -d
anthropic/claude-opus-4.8:low__json_schema PASS 100% 311 in → 15 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-opus-4.8:max PASS 100% 101 in → 8 out (0 reasoning)
git clean -nd
anthropic/claude-opus-4.8:max__json_schema PASS 100% 311 in → 116 out (54 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-opus-4.8:medium PASS 100% 101 in → 11 out (0 reasoning)
`git clean -n -d`
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 311 in → 16 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.8:none PASS 100% 101 in → 12 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 311 in → 15 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-opus-4.8:xhigh PASS 100% 101 in → 9 out (0 reasoning)
git clean -n -d
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 311 in → 16 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
anthropic/claude-sonnet-4.6:high PASS 100% 75 in → 121 out (106 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 238 in → 90 out (68 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-4.6:low PASS 100% 75 in → 11 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 238 in → 11 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-4.6:max PASS 100% 75 in → 135 out (123 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 238 in → 226 out (182 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-4.6:medium PASS 100% 75 in → 12 out (0 reasoning)
``` git clean -nfd ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 238 in → 11 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-4.6:none PASS 100% 75 in → 11 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 238 in → 11 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 75 in → 221 out (193 reasoning)
``` git clean -nfd ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 238 in → 139 out (112 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-5:high PASS 100% 101 in → 12 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 311 in → 15 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
anthropic/claude-sonnet-5:low PASS 100% 101 in → 8 out (0 reasoning)
git clean -nd
anthropic/claude-sonnet-5:low__json_schema PASS 100% 311 in → 15 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-5:medium PASS 100% 101 in → 12 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 311 in → 15 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
anthropic/claude-sonnet-5:none PASS 100% 101 in → 9 out (0 reasoning)
git clean -n -d
anthropic/claude-sonnet-5:none__json_schema PASS 100% 311 in → 14 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
anthropic/claude-sonnet-5:xhigh PASS 100% 101 in → 12 out (0 reasoning)
``` git clean -nd ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 311 in → 15 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
arcee-ai/trinity-large-thinking:high PASS 100% 61 in → 503 out (506 reasoning)
git clean -fdn
arcee-ai/trinity-large-thinking:low PASS 100% 61 in → 639 out (633 reasoning)
git clean -d -n
arcee-ai/trinity-large-thinking:medium PASS 100% 61 in → 813 out (820 reasoning)
git clean -nd
arcee-ai/trinity-large-thinking:xhigh PASS 100% 61 in → 742 out (742 reasoning)
git clean -d -n
arcee-ai/trinity-mini:high PASS 100% 61 in → 327 out (379 reasoning)
git clean -n
arcee-ai/trinity-mini:high__json_schema PASS 100% 61 in → 306 out (350 reasoning)
git clean-n
JSON Schema Structured Output
(raw) { "command": "git clean-n" }
arcee-ai/trinity-mini:low PASS 100% 61 in → 351 out (406 reasoning)
git clean -n
arcee-ai/trinity-mini:low__json_schema PASS 100% 61 in → 281 out (313 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
arcee-ai/trinity-mini:medium PASS 100% 61 in → 536 out (596 reasoning)
git clean -n
arcee-ai/trinity-mini:medium__json_schema PASS 100% 61 in → 533 out (579 reasoning)
git clean -fdn
JSON Schema Structured Output
(raw) { "command": "git clean -fdn" }
arcee-ai/trinity-mini:xhigh PASS 100% 61 in → 258 out (302 reasoning)
git clean -n
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 61 in → 281 out (312 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 63 in → 347 out (336 reasoning)
git clean -nd
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 63 in → 566 out (542 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 63 in → 74 out (78 reasoning)
git clean -n
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 63 in → 163 out (152 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-flash-0731:max PASS 100% 142 in → 138 out (131 reasoning)
git clean -nd
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 142 in → 196 out (176 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 63 in → 269 out (262 reasoning)
git clean -n -d
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 69 in → 348 out (360 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
deepseek/deepseek-v4-flash-0731:none PASS 100% 63 in → 5 out (0 reasoning)
git clean -nd
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 69 in → 14 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 142 in → 94 out (89 reasoning)
git clean -nd
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 63 in → 295 out (273 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
deepseek/deepseek-v4-flash:high PASS 100% 63 in → 244 out (238 reasoning)
git clean -dn
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 65 in → 348 out (334 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash:low PASS 100% 63 in → 259 out (251 reasoning)
git clean -n -d
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 155 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
deepseek/deepseek-v4-flash:medium PASS 100% 63 in → 279 out (275 reasoning)
git clean -dn
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 65 in → 460 out (445 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-flash:none PASS 100% 63 in → 4 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 63 in → 13 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 142 in → 854 out (848 reasoning)
git clean -nd
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 144 in → 550 out (534 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-pro:high PASS 100% 63 in → 5 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 63 in → 13 out (0 reasoning)
git clean -fdn
JSON Schema Structured Output
(raw) { "command": "git clean -fdn" }
deepseek/deepseek-v4-pro:low PASS 100% 63 in → 5 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 250 in → 83 out (72 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
deepseek/deepseek-v4-pro:medium PASS 100% 63 in → 214 out (206 reasoning)
git clean -n -d
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 63 in → 12 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
deepseek/deepseek-v4-pro:none PASS 100% 63 in → 4 out (0 reasoning)
git clean -n
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 65 in → 14 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 142 in → 262 out (254 reasoning)
git clean -n -d
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 328 in → 788 out (775 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
google/gemini-3-flash-preview:high PASS 100% 64 in → 181 out (177 reasoning)
git clean -nd
google/gemini-3-flash-preview:high__json_schema PASS 100% 65 in → 271 out (263 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3-flash-preview:low PASS 100% 64 in → 140 out (136 reasoning)
git clean -nd
google/gemini-3-flash-preview:low__json_schema PASS 100% 65 in → 186 out (177 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3-flash-preview:medium PASS 100% 64 in → 171 out (167 reasoning)
git clean -nd
google/gemini-3-flash-preview:medium__json_schema PASS 100% 121 in → 169 out (160 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3-flash-preview:xhigh PASS 100% 64 in → 184 out (180 reasoning)
git clean -nd
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 121 in → 171 out (162 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 65 in → 190 out (186 reasoning)
git clean -nd
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 121 in → 530 out (522 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 65 in → 134 out (130 reasoning)
git clean -nd
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 65 in → 303 out (294 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 64 in → 141 out (137 reasoning)
git clean -nd
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 121 in → 158 out (149 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 64 in → 339 out (335 reasoning)
git clean -nd
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 121 in → 217 out (208 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.1-pro-preview:high PASS 100% 64 in → 212 out (206 reasoning)
git clean -n -d
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 121 in → 195 out (184 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
google/gemini-3.1-pro-preview:low PASS 100% 64 in → 175 out (171 reasoning)
git clean -nd
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 121 in → 92 out (83 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.1-pro-preview:medium PASS 100% 64 in → 269 out (265 reasoning)
git clean -nd
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 121 in → 322 out (312 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 64 in → 299 out (295 reasoning)
git clean -nd
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 65 in → 266 out (255 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
google/gemini-3.5-flash-lite:high PASS 100% 64 in → 272 out (268 reasoning)
git clean -nd
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 121 in → 340 out (331 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash-lite:low PASS 100% 65 in → 4 out (0 reasoning)
git clean -nd
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 65 in → 9 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash-lite:max PASS 100% 64 in → 225 out (221 reasoning)
git clean -nd
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 121 in → 352 out (343 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash-lite:medium PASS 100% 65 in → 156 out (152 reasoning)
git clean -nd
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 65 in → 184 out (175 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 65 in → 299 out (295 reasoning)
git clean -nd
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 65 in → 385 out (377 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3.5-flash:high PASS 100% 64 in → 530 out (526 reasoning)
git clean -nd
google/gemini-3.5-flash:high__json_schema PASS 100% 121 in → 520 out (511 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash:low PASS 100% 64 in → 326 out (322 reasoning)
git clean -nd
google/gemini-3.5-flash:low__json_schema PASS 100% 121 in → 80 out (71 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.5-flash:medium PASS 100% 64 in → 315 out (311 reasoning)
git clean -nd
google/gemini-3.5-flash:medium__json_schema PASS 100% 121 in → 402 out (386 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
google/gemini-3.5-flash:xhigh PASS 100% 64 in → 584 out (580 reasoning)
git clean -nd
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 121 in → 326 out (317 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemini-3.6-flash:high PASS 100% 65 in → 574 out (570 reasoning)
git clean -nd
google/gemini-3.6-flash:high__json_schema PASS 100% 65 in → 275 out (267 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3.6-flash:low PASS 100% 65 in → 214 out (210 reasoning)
git clean -nd
google/gemini-3.6-flash:low__json_schema PASS 100% 121 in → 14 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
google/gemini-3.6-flash:max PASS 100% 65 in → 464 out (460 reasoning)
git clean -nd
google/gemini-3.6-flash:max__json_schema PASS 100% 121 in → 520 out (512 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3.6-flash:medium PASS 100% 65 in → 397 out (393 reasoning)
git clean -nd
google/gemini-3.6-flash:medium__json_schema PASS 100% 65 in → 329 out (321 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemini-3.6-flash:xhigh PASS 100% 64 in → 745 out (741 reasoning)
git clean -nd
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 65 in → 341 out (333 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemma-4-26b-a4b-it:high PASS 100% 79 in → 252 out (229 reasoning)
git clean -nd
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 80 in → 287 out (257 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
google/gemma-4-26b-a4b-it:low PASS 100% 80 in → 216 out (190 reasoning)
git clean -nd
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 79 in → 278 out (244 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemma-4-26b-a4b-it:medium PASS 100% 80 in → 262 out (237 reasoning)
git clean -nd
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 80 in → 306 out (269 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemma-4-26b-a4b-it:none PASS 100% 77 in → 5 out (0 reasoning)
git clean -nd
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 77 in → 15 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 80 in → 250 out (245 reasoning)
git clean -nd
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 80 in → 224 out (188 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
google/gemma-4-31b-it:high PASS 100% 79 in → 164 out (146 reasoning)
git clean -nd
google/gemma-4-31b-it:high__json_schema PASS 100% 80 in → 159 out (136 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
google/gemma-4-31b-it:low PASS 100% 80 in → 184 out (160 reasoning)
git clean -nd
google/gemma-4-31b-it:low__json_schema PASS 100% 83 in → 178 out (168 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemma-4-31b-it:medium PASS 100% 80 in → 226 out (204 reasoning)
git clean -nd
google/gemma-4-31b-it:medium__json_schema PASS 100% 83 in → 152 out (142 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
google/gemma-4-31b-it:none PASS 100% 77 in → 5 out (0 reasoning)
git clean -nd
google/gemma-4-31b-it:none__json_schema PASS 100% 77 in → 10 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
google/gemma-4-31b-it:xhigh PASS 100% 80 in → 116 out (1 reasoning)
git clean -nd
ibm-granite/granite-4.1-8b PASS 100% 74 in → 10 out
git status --ignored --untracked-files=no
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
inclusionai/ling-2.6-flash PASS 100% 80 in → 10 out
```sh git clean -n ```
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
liquid/lfm-2-24b-a2b PASS 100% 72 in → 36 out
git ls-tree -r --name-only --diff-filter=D --exclude="artifacts/logs/run.log" --exclude="scratch.txt"
minimax/minimax-m2.5:high PASS 100% 92 in → 304 out (298 reasoning)
git clean -nd
minimax/minimax-m2.5:high__json_schema PASS 100% 92 in → 151 out (135 reasoning)
git clean -fd -n
JSON Schema Structured Output
(raw) { "command": "git clean -fd -n" }
minimax/minimax-m2.5:low PASS 100% 92 in → 171 out (163 reasoning)
git clean -n -d
minimax/minimax-m2.5:low__json_schema PASS 100% 92 in → 109 out (113 reasoning)
git clean -nfd
JSON Schema Structured Output
(raw) { "command": "git clean -nfd" }
minimax/minimax-m2.5:medium PASS 100% 105 in → 113 out (105 reasoning)
git clean -n -d
minimax/minimax-m2.5:medium__json_schema PASS 100% 92 in → 227 out (213 reasoning)
git clean -nfd
JSON Schema Structured Output
(raw) { "command": "git clean -nfd" }
minimax/minimax-m2.5:xhigh PASS 100% 92 in → 109 out (117 reasoning)
git clean -n -d
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 105 in → 247 out (258 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
minimax/minimax-m2.7:high PASS 100% 95 in → 628 out (621 reasoning)
git clean -n -d
minimax/minimax-m2.7:low PASS 100% 91 in → 190 out (211 reasoning)
git clean -nd
minimax/minimax-m2.7:low__json_schema PASS 100% 224 in → 387 out (377 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
minimax/minimax-m2.7:medium PASS 100% 92 in → 333 out (325 reasoning)
git clean -n -d
minimax/minimax-m2.7:medium__json_schema PASS 100% 224 in → 559 out (547 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
minimax/minimax-m2.7:xhigh PASS 100% 92 in → 312 out (343 reasoning)
git clean -n -d
minimax/minimax-m3:high PASS 100% 230 in → 88 out (90 reasoning)
git clean -nd
minimax/minimax-m3:high__json_schema PASS 100% 217 in → 15 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
minimax/minimax-m3:low PASS 100% 230 in → 229 out (258 reasoning)
`git clean -nd`
minimax/minimax-m3:low__json_schema PASS 100% 217 in → 12 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
minimax/minimax-m3:medium PASS 100% 230 in → 57 out (56 reasoning)
`git clean -nd`
minimax/minimax-m3:medium__json_schema PASS 100% 217 in → 13 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
minimax/minimax-m3:xhigh PASS 100% 230 in → 68 out (68 reasoning)
`git clean -nd`
minimax/minimax-m3:xhigh__json_schema PASS 100% 217 in → 12 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
mistralai/devstral-2512 PASS 100% 60 in → 10 out
```bash git clean -n ```
mistralai/devstral-2512__json_schema PASS 100% 60 in → 10 out
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:high PASS 100% 72 in → 598 out (628 reasoning)
git clean -n -d
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 60 in → 271 out (281 reasoning)
git clean -dn
JSON Schema Structured Output
(raw) {"command": "git clean -dn"}
mistralai/mistral-medium-3-5:low PASS 100% 72 in → 495 out (550 reasoning)
git clean -dn
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 60 in → 301 out (319 reasoning)
git clean -dn
JSON Schema Structured Output
(raw) {"command": "git clean -dn"}
mistralai/mistral-medium-3-5:medium PASS 100% 72 in → 574 out (640 reasoning)
git clean -nd
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 60 in → 334 out (351 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
mistralai/mistral-medium-3-5:none PASS 100% 72 in → 5 out (0 reasoning)
git clean -n
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 60 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 72 in → 344 out (368 reasoning)
git clean -n -d
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 60 in → 1,213 out (1,185 reasoning)
git clean -fd --dry-run
JSON Schema Structured Output
(raw) {"command": "git clean -fd --dry-run"}
moonshotai/kimi-k2.6:high PASS 100% 61 in → 339 out (331 reasoning)
git clean -n -d
moonshotai/kimi-k2.6:low PASS 100% 61 in → 653 out (672 reasoning)
git clean -n -d
moonshotai/kimi-k2.6:low__json_schema PASS 100% 60 in → 770 out (500 reasoning)
git clean -dn
JSON Schema Structured Output
(raw) { "command": "git clean -dn" }
moonshotai/kimi-k2.6:medium PASS 100% 61 in → 1,097 out (1,092 reasoning)
git clean -dn
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 60 in → 555 out (538 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
moonshotai/kimi-k2.6:xhigh PASS 100% 60 in → 290 out (284 reasoning)
git clean -dn
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 61 in → 13 out (0 reasoning)
git clean -dn
JSON Schema Structured Output
(raw) { "command": "git clean -dn" }
moonshotai/kimi-k2.7-code:high PASS 100% 61 in → 117 out (133 reasoning)
git clean -nd
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 61 in → 118 out (103 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
moonshotai/kimi-k2.7-code:low PASS 100% 61 in → 273 out (283 reasoning)
git clean -dn
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 61 in → 128 out (112 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
moonshotai/kimi-k2.7-code:medium PASS 100% 61 in → 31 out (25 reasoning)
git clean -dn
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 61 in → 237 out (222 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 61 in → 99 out (91 reasoning)
git clean -n -d
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 144 in → 187 out (174 reasoning)
git clean -fd -n
JSON Schema Structured Output
(raw) {"command": "git clean -fd -n"}
moonshotai/kimi-k3:high PASS 100% 138 in → 262 out (243 reasoning)
git clean -nd
moonshotai/kimi-k3:high__json_schema PASS 100% 138 in → 181 out (162 reasoning)
git clean -nd
moonshotai/kimi-k3:low PASS 100% 138 in → 121 out (102 reasoning)
git clean -nd
moonshotai/kimi-k3:low__json_schema PASS 100% 138 in → 104 out (85 reasoning)
git clean -nd
moonshotai/kimi-k3:max PASS 100% 138 in → 206 out (187 reasoning)
git clean -nd
moonshotai/kimi-k3:max__json_schema PASS 100% 138 in → 165 out (141 reasoning)
```bash git clean -nd ```
moonshotai/kimi-k3:medium PASS 100% 138 in → 172 out (153 reasoning)
git clean -nd
moonshotai/kimi-k3:medium__json_schema PASS 100% 138 in → 196 out (177 reasoning)
git clean -nd
moonshotai/kimi-k3:xhigh PASS 100% 138 in → 166 out (147 reasoning)
git clean -nd
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 138 in → 134 out (115 reasoning)
git clean -nd
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 73 in → 826 out (804 reasoning)
git clean -n -fdx .
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 73 in → 363 out (366 reasoning)
git clean -n -d .
JSON Schema Structured Output
(raw) { "command": "git clean -n -d ." }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 73 in → 195 out (214 reasoning)
git clean -nfd
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 73 in → 143 out (157 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 73 in → 377 out (356 reasoning)
git clean -n -d
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 73 in → 590 out (622 reasoning)
git clean -nx
JSON Schema Structured Output
(raw) { "command": "git clean -nx" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 73 in → 6 out (0 reasoning)
git clean -nxd
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 73 in → 30 out (0 reasoning)
git clean --dry-run --exclude='artifacts/logs/run.log' --exclude='scratch.txt'
JSON Schema Structured Output
(raw) { "command": "git clean --dry-run --exclude='artifacts/logs/run.log' --exclude='scratch.txt'" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 73 in → 729 out (790 reasoning)
gitclean -n
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 73 in → 229 out (248 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 73 in → 120 out (108 reasoning)
git clean -n -d
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 73 in → 119 out (102 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 73 in → 110 out (97 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 73 in → 121 out (101 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 73 in → 81 out (71 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 73 in → 189 out (171 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 73 in → 7 out (0 reasoning)
git clean -n -d
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 73 in → 13 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 73 in → 105 out (105 reasoning)
git clean -n
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 73 in → 163 out (144 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
openai/gpt-5.4-mini:high PASS 100% 59 in → 207 out (197 reasoning)
git clean -nd
openai/gpt-5.4-mini:high__json_schema PASS 100% 94 in → 159 out (141 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-mini:low PASS 100% 59 in → 36 out (25 reasoning)
git clean -fdn
openai/gpt-5.4-mini:low__json_schema PASS 100% 94 in → 96 out (78 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-mini:medium PASS 100% 59 in → 80 out (70 reasoning)
git clean -nd
openai/gpt-5.4-mini:medium__json_schema PASS 100% 94 in → 235 out (217 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-mini:none PASS 100% 59 in → 9 out (0 reasoning)
git clean -nd
openai/gpt-5.4-mini:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-mini:xhigh PASS 100% 59 in → 133 out (123 reasoning)
git clean -nd
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 94 in → 167 out (149 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:high PASS 100% 59 in → 9 out (0 reasoning)
git clean -ndX
openai/gpt-5.4-nano:high__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:low PASS 100% 59 in → 9 out (0 reasoning)
git clean -ndX
openai/gpt-5.4-nano:low__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:medium PASS 100% 59 in → 15 out (0 reasoning)
git clean -nd -x -- artifacts/ scratch.txt
openai/gpt-5.4-nano:medium__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:none PASS 100% 59 in → 14 out (0 reasoning)
git clean -nd --force artifacts/ scratch.txt
openai/gpt-5.4-nano:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4-nano:xhigh PASS 100% 59 in → 13 out (0 reasoning)
git clean -ndfX artifacts scratch.txt
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:high PASS 100% 59 in → 126 out (116 reasoning)
git clean -nd
openai/gpt-5.4:high__json_schema PASS 100% 94 in → 117 out (99 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:low PASS 100% 59 in → 50 out (40 reasoning)
git clean -nd
openai/gpt-5.4:low__json_schema PASS 100% 94 in → 48 out (30 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:medium PASS 100% 59 in → 137 out (127 reasoning)
git clean -nd
openai/gpt-5.4:medium__json_schema PASS 100% 94 in → 137 out (117 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
openai/gpt-5.4:none PASS 100% 59 in → 9 out (0 reasoning)
git clean -nd
openai/gpt-5.4:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.4:xhigh PASS 100% 59 in → 292 out (282 reasoning)
git clean -nd
openai/gpt-5.4:xhigh__json_schema PASS 100% 94 in → 271 out (253 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.5:high PASS 100% 59 in → 67 out (57 reasoning)
git clean -nd
openai/gpt-5.5:high__json_schema PASS 100% 94 in → 121 out (102 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
openai/gpt-5.5:low PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.5:low__json_schema PASS 100% 94 in → 17 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
openai/gpt-5.5:medium PASS 100% 59 in → 70 out (60 reasoning)
git clean -nd
openai/gpt-5.5:medium__json_schema PASS 100% 94 in → 17 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
openai/gpt-5.5:none PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.5:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.5:xhigh PASS 100% 59 in → 110 out (100 reasoning)
git clean -nd
openai/gpt-5.5:xhigh__json_schema PASS 100% 94 in → 114 out (96 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-luna:high PASS 100% 59 in → 61 out (51 reasoning)
git clean -nd
openai/gpt-5.6-luna:high__json_schema PASS 100% 94 in → 86 out (68 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-luna:low PASS 100% 59 in → 51 out (40 reasoning)
git clean -nd
openai/gpt-5.6-luna:low__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-luna:max PASS 100% 59 in → 134 out (124 reasoning)
git clean -nd
openai/gpt-5.6-luna:max__json_schema PASS 100% 94 in → 324 out (304 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
openai/gpt-5.6-luna:medium PASS 100% 59 in → 77 out (67 reasoning)
git clean -nd
openai/gpt-5.6-luna:medium__json_schema PASS 100% 94 in → 48 out (28 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
openai/gpt-5.6-luna:none PASS 100% 59 in → 9 out (0 reasoning)
git clean -ndarsuup
openai/gpt-5.6-luna:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-luna:xhigh PASS 100% 59 in → 88 out (78 reasoning)
git clean -nd
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 94 in → 60 out (40 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
openai/gpt-5.6-sol:high PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-sol:high__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-sol:low PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-sol:low__json_schema PASS 100% 94 in → 18 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
openai/gpt-5.6-sol:max PASS 100% 59 in → 104 out (94 reasoning)
git clean -nd
openai/gpt-5.6-sol:max__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-sol:medium PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-sol:medium__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-sol:none PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-sol:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-sol:xhigh PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:high PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-terra:high__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:low PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-terra:low__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:max PASS 100% 59 in → 92 out (82 reasoning)
git clean -nd
openai/gpt-5.6-terra:max__json_schema PASS 100% 94 in → 79 out (61 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:medium PASS 100% 59 in → 33 out (23 reasoning)
git clean -nd
openai/gpt-5.6-terra:medium__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:none PASS 100% 59 in → 8 out (0 reasoning)
git clean -nd
openai/gpt-5.6-terra:none__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-5.6-terra:xhigh PASS 100% 59 in → 65 out (55 reasoning)
git clean -nd
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 94 in → 16 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-oss-120b:high PASS 100% 107 in → 450 out (455 reasoning)
git clean -fdn
openai/gpt-oss-120b:high__json_schema PASS 100% 120 in → 5 out (0 reasoning)
analysis
JSON Schema Structured Output
(raw) {"command":"analysis"}
openai/gpt-oss-120b:low PASS 100% 118 in → 52 out (39 reasoning)
git clean -n -d
openai/gpt-oss-120b:low__json_schema PASS 100% 120 in → 43 out (25 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-oss-120b:medium PASS 100% 118 in → 252 out (262 reasoning)
git clean -nd
openai/gpt-oss-120b:medium__json_schema PASS 100% 118 in → 154 out (144 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-oss-120b:xhigh PASS 100% 107 in → 336 out (345 reasoning)
git clean -n -d
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 107 in → 354 out (363 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-oss-20b:high PASS 100% 118 in → 351 out (336 reasoning)
git clean -n -d
openai/gpt-oss-20b:high__json_schema PASS 100% 118 in → 191 out (190 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
openai/gpt-oss-20b:low PASS 100% 120 in → 241 out (235 reasoning)
git clean -n -d
openai/gpt-oss-20b:low__json_schema PASS 100% 118 in → 50 out (31 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
openai/gpt-oss-20b:medium PASS 100% 122 in → 337 out (368 reasoning)
git clean -n -d
openai/gpt-oss-20b:medium__json_schema PASS 100% 107 in → 235 out (229 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
openai/gpt-oss-20b:xhigh PASS 100% 120 in → 473 out (448 reasoning)
git clean -n -d
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 120 in → 514 out (492 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
poolside/laguna-m.1:high PASS 100% 72 in → 427 out (420 reasoning)
git clean -nd
poolside/laguna-m.1:high__json_schema PASS 100% 72 in → 526 out (515 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
poolside/laguna-m.1:low PASS 100% 72 in → 441 out (434 reasoning)
git clean -dn
poolside/laguna-m.1:low__json_schema PASS 100% 72 in → 286 out (271 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
poolside/laguna-m.1:medium PASS 100% 72 in → 302 out (295 reasoning)
git clean -nd
poolside/laguna-m.1:medium__json_schema PASS 100% 72 in → 388 out (372 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
poolside/laguna-m.1:none PASS 100% 72 in → 6 out (0 reasoning)
git clean -n
poolside/laguna-m.1:none__json_schema PASS 100% 72 in → 10 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
poolside/laguna-m.1:xhigh PASS 100% 72 in → 262 out (255 reasoning)
git clean -nd
poolside/laguna-m.1:xhigh__json_schema PASS 100% 72 in → 251 out (236 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
poolside/laguna-xs-2.1:high PASS 100% 72 in → 290 out (283 reasoning)
git clean -n -d
poolside/laguna-xs-2.1:high__json_schema PASS 100% 72 in → 267 out (249 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
poolside/laguna-xs-2.1:low PASS 100% 72 in → 408 out (403 reasoning)
git clean -nd
poolside/laguna-xs-2.1:low__json_schema PASS 100% 72 in → 416 out (405 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
poolside/laguna-xs-2.1:medium PASS 100% 72 in → 284 out (277 reasoning)
git clean -n -f
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 72 in → 1,996 out (1,983 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
poolside/laguna-xs-2.1:none PASS 100% 72 in → 5 out (0 reasoning)
git clean -nfd
poolside/laguna-xs-2.1:none__json_schema PASS 100% 72 in → 11 out (0 reasoning)
git clean -nfd
JSON Schema Structured Output
(raw) {"command": "git clean -nfd"}
poolside/laguna-xs-2.1:xhigh PASS 100% 72 in → 328 out (321 reasoning)
git clean -n -d
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 72 in → 237 out (224 reasoning)
git clean -n -f
JSON Schema Structured Output
(raw) {"command": "git clean -n -f"}
poolside/laguna-xs.2:high PASS 100% 109 in → 146 out (139 reasoning)
git clean -n
poolside/laguna-xs.2:high__json_schema PASS 100% 109 in → 166 out (155 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
poolside/laguna-xs.2:low PASS 100% 109 in → 188 out (179 reasoning)
git clean -n -d
poolside/laguna-xs.2:low__json_schema PASS 100% 109 in → 132 out (116 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-xs.2:medium PASS 100% 109 in → 247 out (240 reasoning)
git clean -nd
poolside/laguna-xs.2:medium__json_schema PASS 100% 109 in → 130 out (114 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
poolside/laguna-xs.2:none PASS 100% 109 in → 6 out (0 reasoning)
git clean -dn
poolside/laguna-xs.2:none__json_schema PASS 100% 109 in → 10 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
poolside/laguna-xs.2:xhigh PASS 100% 109 in → 145 out (136 reasoning)
git clean -n -d
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 109 in → 129 out (116 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command": "git clean -n -d"}
qwen/qwen3.6-27b:high PASS 100% 68 in → 907 out (898 reasoning)
git clean -nd
qwen/qwen3.6-27b:low PASS 100% 68 in → 763 out (744 reasoning)
git clean -nd
qwen/qwen3.6-27b:low__json_schema PASS 100% 68 in → 725 out (708 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
qwen/qwen3.6-27b:medium PASS 100% 68 in → 926 out (905 reasoning)
git clean -nd
qwen/qwen3.6-27b:medium__json_schema PASS 100% 68 in → 993 out (928 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
qwen/qwen3.6-27b:none PASS 100% 70 in → 5 out (0 reasoning)
git clean -n
qwen/qwen3.6-27b:none__json_schema PASS 100% 70 in → 16 out (0 reasoning)
git clean -fdn
JSON Schema Structured Output
(raw) { "command": "git clean -fdn" }
qwen/qwen3.6-27b:xhigh PASS 100% 68 in → 960 out (937 reasoning)
git clean -nd
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 68 in → 1,071 out (1,035 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
qwen/qwen3.6-35b-a3b:high PASS 100% 68 in → 769 out (740 reasoning)
git clean -nd
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 68 in → 1,500 out (1,408 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-35b-a3b:low PASS 100% 68 in → 1,009 out (933 reasoning)
git clean -nd
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 68 in → 1,555 out (1,446 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 68 in → 1,626 out (1,547 reasoning)
git clean -n
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 68 in → 1,462 out (1,356 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
qwen/qwen3.6-35b-a3b:none PASS 100% 70 in → 5 out (0 reasoning)
git clean --dry-run
qwen/qwen3.6-35b-a3b:none__json_schema PASS 100% 70 in → 15 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 68 in → 758 out (727 reasoning)
git clean -nd
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 68 in → 1,270 out (1,178 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
qwen/qwen3.6-flash:high PASS 100% 68 in → 861 out (852 reasoning)
git clean -dn
qwen/qwen3.6-flash:low PASS 100% 68 in → 1,634 out (1,625 reasoning)
git clean -nd
qwen/qwen3.6-flash:medium PASS 100% 68 in → 947 out (938 reasoning)
git clean -nd
qwen/qwen3.6-flash:none PASS 100% 70 in → 5 out (0 reasoning)
git clean --dry-run
qwen/qwen3.6-flash:xhigh PASS 100% 68 in → 394 out (385 reasoning)
git clean -dn
qwen/qwen3.7-flash:high PASS 100% 68 in → 988 out (979 reasoning)
git clean -nd
qwen/qwen3.7-flash:low PASS 100% 68 in → 827 out (818 reasoning)
git clean -nd
qwen/qwen3.7-flash:max PASS 100% 68 in → 597 out (588 reasoning)
git clean -n
qwen/qwen3.7-flash:medium PASS 100% 68 in → 1,193 out (1,184 reasoning)
git clean -dn
qwen/qwen3.7-flash:none PASS 100% 70 in → 5 out (0 reasoning)
git clean --dry-run
qwen/qwen3.7-flash:xhigh PASS 100% 68 in → 1,377 out (1,368 reasoning)
git clean -n
qwen/qwen3.7-max:high PASS 100% 68 in → 616 out (607 reasoning)
git clean -nd
qwen/qwen3.7-max:low PASS 100% 68 in → 624 out (615 reasoning)
git clean -nd
qwen/qwen3.7-max:medium PASS 100% 68 in → 626 out (617 reasoning)
git clean -nd
qwen/qwen3.7-max:none PASS 100% 70 in → 4 out (0 reasoning)
git clean -dn
qwen/qwen3.7-max:xhigh PASS 100% 68 in → 207 out (198 reasoning)
git clean -nd
qwen/qwen3.7-plus:high PASS 100% 68 in → 502 out (493 reasoning)
git clean -n
qwen/qwen3.7-plus:low PASS 100% 68 in → 536 out (527 reasoning)
git clean -nd
qwen/qwen3.7-plus:medium PASS 100% 68 in → 175 out (166 reasoning)
git clean -nd
qwen/qwen3.7-plus:none PASS 100% 70 in → 4 out (0 reasoning)
git clean -n
qwen/qwen3.7-plus:xhigh PASS 100% 68 in → 890 out (879 reasoning)
git clean -n -d
tencent/hy3:high PASS 100% 68 in → 966 out (960 reasoning)
git clean -nd
tencent/hy3:high__json_schema PASS 100% 68 in → 575 out (540 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
tencent/hy3:low PASS 100% 68 in → 177 out (170 reasoning)
git clean -fdn
tencent/hy3:low__json_schema PASS 100% 68 in → 667 out (633 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
tencent/hy3:medium PASS 100% 68 in → 367 out (361 reasoning)
git clean -nd
tencent/hy3:medium__json_schema PASS 100% 68 in → 476 out (428 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
tencent/hy3:none PASS 100% 71 in → 6 out (0 reasoning)
git clean -ndx
tencent/hy3:none__json_schema PASS 100% 71 in → 12 out (0 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
tencent/hy3:xhigh PASS 100% 68 in → 660 out (654 reasoning)
git clean -nd
tencent/hy3:xhigh__json_schema PASS 100% 68 in → 721 out (679 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
thinkingmachines/inkling-small:high PASS 100% 68 in → 195 out (183 reasoning)
git clean -n -d
thinkingmachines/inkling-small:high__json_schema PASS 100% 68 in → 118 out (101 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
thinkingmachines/inkling-small:low PASS 100% 68 in → 102 out (111 reasoning)
git clean -dn
thinkingmachines/inkling-small:low__json_schema PASS 100% 68 in → 70 out (53 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
thinkingmachines/inkling-small:max PASS 100% 68 in → 145 out (149 reasoning)
git clean -nd
thinkingmachines/inkling-small:max__json_schema PASS 100% 68 in → 202 out (186 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
thinkingmachines/inkling-small:medium PASS 100% 68 in → 92 out (80 reasoning)
git clean -n -d
thinkingmachines/inkling-small:medium__json_schema PASS 100% 68 in → 194 out (175 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
thinkingmachines/inkling-small:none PASS 100% 68 in → 7 out (0 reasoning)
git clean -dn<|end_message|>
thinkingmachines/inkling-small:xhigh PASS 100% 68 in → 155 out (170 reasoning)
git clean -nd
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 68 in → 157 out (138 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
thinkingmachines/inkling:high PASS 100% 68 in → 184 out (173 reasoning)
git clean -nd
thinkingmachines/inkling:high__json_schema PASS 100% 68 in → 136 out (119 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
thinkingmachines/inkling:low PASS 100% 68 in → 58 out (47 reasoning)
git clean -n
thinkingmachines/inkling:max PASS 100% 68 in → 163 out (181 reasoning)
git clean -nd
thinkingmachines/inkling:max__json_schema PASS 100% 68 in → 226 out (210 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
thinkingmachines/inkling:medium PASS 100% 68 in → 125 out (135 reasoning)
git clean -nd
thinkingmachines/inkling:none PASS 100% 66 in → 8 out (0 reasoning)
git clean -nd
thinkingmachines/inkling:xhigh PASS 100% 68 in → 184 out (173 reasoning)
git clean -nd
x-ai/grok-4.3:high PASS 100% 237 in → 244 out (236 reasoning)
`git clean -n -d`
x-ai/grok-4.3:high__json_schema PASS 100% 293 in → 670 out (656 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
x-ai/grok-4.3:low PASS 100% 243 in → 316 out (310 reasoning)
`git clean -n`
x-ai/grok-4.3:low__json_schema PASS 100% 299 in → 427 out (419 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
x-ai/grok-4.3:max PASS 100% 237 in → 213 out (204 reasoning)
```bash git clean -nd ```
x-ai/grok-4.3:max__json_schema PASS 100% 293 in → 435 out (425 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
x-ai/grok-4.3:medium PASS 100% 243 in → 613 out (607 reasoning)
`git clean -nd`
x-ai/grok-4.3:medium__json_schema PASS 100% 299 in → 508 out (498 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
x-ai/grok-4.3:none PASS 100% 235 in → 6 out (0 reasoning)
git clean -n -d
x-ai/grok-4.3:none__json_schema PASS 100% 291 in → 10 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
x-ai/grok-4.3:xhigh PASS 100% 237 in → 597 out (588 reasoning)
```bash git clean -nd ```
x-ai/grok-4.3:xhigh__json_schema PASS 100% 293 in → 609 out (599 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) {"command":"git clean -n -d"}
x-ai/grok-4.5:high PASS 100% 259 in → 208 out (200 reasoning)
``` git clean -nd ```
x-ai/grok-4.5:high__json_schema PASS 100% 325 in → 403 out (394 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
x-ai/grok-4.5:low PASS 100% 259 in → 164 out (156 reasoning)
``` git clean -nd ```
x-ai/grok-4.5:low__json_schema PASS 100% 325 in → 315 out (306 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
x-ai/grok-4.5:max PASS 100% 259 in → 249 out (245 reasoning)
git clean -nd
x-ai/grok-4.5:max__json_schema PASS 100% 325 in → 468 out (459 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
x-ai/grok-4.5:medium PASS 100% 259 in → 577 out (569 reasoning)
``` git clean -nd ```
x-ai/grok-4.5:medium__json_schema PASS 100% 325 in → 413 out (404 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
x-ai/grok-4.5:xhigh PASS 100% 259 in → 323 out (315 reasoning)
``` git clean -nd ```
x-ai/grok-4.5:xhigh__json_schema PASS 100% 325 in → 535 out (526 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command":"git clean -nd"}
z-ai/glm-4.7-flash:high PASS 100% 58 in → 563 out (549 reasoning)
git clean -n -d
z-ai/glm-4.7-flash:high__json_schema PASS 100% 58 in → 948 out (948 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
z-ai/glm-4.7-flash:low PASS 100% 58 in → 689 out (703 reasoning)
git clean -d -n
z-ai/glm-4.7-flash:medium PASS 100% 58 in → 591 out (600 reasoning)
git clean -n -d
z-ai/glm-4.7-flash:none PASS 100% 58 in → 6 out (0 reasoning)
git clean -n -d
z-ai/glm-4.7-flash:none__json_schema PASS 100% 58 in → 10 out (0 reasoning)
git clean -n
JSON Schema Structured Output
(raw) {"command": "git clean -n"}
z-ai/glm-4.7-flash:xhigh PASS 100% 58 in → 1,533 out (1,567 reasoning)
git clean -n
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 58 in → 559 out (556 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) {"command": "git clean -nd"}
z-ai/glm-5.2:high PASS 100% 65 in → 271 out (0 reasoning)
git clean -n -d
z-ai/glm-5.2:high__json_schema PASS 100% 65 in → 261 out (248 reasoning)
git clean -nd
JSON Schema Structured Output
(raw) { "command": "git clean -nd" }
z-ai/glm-5.2:low PASS 100% 65 in → 271 out (262 reasoning)
git clean -n -d
z-ai/glm-5.2:low__json_schema PASS 100% 65 in → 265 out (235 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
z-ai/glm-5.2:medium PASS 100% 65 in → 243 out (227 reasoning)
git clean -n -d
z-ai/glm-5.2:none PASS 100% 59 in → 7 out (0 reasoning)
git clean -n -d
z-ai/glm-5.2:none__json_schema PASS 100% 59 in → 14 out (0 reasoning)
git clean -n -d
JSON Schema Structured Output
(raw) { "command": "git clean -n -d" }
z-ai/glm-5.2:xhigh PASS 100% 65 in → 533 out (510 reasoning)
git clean -nd
z-ai/glm-5.2:xhigh__json_schema PASS 100% 67 in → 512 out (499 reasoning)
git clean -n
JSON Schema Structured Output
(raw) { "command": "git clean -n" }
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -{ "command": "git clean -nd" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git ls-files --others --exclude-standard --name-only --directory=artifacts/logs --exclude="artifacts/logs/run.log" --exclude="scratch.txt"
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4783192f-8747-9b80-900a-ee04546acd37","request_id":"4783192f-8747-9b80-900a-ee04546acd37"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e46bdf53-23ac-9c6f-8c62-b0d2fd07fa40","request_id":"e46bdf53-23ac-9c6f-8c62-b0d2fd07fa40"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9d1f6bbb-9160-9e5d-bbd3-2023d8532dff","request_id":"9d1f6bbb-9160-9e5d-bbd3-2023d8532dff"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b173e338-576e-9994-95eb-e95edcf3824f","request_id":"b173e338-576e-9994-95eb-e95edcf3824f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d2fab7aa-952b-9e1c-8fd3-41cd970f6811","request_id":"d2fab7aa-952b-9e1c-8fd3-41cd970f6811"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-2ca6a572-dcdb-97c7-8f3a-2817e2a01e86"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-d5ccba25-2fb7-92a2-9baf-94ab4e82b700"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-1fc4dd9d-f041-941c-8410-d844faf73f4e"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-19149711-6b59-9862-a6df-9a33902d9a48"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-bb642357-f2b7-94d3-a7d0-c109778ed0bf"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-af8aa9eb-4015-9893-a7a7-2a9c0a56a5f1"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-54f261ff-2925-95b1-8c7e-fdcbb65e0743","request_id":"54f261ff-2925-95b1-8c7e-fdcbb65e0743"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1ed37c99-db79-964e-af41-b12285205f66","request_id":"1ed37c99-db79-964e-af41-b12285205f66"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7a25d707-72d5-9358-81f6-d6a31c9c2640","request_id":"7a25d707-72d5-9358-81f6-d6a31c9c2640"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3214064a-62f1-9996-8df9-985b230c1f53","request_id":"3214064a-62f1-9996-8df9-985b230c1f53"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-15e54d09-c8dc-942b-8b82-0ec9d176b894","request_id":"15e54d09-c8dc-942b-8b82-0ec9d176b894"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8451170e-4481-977d-abaf-20063fac8642","request_id":"8451170e-4481-977d-abaf-20063fac8642"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c2d2e51f-9691-9d35-b799-ff39e0837c3a","request_id":"c2d2e51f-9691-9d35-b799-ff39e0837c3a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c7e38d2f-eec6-9529-85aa-330493d835cc","request_id":"c7e38d2f-eec6-9529-85aa-330493d835cc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6a7860aa-7006-9e06-b122-07259f77133f","request_id":"6a7860aa-7006-9e06-b122-07259f77133f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-169d18b1-31b8-9d9b-8fa4-9c1d65c10292","request_id":"169d18b1-31b8-9d9b-8fa4-9c1d65c10292"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git clean -dn{"command": "git clean -dn"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -nd
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -nd
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -nd{"command": "git clean -nd"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -nd
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git clean -nd
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)