Create worktree from tag
Tests ability to create a worktree from a specific tag. Evaluates tag-based worktree creation.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'v1.0' > version.txt
  5. 05 git add version.txt
  6. 06 git commit -m 'Version 1.0'
  7. 07 git tag v1.0
  8. 08 echo 'v2.0' > version.txt
  9. 09 git add version.txt
  10. 10 git commit -m 'Version 2.0'
  11. 11 git tag v2.0
Prompt
Create a worktree at '../v1-checkout' pointing at the tag 'v1.0'. Output ONLY the git worktree command, nothing else.
Expected
git worktree add ../v1-checkout v1.0

Scoped model quality, cost, API time, and token usage for worktree_usage/f012.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 165 in → 38 out (14 reasoning)
``` git worktree add --detach ../v1-checkout v1.0 ```
anthropic/claude-fable-5:high__json_schema PASS 100% 374 in → 100 out (45 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-fable-5:low PASS 100% 160 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-fable-5:low__json_schema PASS 100% 371 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-fable-5:max PASS 100% 166 in → 169 out (44 reasoning)
git worktree add ../v1-checkout v1.0
anthropic/claude-fable-5:max__json_schema PASS 100% 374 in → 167 out (52 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-fable-5:medium PASS 100% 164 in → 38 out (14 reasoning)
``` git worktree add --detach ../v1-checkout v1.0 ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 371 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-fable-5:xhigh PASS 100% 162 in → 329 out (61 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 373 in → 147 out (58 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-haiku-4.5:high PASS 100% 164 in → 119 out (80 reasoning)
git worktree add ../v1-checkout v1.0
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 323 in → 185 out (141 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-haiku-4.5:low PASS 100% 168 in → 146 out (93 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 328 in → 163 out (113 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-haiku-4.5:medium PASS 100% 164 in → 153 out (105 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 331 in → 170 out (126 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-haiku-4.5:none PASS 100% 135 in → 21 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 300 in → 22 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 164 in → 182 out (132 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 333 in → 211 out (177 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.6:high PASS 100% 138 in → 21 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 298 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.6:low PASS 100% 136 in → 21 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 301 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.6:max PASS 100% 135 in → 78 out (35 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 296 in → 65 out (22 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.6:medium PASS 100% 138 in → 21 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 299 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.6:none PASS 100% 135 in → 21 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 300 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.6:xhigh PASS 100% 139 in → 22 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 300 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.7:high PASS 100% 170 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
anthropic/claude-opus-4.7:high__json_schema PASS 100% 379 in → 28 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.7:low PASS 100% 169 in → 23 out (0 reasoning)
`git worktree add ../v1-checkout v1.0`
anthropic/claude-opus-4.7:low__json_schema PASS 100% 379 in → 28 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.7:max PASS 100% 169 in → 26 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 376 in → 29 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.7:medium PASS 100% 170 in → 23 out (0 reasoning)
`git worktree add ../v1-checkout v1.0`
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 376 in → 28 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.7:none PASS 100% 176 in → 26 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.7:none__json_schema PASS 100% 382 in → 28 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.7:xhigh PASS 100% 170 in → 23 out (0 reasoning)
`git worktree add ../v1-checkout v1.0`
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 381 in → 28 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.8:high PASS 100% 168 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 375 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.8:low PASS 100% 161 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 376 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.8:max PASS 100% 165 in → 75 out (29 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 373 in → 86 out (29 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.8:medium PASS 100% 161 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 377 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.8:none PASS 100% 161 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 379 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-opus-4.8:xhigh PASS 100% 158 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 380 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-4.6:high PASS 100% 138 in → 64 out (22 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 298 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-4.6:low PASS 100% 137 in → 19 out (0 reasoning)
`git worktree add ../v1-checkout v1.0`
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 299 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-4.6:max PASS 100% 132 in → 86 out (41 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 296 in → 86 out (41 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-4.6:medium PASS 100% 133 in → 19 out (0 reasoning)
`git worktree add ../v1-checkout v1.0`
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-4.6:none PASS 100% 141 in → 21 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 301 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 138 in → 64 out (22 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 299 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-5:high PASS 100% 167 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 371 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-5:low PASS 100% 163 in → 19 out (0 reasoning)
git worktree add ../v1-checkout v1.0
anthropic/claude-sonnet-5:low__json_schema PASS 100% 370 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-5:medium PASS 100% 160 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 373 in → 49 out (30 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-5:none PASS 100% 158 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 375 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
anthropic/claude-sonnet-5:xhigh PASS 100% 165 in → 23 out (0 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 371 in → 67 out (23 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
arcee-ai/trinity-large-thinking:high PASS 100% 119 in → 560 out (557 reasoning)
git worktree add ../v1-checkout v1.0
arcee-ai/trinity-large-thinking:low PASS 100% 116 in → 545 out (543 reasoning)
git worktree add '../v1-checkout' v1.0
arcee-ai/trinity-large-thinking:medium PASS 100% 118 in → 402 out (392 reasoning)
git worktree add '../v1-checkout' v1.0
arcee-ai/trinity-large-thinking:xhigh PASS 100% 117 in → 441 out (436 reasoning)
git worktree add ../v1-checkout v1.0
arcee-ai/trinity-mini:high PASS 100% 115 in → 278 out (277 reasoning)
git worktree add ../v1-checkout v1.0
arcee-ai/trinity-mini:high__json_schema PASS 100% 116 in → 539 out (535 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
arcee-ai/trinity-mini:low PASS 100% 120 in → 288 out (289 reasoning)
git worktree add ../v1-checkout v1.0
arcee-ai/trinity-mini:low__json_schema PASS 100% 119 in → 271 out (260 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
arcee-ai/trinity-mini:xhigh PASS 100% 115 in → 497 out (494 reasoning)
git worktree add ../v1-checkout v1.0
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 114 in → 278 out (268 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 197 in → 67 out (48 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 204 in → 111 out (80 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 118 in → 125 out (99 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 262 in → 77 out (52 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 200 in → 80 out (63 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 203 in → 124 out (105 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-flash-0731:medium PASS 100% 118 in → 119 out (97 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 125 in → 153 out (118 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 118 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 114 in → 18 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 119 in → 86 out (67 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 118 in → 170 out (141 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash:high PASS 100% 120 in → 134 out (117 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 118 in → 119 out (97 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash:low PASS 100% 114 in → 199 out (186 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 119 in → 124 out (106 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash:medium PASS 100% 132 in → 101 out (86 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 122 in → 89 out (66 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-flash:none PASS 100% 118 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 118 in → 22 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
deepseek/deepseek-v4-pro:high PASS 100% 119 in → 119 out (104 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 306 in → 80 out (60 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-pro:low PASS 100% 119 in → 79 out (65 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 306 in → 195 out (174 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 303 in → 95 out (75 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-pro:none PASS 100% 119 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 304 in → 19 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
deepseek/deepseek-v4-pro:xhigh PASS 100% 118 in → 253 out (198 reasoning)
git worktree add ../v1-checkout v1.0
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 386 in → 515 out (495 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3-flash-preview:high PASS 100% 140 in → 197 out (184 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3-flash-preview:high__json_schema PASS 100% 200 in → 211 out (194 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3-flash-preview:low PASS 100% 139 in → 130 out (117 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3-flash-preview:low__json_schema PASS 100% 197 in → 228 out (210 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3-flash-preview:medium PASS 100% 145 in → 137 out (124 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3-flash-preview:medium__json_schema PASS 100% 201 in → 193 out (176 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3-flash-preview:xhigh PASS 100% 142 in → 179 out (166 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 201 in → 135 out (119 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 140 in → 411 out (398 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 143 in → 263 out (245 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 139 in → 149 out (136 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 197 in → 130 out (112 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 143 in → 138 out (125 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 196 in → 195 out (177 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 142 in → 186 out (173 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 199 in → 367 out (349 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-pro-preview:high PASS 100% 139 in → 785 out (772 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 199 in → 604 out (586 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-pro-preview:low PASS 100% 141 in → 291 out (278 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 198 in → 71 out (53 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-pro-preview:medium PASS 100% 143 in → 801 out (788 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 197 in → 81 out (63 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 143 in → 368 out (355 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 205 in → 283 out (265 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash-lite:high PASS 100% 141 in → 270 out (257 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 200 in → 232 out (214 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash-lite:low PASS 100% 143 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 200 in → 23 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemini-3.5-flash-lite:max PASS 100% 142 in → 255 out (242 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 199 in → 304 out (286 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash-lite:medium PASS 100% 143 in → 239 out (226 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 201 in → 188 out (170 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 143 in → 279 out (266 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 139 in → 242 out (224 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash:high PASS 100% 139 in → 474 out (461 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash:high__json_schema PASS 100% 201 in → 281 out (263 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash:low PASS 100% 140 in → 382 out (369 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash:low__json_schema PASS 100% 202 in → 99 out (81 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash:medium PASS 100% 140 in → 309 out (296 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash:medium__json_schema PASS 100% 198 in → 338 out (320 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.5-flash:xhigh PASS 100% 140 in → 263 out (250 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 199 in → 311 out (293 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
google/gemini-3.6-flash:high PASS 100% 145 in → 381 out (368 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.6-flash:high__json_schema PASS 100% 142 in → 274 out (257 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3.6-flash:low PASS 100% 143 in → 203 out (190 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.6-flash:low__json_schema PASS 100% 147 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3.6-flash:max PASS 100% 139 in → 276 out (263 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.6-flash:max__json_schema PASS 100% 202 in → 244 out (227 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3.6-flash:medium PASS 100% 146 in → 265 out (252 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.6-flash:medium__json_schema PASS 100% 202 in → 232 out (215 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemini-3.6-flash:xhigh PASS 100% 142 in → 271 out (258 reasoning)
git worktree add ../v1-checkout v1.0
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 141 in → 235 out (218 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemma-4-26b-a4b-it:high PASS 100% 158 in → 156 out (112 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 158 in → 205 out (138 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-26b-a4b-it:low PASS 100% 155 in → 147 out (119 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 158 in → 177 out (121 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-26b-a4b-it:medium PASS 100% 158 in → 226 out (172 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 160 in → 233 out (169 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-26b-a4b-it:none PASS 100% 152 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 158 in → 24 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 157 in → 151 out (104 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 158 in → 174 out (115 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-31b-it:high PASS 100% 157 in → 153 out (103 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-31b-it:high__json_schema PASS 100% 158 in → 146 out (87 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-31b-it:low PASS 100% 152 in → 185 out (126 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-31b-it:low__json_schema PASS 100% 155 in → 149 out (96 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-31b-it:medium__json_schema PASS 100% 157 in → 142 out (83 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
google/gemma-4-31b-it:none PASS 100% 157 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-31b-it:none__json_schema PASS 100% 154 in → 19 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
google/gemma-4-31b-it:xhigh PASS 100% 160 in → 126 out (112 reasoning)
git worktree add ../v1-checkout v1.0
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 160 in → 244 out (176 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
ibm-granite/granite-4.1-8b PASS 100% 129 in → 14 out
git worktree add ../v1-checkout v1.0
ibm-granite/granite-4.1-8b__json_schema PASS 100% 131 in → 24 out
git worktree add ../v1-checkout refs/tags/v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout refs/tags/v1.0" }
inclusionai/ling-2.6-flash PASS 100% 157 in → 14 out
git worktree add ../v1-checkout v1.0
inclusionai/ling-2.6-flash__json_schema PASS 100% 147 in → 18 out
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
minimax/minimax-m2.5:high PASS 100% 163 in → 219 out (204 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.5:high__json_schema PASS 100% 146 in → 110 out (88 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m2.5:low PASS 100% 143 in → 100 out (85 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.5:low__json_schema PASS 100% 145 in → 209 out (163 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m2.5:medium PASS 100% 146 in → 94 out (75 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.5:medium__json_schema PASS 100% 149 in → 101 out (80 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m2.5:xhigh PASS 100% 142 in → 102 out (87 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 163 in → 91 out (69 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m2.7:high PASS 100% 147 in → 212 out (197 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.7:high__json_schema PASS 100% 143 in → 294 out (268 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m2.7:low PASS 100% 142 in → 284 out (270 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.7:low__json_schema PASS 100% 278 in → 541 out (518 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m2.7:medium PASS 100% 152 in → 79 out (65 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.7:medium__json_schema PASS 100% 281 in → 320 out (301 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
minimax/minimax-m2.7:xhigh PASS 100% 142 in → 229 out (213 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 284 in → 448 out (425 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
minimax/minimax-m3:high PASS 100% 282 in → 52 out (36 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m3:high__json_schema PASS 100% 271 in → 20 out (0 reasoning)
git worktree add '../v1-checkout' v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add '../v1-checkout' v1.0"}
minimax/minimax-m3:low PASS 100% 280 in → 96 out (77 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m3:low__json_schema PASS 100% 269 in → 20 out (0 reasoning)
git worktree add '../v1-checkout' v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add '../v1-checkout' v1.0"}
minimax/minimax-m3:medium PASS 100% 285 in → 71 out (54 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m3:medium__json_schema PASS 100% 270 in → 20 out (0 reasoning)
git worktree add '../v1-checkout' v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add '../v1-checkout' v1.0"}
minimax/minimax-m3:xhigh PASS 100% 285 in → 72 out (55 reasoning)
git worktree add ../v1-checkout v1.0
minimax/minimax-m3:xhigh__json_schema PASS 100% 231 in → 24 out (23 reasoning)
git worktree add -f ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add -f ../v1-checkout v1.0" }
mistralai/devstral-2512 PASS 100% 139 in → 19 out
```sh git worktree add ../v1-checkout v1.0 ```
mistralai/devstral-2512__json_schema PASS 100% 144 in → 19 out
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
mistralai/mistral-medium-3-5:high PASS 100% 152 in → 494 out (482 reasoning)
git worktree add ../v1-checkout v1.0
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 146 in → 201 out (183 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
mistralai/mistral-medium-3-5:low PASS 100% 155 in → 512 out (484 reasoning)
git worktree add ../v1-checkout v1.0
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 141 in → 593 out (582 reasoning)
git worktree add '../v1-checkout' v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add '../v1-checkout' v1.0"}
mistralai/mistral-medium-3-5:medium PASS 100% 155 in → 576 out (576 reasoning)
git worktree add ../v1-checkout v1.0
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 144 in → 566 out (514 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
mistralai/mistral-medium-3-5:none PASS 100% 152 in → 19 out (0 reasoning)
```sh git worktree add ../v1-checkout v1.0 ```
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 141 in → 19 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 158 in → 233 out (207 reasoning)
git worktree add ../v1-checkout v1.0
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 142 in → 318 out (288 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
moonshotai/kimi-k2.6:high PASS 100% 112 in → 107 out (105 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.6:high__json_schema PASS 100% 111 in → 1,312 out (177 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command" : "git worktree add ../v1-checkout v1.0" }
moonshotai/kimi-k2.6:low PASS 100% 108 in → 241 out (221 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.6:low__json_schema PASS 100% 114 in → 21 out (0 reasoning)
git worktree add '../v1-checkout' 'v1.0'
JSON Schema Structured Output
(raw) {"command": "git worktree add '../v1-checkout' 'v1.0'"}
moonshotai/kimi-k2.6:medium PASS 100% 106 in → 270 out (256 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 294 in → 350 out (332 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
moonshotai/kimi-k2.6:xhigh PASS 100% 110 in → 177 out (167 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 113 in → 527 out (386 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
moonshotai/kimi-k2.7-code:high PASS 100% 115 in → 242 out (228 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 110 in → 182 out (161 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
moonshotai/kimi-k2.7-code:low PASS 100% 111 in → 158 out (144 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.7-code:medium PASS 100% 110 in → 141 out (122 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 112 in → 88 out (66 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 114 in → 83 out (69 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:high PASS 100% 188 in → 132 out (102 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
moonshotai/kimi-k3:high__json_schema PASS 100% 186 in → 147 out (120 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:low PASS 100% 189 in → 89 out (62 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:low__json_schema PASS 100% 186 in → 101 out (74 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:max PASS 100% 190 in → 93 out (63 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
moonshotai/kimi-k3:max__json_schema PASS 100% 189 in → 176 out (149 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:medium PASS 100% 191 in → 88 out (61 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:medium__json_schema PASS 100% 191 in → 97 out (67 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
moonshotai/kimi-k3:xhigh PASS 100% 191 in → 125 out (98 reasoning)
git worktree add ../v1-checkout v1.0
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 189 in → 193 out (166 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 152 in → 177 out (159 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 153 in → 1,266 out (1,348 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 157 in → 119 out (88 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 157 in → 222 out (207 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 154 in → 211 out (182 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 153 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 157 in → 26 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 158 in → 106 out (87 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 154 in → 218 out (192 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 155 in → 126 out (109 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 155 in → 138 out (114 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 150 in → 73 out (57 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 158 in → 108 out (86 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 156 in → 86 out (59 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 155 in → 120 out (98 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 154 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 155 in → 22 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 158 in → 81 out (63 reasoning)
git worktree add ../v1-checkout v1.0
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 154 in → 118 out (95 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
openai/gpt-5.4-mini:high PASS 100% 114 in → 160 out (141 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-mini:high__json_schema PASS 100% 150 in → 245 out (218 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-mini:low PASS 100% 117 in → 98 out (79 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-mini:low__json_schema PASS 100% 154 in → 61 out (34 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-mini:medium PASS 100% 115 in → 78 out (59 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-mini:medium__json_schema PASS 100% 155 in → 129 out (102 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-mini:none PASS 100% 115 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-mini:none__json_schema PASS 100% 150 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-mini:xhigh PASS 100% 111 in → 312 out (293 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-nano:high PASS 100% 114 in → 106 out (87 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-nano:high__json_schema PASS 100% 151 in → 109 out (82 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-nano:low PASS 100% 117 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-nano:low__json_schema PASS 100% 156 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-nano:medium PASS 100% 112 in → 96 out (77 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-nano:medium__json_schema PASS 100% 154 in → 71 out (44 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-nano:none PASS 100% 114 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-nano:none__json_schema PASS 100% 151 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4-nano:xhigh PASS 100% 112 in → 129 out (110 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 149 in → 580 out (553 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4:high PASS 100% 115 in → 1,444 out (1,423 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.4:high__json_schema PASS 100% 149 in → 545 out (516 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.4:low PASS 100% 113 in → 80 out (61 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4:low__json_schema PASS 100% 149 in → 74 out (47 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.4:medium PASS 100% 115 in → 98 out (79 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4:medium__json_schema PASS 100% 147 in → 282 out (253 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.4:none PASS 100% 119 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.4:none__json_schema PASS 100% 153 in → 27 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.4:xhigh PASS 100% 118 in → 537 out (516 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.4:xhigh__json_schema PASS 100% 149 in → 9,205 out (9,176 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.5:high PASS 100% 120 in → 94 out (75 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.5:high__json_schema PASS 100% 148 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.5:low PASS 100% 116 in → 60 out (41 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.5:low__json_schema PASS 100% 149 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.5:medium PASS 100% 123 in → 83 out (64 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.5:medium__json_schema PASS 100% 147 in → 114 out (87 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.5:none PASS 100% 113 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.5:none__json_schema PASS 100% 153 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.5:xhigh PASS 100% 116 in → 372 out (351 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.5:xhigh__json_schema PASS 100% 153 in → 247 out (220 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-luna:high PASS 100% 114 in → 107 out (88 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-luna:high__json_schema PASS 100% 153 in → 121 out (94 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-luna:low PASS 100% 112 in → 56 out (37 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-luna:low__json_schema PASS 100% 148 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-luna:max PASS 100% 115 in → 102 out (83 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-luna:max__json_schema PASS 100% 151 in → 543 out (516 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-luna:medium PASS 100% 117 in → 64 out (45 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-luna:medium__json_schema PASS 100% 149 in → 60 out (33 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-luna:none PASS 100% 112 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-luna:none__json_schema PASS 100% 153 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-luna:xhigh PASS 100% 116 in → 118 out (97 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 151 in → 102 out (75 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-sol:high PASS 100% 114 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-sol:high__json_schema PASS 100% 150 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-sol:low PASS 100% 116 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-sol:low__json_schema PASS 100% 149 in → 27 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.6-sol:max PASS 100% 116 in → 80 out (61 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-sol:max__json_schema PASS 100% 148 in → 257 out (230 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-sol:medium PASS 100% 110 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-sol:medium__json_schema PASS 100% 153 in → 27 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.6-sol:none PASS 100% 118 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-sol:none__json_schema PASS 100% 151 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-sol:xhigh PASS 100% 118 in → 19 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 151 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-terra:high PASS 100% 111 in → 124 out (105 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-terra:high__json_schema PASS 100% 149 in → 27 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.6-terra:low PASS 100% 120 in → 59 out (38 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.6-terra:low__json_schema PASS 100% 151 in → 87 out (58 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.6-terra:max PASS 100% 113 in → 240 out (219 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.6-terra:max__json_schema PASS 100% 150 in → 306 out (277 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add --detach ../v1-checkout v1.0"}
openai/gpt-5.6-terra:medium PASS 100% 112 in → 54 out (35 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-terra:medium__json_schema PASS 100% 152 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-terra:none PASS 100% 111 in → 19 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
openai/gpt-5.6-terra:none__json_schema PASS 100% 150 in → 25 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-5.6-terra:xhigh PASS 100% 117 in → 147 out (128 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 147 in → 96 out (69 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-oss-120b:high PASS 100% 175 in → 289 out (250 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-120b:high__json_schema PASS 100% 164 in → 166 out (142 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-oss-120b:low PASS 100% 176 in → 82 out (49 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-120b:low__json_schema PASS 100% 174 in → 112 out (72 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
openai/gpt-oss-120b:medium PASS 100% 177 in → 174 out (144 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-120b:medium__json_schema PASS 100% 177 in → 21 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
openai/gpt-oss-120b:xhigh PASS 100% 178 in → 691 out (621 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 175 in → 316 out (297 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-oss-20b:high PASS 100% 164 in → 182 out (154 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-20b:high__json_schema PASS 100% 160 in → 328 out (304 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
openai/gpt-oss-20b:low PASS 100% 175 in → 40 out (15 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-20b:low__json_schema PASS 100% 180 in → 85 out (55 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
openai/gpt-oss-20b:medium PASS 100% 163 in → 120 out (88 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-20b:medium__json_schema PASS 100% 162 in → 168 out (126 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
openai/gpt-oss-20b:xhigh PASS 100% 179 in → 541 out (513 reasoning)
git worktree add ../v1-checkout v1.0
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 174 in → 231 out (203 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
poolside/laguna-m.1:high PASS 100% 141 in → 404 out (389 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-m.1:high__json_schema PASS 100% 143 in → 352 out (334 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-m.1:low PASS 100% 141 in → 287 out (273 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-m.1:low__json_schema PASS 100% 141 in → 346 out (323 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
poolside/laguna-m.1:medium PASS 100% 143 in → 411 out (396 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-m.1:medium__json_schema PASS 100% 137 in → 336 out (315 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
poolside/laguna-m.1:none PASS 100% 139 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-m.1:none__json_schema PASS 100% 143 in → 23 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
poolside/laguna-m.1:xhigh PASS 100% 142 in → 346 out (332 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-m.1:xhigh__json_schema PASS 100% 142 in → 235 out (211 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
poolside/laguna-xs-2.1:high PASS 100% 142 in → 304 out (292 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs-2.1:high__json_schema PASS 100% 141 in → 348 out (331 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs-2.1:low PASS 100% 144 in → 255 out (243 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs-2.1:low__json_schema PASS 100% 139 in → 306 out (288 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs-2.1:medium PASS 100% 145 in → 278 out (266 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 133 in → 290 out (272 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs-2.1:none PASS 100% 139 in → 12 out (0 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs-2.1:none__json_schema PASS 100% 138 in → 23 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
poolside/laguna-xs-2.1:xhigh PASS 100% 145 in → 551 out (540 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 139 in → 463 out (446 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs.2:high PASS 100% 180 in → 161 out (147 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs.2:high__json_schema PASS 100% 179 in → 180 out (162 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs.2:low PASS 100% 178 in → 210 out (196 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs.2:low__json_schema PASS 100% 179 in → 118 out (100 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs.2:medium PASS 100% 180 in → 152 out (138 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs.2:medium__json_schema PASS 100% 179 in → 124 out (106 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs.2:none PASS 100% 180 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs.2:none__json_schema PASS 100% 179 in → 18 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
poolside/laguna-xs.2:xhigh PASS 100% 181 in → 133 out (119 reasoning)
git worktree add ../v1-checkout v1.0
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 176 in → 118 out (100 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
qwen/qwen3.6-27b:high PASS 100% 141 in → 1,199 out (1,181 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-27b:high__json_schema PASS 100% 145 in → 1,241 out (1,107 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-27b:low PASS 100% 141 in → 183 out (173 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-27b:low__json_schema PASS 100% 144 in → 774 out (723 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-27b:medium PASS 100% 144 in → 2,850 out (2,644 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-27b:medium__json_schema PASS 100% 145 in → 1,331 out (1 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-27b:none PASS 100% 143 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-27b:none__json_schema PASS 100% 142 in → 24 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-27b:xhigh PASS 100% 142 in → 200 out (1 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 146 in → 379 out (332 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-35b-a3b:high PASS 100% 144 in → 876 out (798 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 145 in → 1,119 out (968 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-35b-a3b:low PASS 100% 144 in → 1,121 out (1,005 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 144 in → 703 out (636 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 145 in → 646 out (543 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 146 in → 1,085 out (949 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-35b-a3b:none PASS 100% 146 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 146 in → 870 out (817 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 143 in → 653 out (579 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
qwen/qwen3.6-flash:high PASS 100% 143 in → 428 out (410 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-flash:low PASS 100% 146 in → 709 out (691 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-flash:medium PASS 100% 146 in → 727 out (709 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-flash:none PASS 100% 146 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.6-flash:xhigh PASS 100% 144 in → 969 out (951 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-flash:high PASS 100% 144 in → 1,022 out (1,004 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-flash:low PASS 100% 142 in → 771 out (753 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-flash:max PASS 100% 141 in → 809 out (791 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-flash:medium PASS 100% 144 in → 614 out (596 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-flash:none PASS 100% 144 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-flash:xhigh PASS 100% 146 in → 684 out (666 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-max:high PASS 100% 142 in → 280 out (262 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-max:low PASS 100% 146 in → 751 out (733 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-max:medium PASS 100% 141 in → 220 out (202 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-max:none PASS 100% 147 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-max:xhigh PASS 100% 139 in → 226 out (208 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-plus:high PASS 100% 146 in → 700 out (682 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-plus:low PASS 100% 146 in → 464 out (446 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-plus:medium PASS 100% 139 in → 578 out (560 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-plus:none PASS 100% 147 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
qwen/qwen3.7-plus:xhigh PASS 100% 144 in → 492 out (474 reasoning)
git worktree add ../v1-checkout v1.0
tencent/hy3:high PASS 100% 121 in → 502 out (461 reasoning)
git worktree add ../v1-checkout v1.0
tencent/hy3:high__json_schema PASS 100% 122 in → 541 out (476 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
tencent/hy3:low PASS 100% 124 in → 205 out (190 reasoning)
git worktree add ../v1-checkout v1.0
tencent/hy3:low__json_schema PASS 100% 125 in → 252 out (205 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
tencent/hy3:medium PASS 100% 122 in → 255 out (240 reasoning)
git worktree add ../v1-checkout v1.0
tencent/hy3:medium__json_schema PASS 100% 123 in → 495 out (418 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
tencent/hy3:none PASS 100% 127 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
tencent/hy3:none__json_schema PASS 100% 129 in → 22 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
tencent/hy3:xhigh PASS 100% 124 in → 575 out (560 reasoning)
git worktree add ../v1-checkout v1.0
tencent/hy3:xhigh__json_schema PASS 100% 120 in → 342 out (304 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
thinkingmachines/inkling-small:high PASS 100% 126 in → 132 out (105 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling-small:high__json_schema PASS 100% 126 in → 178 out (155 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
thinkingmachines/inkling-small:low PASS 100% 127 in → 74 out (55 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling-small:low__json_schema PASS 100% 124 in → 76 out (50 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
thinkingmachines/inkling-small:max PASS 100% 125 in → 213 out (194 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling-small:max__json_schema PASS 100% 116 in → 148 out (122 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
thinkingmachines/inkling-small:medium PASS 100% 127 in → 80 out (58 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling-small:medium__json_schema PASS 100% 123 in → 132 out (109 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
thinkingmachines/inkling-small:xhigh PASS 100% 124 in → 156 out (117 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 127 in → 168 out (145 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
thinkingmachines/inkling:high PASS 100% 129 in → 201 out (162 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling:low PASS 100% 124 in → 38 out (19 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling:max PASS 100% 126 in → 118 out (94 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling:max__json_schema PASS 100% 124 in → 182 out (157 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
thinkingmachines/inkling:medium PASS 100% 125 in → 108 out (77 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling:medium__json_schema PASS 100% 125 in → 85 out (62 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
thinkingmachines/inkling:none__json_schema PASS 100% 122 in → 24 out (0 reasoning)
git worktree add --detach ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add --detach ../v1-checkout v1.0" }
thinkingmachines/inkling:xhigh PASS 100% 125 in → 148 out (129 reasoning)
git worktree add ../v1-checkout v1.0
thinkingmachines/inkling:xhigh__json_schema PASS 100% 124 in → 221 out (198 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.3:high PASS 100% 288 in → 104 out (91 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.3:high__json_schema PASS 100% 349 in → 270 out (253 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.3:low PASS 100% 301 in → 364 out (351 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.3:low__json_schema PASS 100% 346 in → 231 out (210 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
x-ai/grok-4.3:max PASS 100% 293 in → 176 out (163 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.3:max__json_schema PASS 100% 347 in → 519 out (498 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
x-ai/grok-4.3:medium PASS 100% 293 in → 509 out (496 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.3:medium__json_schema PASS 100% 352 in → 379 out (358 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add ../v1-checkout v1.0" }
x-ai/grok-4.3:none PASS 100% 284 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.3:none__json_schema PASS 100% 342 in → 17 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.3:xhigh PASS 100% 291 in → 173 out (160 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.3:xhigh__json_schema PASS 100% 347 in → 336 out (319 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.5:high PASS 100% 327 in → 209 out (190 reasoning)
``` git worktree add '../v1-checkout' 'v1.0' ```
x-ai/grok-4.5:high__json_schema PASS 100% 387 in → 258 out (240 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.5:low PASS 100% 324 in → 78 out (61 reasoning)
``` git worktree add ../v1-checkout v1.0 ```
x-ai/grok-4.5:low__json_schema PASS 100% 393 in → 115 out (97 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.5:max PASS 100% 329 in → 243 out (230 reasoning)
git worktree add ../v1-checkout v1.0
x-ai/grok-4.5:max__json_schema PASS 100% 392 in → 244 out (225 reasoning)
git worktree add '../v1-checkout' v1.0
JSON Schema Structured Output
(raw) {"command":"git worktree add '../v1-checkout' v1.0"}
x-ai/grok-4.5:medium PASS 100% 331 in → 260 out (242 reasoning)
``` git worktree add '../v1-checkout' v1.0 ```
x-ai/grok-4.5:medium__json_schema PASS 100% 388 in → 234 out (216 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
x-ai/grok-4.5:xhigh PASS 100% 323 in → 268 out (252 reasoning)
git worktree add '../v1-checkout' 'v1.0'
x-ai/grok-4.5:xhigh__json_schema PASS 100% 392 in → 337 out (318 reasoning)
git worktree add '../v1-checkout' v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add '../v1-checkout' v1.0"}
z-ai/glm-4.7-flash:high PASS 100% 119 in → 349 out (305 reasoning)
git worktree add '../v1-checkout' v1.0
z-ai/glm-4.7-flash:low PASS 100% 122 in → 300 out (258 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-4.7-flash:low__json_schema PASS 100% 123 in → 281 out (230 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
z-ai/glm-4.7-flash:medium PASS 100% 121 in → 285 out (235 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 125 in → 281 out (222 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
z-ai/glm-4.7-flash:none PASS 100% 118 in → 13 out (0 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-4.7-flash:none__json_schema PASS 100% 119 in → 19 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
z-ai/glm-4.7-flash:xhigh PASS 100% 118 in → 287 out (232 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 122 in → 270 out (227 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
z-ai/glm-5.2:high PASS 100% 119 in → 282 out (258 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-5.2:low PASS 100% 128 in → 218 out (203 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-5.2:low__json_schema PASS 100% 128 in → 225 out (173 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command":"git worktree add ../v1-checkout v1.0" }
z-ai/glm-5.2:medium__json_schema PASS 100% 130 in → 238 out (203 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
z-ai/glm-5.2:none PASS 100% 119 in → 14 out (0 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-5.2:none__json_schema PASS 100% 124 in → 19 out (0 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
z-ai/glm-5.2:xhigh PASS 100% 128 in → 471 out (456 reasoning)
git worktree add ../v1-checkout v1.0
z-ai/glm-5.2:xhigh__json_schema PASS 100% 304 in → 68 out (48 reasoning)
git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
(raw) {"command": "git worktree add ../v1-checkout v1.0"}
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
arcee-ai/trinity-mini:medium FAIL 0% 114 in → 411 out (402 reasoning)
git worktreeadd ../v1-checkout v1.0
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
arcee-ai/trinity-mini:medium__json_schema FAIL 0% 113 in → 542 out (532 reasoning)
git worktree add../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add../v1-checkout v1.0" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
deepseek/deepseek-v4-flash:xhigh FAIL 0% 196 in → 897 out (827 reasoning)
gitworktree add ../v1-checkout v1.0
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
Invalid JSON. Output: gitworktree add ../v1-checkout v1.0
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
deepseek/deepseek-v4-pro:medium FAIL 0% 116 in → 518 out (464 reasoning)
gitworktree add ../v1-checkout v1.0
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
google/gemma-4-31b-it:medium FAIL 0% 160 in → 137 out (89 reasoning)
git worktree add ../v1-checkout v1.L0
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
liquid/lfm-2-24b-a2b FAIL 0% 131 in → 58 out
worktree /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f012_toay4oub/worktree_usage_f012 v1.0
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
Invalid JSON. Output: worktree /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f012_j8xoj3pr/worktree_usage_f012 v1.0
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
nvidia/nemotron-3-nano-30b-a3b:high FAIL 0% 156 in → 172 out (150 reasoning)
gitworktree add ../v1-checkout tags/v1.0
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
(empty output)
JSON Schema Structured Output
Failure: Expecting value: line 535 column 1 (char 2937)
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 0% 144 in → 27 out (0 reasoning)
git worktree add -b main ../v1-checkout v1.0
JSON Schema Structured Output
(raw) { "command": "git worktree add -b main ../v1-checkout v1.0" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-ba59cfb2-3fc5-9fb2-b799-5ae8de1ff163","request_id":"ba59cfb2-3fc5-9fb2-b799-5ae8de1ff163"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-049adb43-543e-92e3-a247-ce1c821c7403","request_id":"049adb43-543e-92e3-a247-ce1c821c7403"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a692686d-7c5c-91c1-ae81-04ec51e81eb1","request_id":"a692686d-7c5c-91c1-ae81-04ec51e81eb1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-83c0cb84-7fb6-9829-aaad-3a05625a66db","request_id":"83c0cb84-7fb6-9829-aaad-3a05625a66db"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-731c6f75-3230-9640-adb9-2517258f7f16","request_id":"731c6f75-3230-9640-adb9-2517258f7f16"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-05a52afb-6b71-9683-8a27-588e0475daab"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-92f8f458-7119-9d55-9876-c073eb1b1244"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-c30e8cf3-5dcf-9f5d-9629-518e7d375212"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-1e53a656-edb6-91a5-8c92-abce49823a4c"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-fd351e17-39fb-9f17-8118-a1c9977d1756"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ed2ee9eb-38a2-9a65-a273-0367212d12b3"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4688dd14-8574-97aa-9f0a-9d32cad97271","request_id":"4688dd14-8574-97aa-9f0a-9d32cad97271"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-bdcb0143-1940-9e70-b8c9-7e2bc9c2eda6","request_id":"bdcb0143-1940-9e70-b8c9-7e2bc9c2eda6"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f5f5e07f-2eb7-966b-9a8a-9f11c106d9d4","request_id":"f5f5e07f-2eb7-966b-9a8a-9f11c106d9d4"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c7f80dac-9357-9a77-85bd-8542669e5769","request_id":"c7f80dac-9357-9a77-85bd-8542669e5769"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-19c9b9a1-62af-91a8-a524-e20dc0dfb44a","request_id":"19c9b9a1-62af-91a8-a524-e20dc0dfb44a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9db47a9b-341f-93b2-aae7-6d5e3839e21c","request_id":"9db47a9b-341f-93b2-aae7-6d5e3839e21c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3ba9e485-9445-95d9-b4c7-eb64aeacee6a","request_id":"3ba9e485-9445-95d9-b4c7-eb64aeacee6a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-21a8ce7d-52a7-9c2c-b98b-a0cf0d1d138e","request_id":"21a8ce7d-52a7-9c2c-b98b-a0cf0d1d138e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1fadd3e2-c1ef-979b-9a4a-befd6276ab61","request_id":"1fadd3e2-c1ef-979b-9a4a-befd6276ab61"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6e860849-7e89-9b73-b22b-25c42cc0e396","request_id":"6e860849-7e89-9b73-b22b-25c42cc0e396"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
thinkingmachines/inkling-small:none FAIL 0% 122 in → 16 out (0 reasoning)
git worktree add ../v1-checkout v1.0<|end_message|>
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
Invalid JSON. Output: git worktree add ../v1-checkout v1.0{ "command": "git worktree add ../v1-checkout v1.0" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
thinkingmachines/inkling:none FAIL 0% 124 in → 16 out (0 reasoning)
git worktree add ../v1-checkout v1.0<|end_message|>
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../v1-checkout v1.0
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-5.2:medium FAIL 0% 129 in → 220 out (0 reasoning)
git worktree add ../v1-checkout v1.
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../v1-checkout'}, {'type': 'file_content', 'path': '../v1-checkout/version.txt', 'value': 'v1.0'}]