Create multiple worktrees
Tests ability to create and manage multiple worktrees simultaneously. Evaluates multi-worktree management.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'Hello' > hello.txt
  5. 05 git add hello.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 git checkout -b feature-a
  8. 08 echo 'A' > a.txt
  9. 09 git add a.txt
  10. 10 git commit -m 'Feature A'
  11. 11 git checkout main
  12. 12 git checkout -b feature-b
  13. 13 echo 'B' > b.txt
  14. 14 git add b.txt
  15. 15 git commit -m 'Feature B'
  16. 16 git checkout main
Prompt
Create worktrees for both 'feature-a' at '../wt-a' and 'feature-b' at '../wt-b'. Output ONLY the commands to run, one per line.
Expected
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b

Scoped model quality, cost, API time, and token usage for worktree_usage/f007.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high__json_schema PASS 100% 409 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-fable-5:low PASS 100% 167 in → 37 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-fable-5:low__json_schema PASS 100% 405 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-fable-5:max PASS 100% 170 in → 2,083 out (287 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-fable-5:max__json_schema PASS 100% 406 in → 163 out (40 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-fable-5:medium PASS 100% 171 in → 37 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 404 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-fable-5:xhigh PASS 100% 170 in → 106 out (52 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 400 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-haiku-4.5:high PASS 100% 171 in → 219 out (163 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 359 in → 361 out (236 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-haiku-4.5:low PASS 100% 177 in → 173 out (99 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 360 in → 711 out (473 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 360 in → 422 out (337 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-haiku-4.5:none PASS 100% 142 in → 34 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 328 in → 37 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-haiku-4.5:xhigh PASS 100% 174 in → 174 out (104 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 358 in → 341 out (262 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.6:high PASS 100% 147 in → 91 out (32 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 328 in → 93 out (28 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.6:low PASS 100% 146 in → 34 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 331 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.6:max PASS 100% 142 in → 107 out (42 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 332 in → 110 out (40 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.6:medium PASS 100% 142 in → 35 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 334 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.6:none PASS 100% 144 in → 34 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 335 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.6:xhigh PASS 100% 156 in → 91 out (28 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 333 in → 93 out (28 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.7:high PASS 100% 178 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.7:high__json_schema PASS 100% 411 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.7:low PASS 100% 178 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.7:low__json_schema PASS 100% 408 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.7:max__json_schema PASS 100% 404 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.7:medium PASS 100% 173 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 408 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.7:none PASS 100% 175 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.7:none__json_schema PASS 100% 407 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.7:xhigh PASS 100% 180 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 411 in → 44 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.8:high PASS 100% 175 in → 37 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 409 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.8:low PASS 100% 170 in → 33 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.8:low__json_schema PASS 100% 405 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.8:max__json_schema PASS 100% 403 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.8:medium PASS 100% 169 in → 33 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 407 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.8:none PASS 100% 171 in → 37 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 406 in → 41 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-opus-4.8:xhigh PASS 100% 176 in → 33 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 401 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-4.6:high PASS 100% 146 in → 85 out (24 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 330 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-4.6:low PASS 100% 146 in → 72 out (18 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 329 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-4.6:max PASS 100% 143 in → 153 out (77 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 332 in → 101 out (39 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-4.6:medium PASS 100% 146 in → 85 out (24 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 329 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-4.6:none PASS 100% 144 in → 34 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 330 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 144 in → 85 out (24 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 331 in → 36 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-5:high__json_schema PASS 100% 409 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-5:low__json_schema PASS 100% 407 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-5:medium PASS 100% 168 in → 37 out (0 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 407 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-5:none__json_schema PASS 100% 405 in → 41 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 405 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
arcee-ai/trinity-large-thinking:high PASS 100% 117 in → 425 out (406 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-large-thinking:low PASS 100% 114 in → 1,597 out (1,594 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-large-thinking:medium PASS 100% 115 in → 793 out (781 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-large-thinking:xhigh PASS 100% 117 in → 667 out (651 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-mini:low PASS 100% 119 in → 451 out (464 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-mini:low__json_schema PASS 100% 119 in → 474 out (477 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
arcee-ai/trinity-mini:medium PASS 100% 121 in → 433 out (455 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-mini:medium__json_schema PASS 100% 120 in → 645 out (641 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
arcee-ai/trinity-mini:xhigh PASS 100% 125 in → 421 out (403 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 120 in → 469 out (466 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash-0731:high PASS 100% 120 in → 452 out (434 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash-0731:low PASS 100% 123 in → 578 out (541 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 115 in → 346 out (286 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 123 in → 361 out (345 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash-0731:none PASS 100% 125 in → 20 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 131 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 124 in → 288 out (273 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 124 in → 162 out (133 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash:high PASS 100% 124 in → 310 out (289 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash:low PASS 100% 121 in → 389 out (368 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 126 in → 210 out (175 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash:medium PASS 100% 120 in → 628 out (609 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 121 in → 341 out (306 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash:none PASS 100% 135 in → 19 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 125 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-flash:xhigh PASS 100% 199 in → 577 out (556 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 204 in → 1,257 out (1,128 reasoning)
cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_qga3_tpd/worktree_usage_f007 && git worktree add ../wt-a feature-a cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_qga3_tpd/worktree_usage_f007 && git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_qga3_tpd/worktree_usage_f007 && git worktree add ../wt-a feature-a", "cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_qga3_tpd/worktree_usage_f007 && git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-pro:high PASS 100% 121 in → 443 out (422 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-pro:low PASS 100% 126 in → 381 out (360 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 123 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 326 in → 255 out (228 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
deepseek/deepseek-v4-pro:xhigh PASS 100% 203 in → 860 out (839 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 123 in → 28 out (27 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"] }
google/gemini-3-flash-preview:high__json_schema PASS 100% 152 in → 520 out (493 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3-flash-preview:low PASS 100% 147 in → 208 out (185 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3-flash-preview:low__json_schema PASS 100% 228 in → 228 out (201 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3-flash-preview:medium PASS 100% 147 in → 216 out (193 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3-flash-preview:medium__json_schema PASS 100% 228 in → 269 out (240 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
google/gemini-3-flash-preview:xhigh PASS 100% 150 in → 212 out (189 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 227 in → 192 out (165 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 149 in → 404 out (378 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.1-flash-lite-preview:low PASS 100% 145 in → 155 out (132 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 151 in → 154 out (112 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 226 in → 219 out (190 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 152 in → 280 out (251 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
google/gemini-3.1-pro-preview:high PASS 100% 149 in → 263 out (240 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 227 in → 1,612 out (1,586 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.1-pro-preview:low PASS 100% 151 in → 425 out (402 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 231 in → 414 out (388 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.1-pro-preview:medium PASS 100% 147 in → 1,028 out (1,005 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.1-pro-preview:xhigh PASS 100% 148 in → 973 out (950 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 225 in → 1,227 out (1,200 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.5-flash-lite:low PASS 100% 144 in → 23 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 225 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemini-3.5-flash-lite:max PASS 100% 147 in → 1,344 out (1,321 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 228 in → 714 out (687 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 224 in → 946 out (919 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.5-flash:high__json_schema PASS 100% 226 in → 783 out (756 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.5-flash:low__json_schema PASS 100% 228 in → 121 out (92 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 230 in → 459 out (432 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.6-flash:high PASS 100% 150 in → 767 out (745 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.6-flash:high__json_schema PASS 100% 227 in → 390 out (363 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.6-flash:low PASS 100% 150 in → 112 out (89 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.6-flash:low__json_schema PASS 100% 227 in → 42 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemini-3.6-flash:max PASS 100% 149 in → 810 out (787 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.6-flash:max__json_schema PASS 100% 152 in → 700 out (673 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.6-flash:medium PASS 100% 151 in → 356 out (333 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.6-flash:medium__json_schema PASS 100% 228 in → 233 out (206 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemini-3.6-flash:xhigh PASS 100% 148 in → 324 out (301 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 221 in → 405 out (378 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 167 in → 902 out (719 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemma-4-26b-a4b-it:low PASS 100% 162 in → 631 out (607 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-26b-a4b-it:medium PASS 100% 165 in → 870 out (691 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 165 in → 436 out (307 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemma-4-26b-a4b-it:none PASS 100% 164 in → 24 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 160 in → 30 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
google/gemma-4-26b-a4b-it:xhigh PASS 100% 165 in → 617 out (444 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 166 in → 442 out (336 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemma-4-31b-it:high PASS 100% 168 in → 620 out (535 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-31b-it:high__json_schema PASS 100% 162 in → 804 out (617 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemma-4-31b-it:low PASS 100% 162 in → 226 out (173 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-31b-it:low__json_schema PASS 100% 167 in → 194 out (116 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemma-4-31b-it:medium PASS 100% 164 in → 375 out (303 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-31b-it:medium__json_schema PASS 100% 166 in → 317 out (239 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
google/gemma-4-31b-it:none PASS 100% 161 in → 24 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-31b-it:xhigh PASS 100% 162 in → 215 out (152 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 165 in → 1,473 out (1,273 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
ibm-granite/granite-4.1-8b__json_schema PASS 100% 138 in → 34 out
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
inclusionai/ling-2.6-flash PASS 100% 158 in → 25 out
```sh git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
inclusionai/ling-2.6-flash__json_schema PASS 100% 155 in → 40 out
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m2.5:high PASS 100% 151 in → 818 out (806 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.5:high__json_schema PASS 100% 154 in → 619 out (585 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m2.5:low PASS 100% 147 in → 323 out (320 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.5:low__json_schema PASS 100% 153 in → 213 out (186 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m2.5:medium PASS 100% 150 in → 233 out (212 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.5:xhigh PASS 100% 147 in → 284 out (274 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.7:high PASS 100% 132 in → 350 out (272 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.7:high__json_schema PASS 100% 299 in → 797 out (762 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m2.7:low PASS 100% 149 in → 1,290 out (1,270 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.7:low__json_schema PASS 100% 298 in → 399 out (371 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
minimax/minimax-m2.7:medium PASS 100% 149 in → 427 out (439 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.7:medium__json_schema PASS 100% 296 in → 544 out (509 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m2.7:xhigh PASS 100% 144 in → 501 out (523 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 148 in → 372 out (348 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m3:high__json_schema PASS 100% 275 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m3:low PASS 100% 285 in → 138 out (119 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m3:low__json_schema PASS 100% 276 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m3:medium PASS 100% 286 in → 124 out (101 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
minimax/minimax-m3:medium__json_schema PASS 100% 272 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
minimax/minimax-m3:xhigh__json_schema PASS 100% 274 in → 35 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
mistralai/devstral-2512__json_schema PASS 100% 145 in → 34 out
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
mistralai/mistral-medium-3-5:high PASS 100% 159 in → 537 out (563 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 142 in → 329 out (333 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
mistralai/mistral-medium-3-5:low PASS 100% 159 in → 575 out (596 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
mistralai/mistral-medium-3-5:medium PASS 100% 158 in → 649 out (575 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 145 in → 221 out (201 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
mistralai/mistral-medium-3-5:none PASS 100% 158 in → 25 out (0 reasoning)
```sh git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 144 in → 27 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
mistralai/mistral-medium-3-5:xhigh PASS 100% 158 in → 1,205 out (1,266 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k2.6:high PASS 100% 115 in → 4,993 out (4,972 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k2.6:high__json_schema PASS 100% 115 in → 967 out (936 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
moonshotai/kimi-k2.6:low__json_schema PASS 100% 116 in → 4,895 out (4,671 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
moonshotai/kimi-k2.6:medium PASS 100% 113 in → 2,560 out (2,751 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 116 in → 1,724 out (1,623 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
moonshotai/kimi-k2.7-code:high PASS 100% 115 in → 528 out (507 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 210 in → 376 out (350 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
moonshotai/kimi-k2.7-code:medium PASS 100% 117 in → 1,385 out (1,364 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 209 in → 124 out (96 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
moonshotai/kimi-k2.7-code:xhigh PASS 100% 121 in → 288 out (267 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 115 in → 2,055 out (1,936 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_t4aku5xl/worktree_usage_f007 worktree add ../wt-a feature-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_t4aku5xl/worktree_usage_f007 worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_t4aku5xl/worktree_usage_f007 worktree add ../wt-a feature-a", "git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_t4aku5xl/worktree_usage_f007 worktree add ../wt-b feature-b" ] }
moonshotai/kimi-k3:high PASS 100% 193 in → 2,187 out (2,153 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k3:medium__json_schema PASS 100% 195 in → 220 out (186 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 301 in → 185 out (144 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 163 in → 155 out (140 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 163 in → 243 out (241 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 154 in → 344 out (320 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 156 in → 391 out (370 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 159 in → 244 out (246 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 158 in → 340 out (306 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 158 in → 40 out (0 reasoning)
git worktree add ../wt-a feature-a && cd ../wt-a && git checkout feature-a git worktree add ../wt-b feature-b && cd ../wt-b && git checkout feature-b
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 156 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 161 in → 279 out (290 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 157 in → 392 out (318 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 157 in → 378 out (335 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 162 in → 207 out (170 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 163 in → 359 out (316 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 163 in → 227 out (190 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 157 in → 191 out (175 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 156 in → 188 out (149 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 162 in → 20 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 159 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 156 in → 280 out (275 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 160 in → 110 out (74 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
openai/gpt-5.4-mini:none__json_schema PASS 100% 162 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.4-nano:high PASS 100% 115 in → 195 out (170 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.4-nano:high__json_schema PASS 100% 167 in → 174 out (140 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.4-nano:low__json_schema PASS 100% 167 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.4-nano:medium PASS 100% 123 in → 127 out (102 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.4-nano:medium__json_schema PASS 100% 162 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.4-nano:none PASS 100% 123 in → 23 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.4-nano:none__json_schema PASS 100% 165 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.4-nano:xhigh PASS 100% 122 in → 504 out (479 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 159 in → 1,146 out (1,112 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.5:high PASS 100% 122 in → 541 out (516 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.5:high__json_schema PASS 100% 164 in → 296 out (262 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.5:low__json_schema PASS 100% 163 in → 100 out (66 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.5:medium PASS 100% 118 in → 130 out (105 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.5:none PASS 100% 118 in → 23 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.5:none__json_schema PASS 100% 168 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.5:xhigh PASS 100% 118 in → 541 out (516 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.5:xhigh__json_schema PASS 100% 168 in → 550 out (516 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.6-luna:max__json_schema PASS 100% 162 in → 550 out (516 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.6-luna:none__json_schema PASS 100% 166 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.6-sol:none PASS 100% 121 in → 23 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-5.6-sol:none__json_schema PASS 100% 163 in → 32 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-5.6-terra:high PASS 100% 120 in → 141 out (116 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-oss-120b:high PASS 100% 184 in → 962 out (1,007 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-oss-120b:low PASS 100% 174 in → 64 out (39 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-oss-120b:low__json_schema PASS 100% 180 in → 76 out (43 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
openai/gpt-oss-120b:medium PASS 100% 163 in → 167 out (155 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-oss-120b:medium__json_schema PASS 100% 180 in → 178 out (156 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-oss-120b:xhigh PASS 100% 163 in → 449 out (454 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 178 in → 1,261 out (1,227 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
openai/gpt-oss-20b:high__json_schema PASS 100% 176 in → 660 out (624 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
openai/gpt-oss-20b:low PASS 100% 167 in → 67 out (40 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
openai/gpt-oss-20b:low__json_schema PASS 100% 178 in → 25 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
openai/gpt-oss-20b:medium PASS 100% 181 in → 233 out (206 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-m.1:high PASS 100% 144 in → 477 out (455 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-m.1:high__json_schema PASS 100% 147 in → 436 out (410 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-m.1:low PASS 100% 139 in → 276 out (254 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-m.1:low__json_schema PASS 100% 137 in → 681 out (645 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
poolside/laguna-m.1:medium PASS 100% 143 in → 330 out (310 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-m.1:medium__json_schema PASS 100% 146 in → 327 out (288 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
poolside/laguna-m.1:none PASS 100% 145 in → 21 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-m.1:none__json_schema PASS 100% 143 in → 39 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
poolside/laguna-m.1:xhigh PASS 100% 146 in → 545 out (524 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-m.1:xhigh__json_schema PASS 100% 146 in → 360 out (333 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs-2.1:high PASS 100% 145 in → 519 out (500 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs-2.1:high__json_schema PASS 100% 145 in → 429 out (404 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs-2.1:low PASS 100% 147 in → 389 out (370 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs-2.1:low__json_schema PASS 100% 141 in → 374 out (348 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs-2.1:medium PASS 100% 145 in → 475 out (457 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 148 in → 341 out (315 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs-2.1:none PASS 100% 145 in → 40 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b </think>git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs-2.1:none__json_schema PASS 100% 151 in → 39 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
poolside/laguna-xs-2.1:xhigh PASS 100% 149 in → 454 out (435 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 140 in → 352 out (326 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs.2:high PASS 100% 184 in → 154 out (132 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs.2:high__json_schema PASS 100% 184 in → 191 out (152 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
poolside/laguna-xs.2:low PASS 100% 180 in → 491 out (470 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs.2:low__json_schema PASS 100% 183 in → 193 out (167 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs.2:medium PASS 100% 182 in → 179 out (158 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs.2:medium__json_schema PASS 100% 186 in → 156 out (116 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
poolside/laguna-xs.2:none PASS 100% 187 in → 21 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs.2:none__json_schema PASS 100% 184 in → 26 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
poolside/laguna-xs.2:xhigh PASS 100% 187 in → 167 out (146 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 184 in → 582 out (556 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
qwen/qwen3.6-27b:high PASS 100% 143 in → 1,102 out (995 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-27b:high__json_schema PASS 100% 141 in → 1,196 out (1,111 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-27b:low PASS 100% 149 in → 1,239 out (1,137 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-27b:medium PASS 100% 147 in → 1,160 out (1,078 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-27b:medium__json_schema PASS 100% 150 in → 1,475 out (1,348 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-27b:none PASS 100% 147 in → 20 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-27b:none__json_schema PASS 100% 148 in → 44 out (0 reasoning)
git worktree add --detach ../wt-a feature-a git worktree add --detach ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add --detach ../wt-a feature-a", "git worktree add --detach ../wt-b feature-b" ] }
qwen/qwen3.6-27b:xhigh PASS 100% 146 in → 1,847 out (1,681 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 147 in → 1,560 out (1,377 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-35b-a3b:high PASS 100% 140 in → 825 out (801 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 145 in → 401 out (371 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-35b-a3b:low PASS 100% 149 in → 892 out (876 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 150 in → 188 out (151 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-35b-a3b:medium PASS 100% 146 in → 938 out (905 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 148 in → 1,478 out (1,383 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-35b-a3b:none PASS 100% 148 in → 20 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 148 in → 1,100 out (1,076 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 142 in → 1,347 out (1,248 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
qwen/qwen3.6-flash:high PASS 100% 147 in → 1,165 out (1,141 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-flash:low PASS 100% 144 in → 1,367 out (1,343 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-flash:medium PASS 100% 150 in → 366 out (342 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-flash:none PASS 100% 147 in → 19 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.6-flash:xhigh PASS 100% 146 in → 875 out (851 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-flash:high PASS 100% 145 in → 1,432 out (1,408 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-flash:low PASS 100% 146 in → 1,243 out (1,219 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-flash:max PASS 100% 148 in → 1,537 out (1,513 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-flash:medium PASS 100% 147 in → 1,233 out (1,209 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-flash:none PASS 100% 148 in → 19 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-flash:xhigh PASS 100% 141 in → 1,391 out (1,367 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-max:low PASS 100% 145 in → 1,674 out (1,650 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-max:none PASS 100% 146 in → 19 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-max:xhigh PASS 100% 145 in → 991 out (967 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-plus:high PASS 100% 151 in → 843 out (819 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-plus:low PASS 100% 147 in → 1,036 out (1,012 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-plus:medium PASS 100% 144 in → 993 out (969 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
qwen/qwen3.7-plus:none PASS 100% 148 in → 131 out (0 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_6jus9nm5/worktree_usage_f007 worktree add ../wt-a feature-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_6jus9nm5/worktree_usage_f007 worktree add ../wt-b feature-b
qwen/qwen3.7-plus:xhigh PASS 100% 148 in → 23 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
tencent/hy3:low__json_schema PASS 100% 127 in → 1,011 out (939 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
tencent/hy3:none PASS 100% 133 in → 20 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
tencent/hy3:none__json_schema PASS 100% 131 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling-small:high PASS 100% 129 in → 326 out (301 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling-small:high__json_schema PASS 100% 128 in → 268 out (229 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling-small:low PASS 100% 131 in → 50 out (25 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling-small:low__json_schema PASS 100% 125 in → 114 out (75 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling-small:max PASS 100% 129 in → 329 out (304 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling-small:max__json_schema PASS 100% 123 in → 495 out (456 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling-small:medium PASS 100% 127 in → 341 out (316 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling-small:medium__json_schema PASS 100% 125 in → 296 out (257 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling-small:xhigh PASS 100% 131 in → 201 out (172 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 129 in → 344 out (305 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling:high PASS 100% 125 in → 578 out (553 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling:high__json_schema PASS 100% 129 in → 534 out (495 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
thinkingmachines/inkling:low PASS 100% 133 in → 49 out (24 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling:max PASS 100% 130 in → 197 out (171 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling:medium PASS 100% 125 in → 183 out (157 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
thinkingmachines/inkling:medium__json_schema PASS 100% 124 in → 178 out (146 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"] }
thinkingmachines/inkling:xhigh PASS 100% 131 in → 150 out (125 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
x-ai/grok-4.3:high PASS 100% 294 in → 249 out (230 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
x-ai/grok-4.3:high__json_schema PASS 100% 357 in → 726 out (693 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
x-ai/grok-4.3:low PASS 100% 301 in → 147 out (128 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
x-ai/grok-4.3:low__json_schema PASS 100% 366 in → 326 out (293 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
x-ai/grok-4.3:max PASS 100% 288 in → 192 out (170 reasoning)
``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
x-ai/grok-4.3:max__json_schema PASS 100% 363 in → 505 out (472 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
x-ai/grok-4.3:medium PASS 100% 299 in → 841 out (822 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
x-ai/grok-4.3:medium__json_schema PASS 100% 366 in → 512 out (479 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
x-ai/grok-4.3:none PASS 100% 286 in → 19 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
x-ai/grok-4.3:none__json_schema PASS 100% 356 in → 23 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../wt-a feature-a","git worktree add ../wt-b feature-b"]}
x-ai/grok-4.3:xhigh PASS 100% 297 in → 183 out (164 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
x-ai/grok-4.3:xhigh__json_schema PASS 100% 363 in → 597 out (564 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
z-ai/glm-4.7-flash:high PASS 100% 121 in → 368 out (350 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-4.7-flash:low PASS 100% 120 in → 394 out (366 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-4.7-flash:medium PASS 100% 123 in → 482 out (451 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 124 in → 633 out (615 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
z-ai/glm-4.7-flash:none PASS 100% 119 in → 20 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-4.7-flash:none__json_schema PASS 100% 121 in → 27 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"]}
z-ai/glm-4.7-flash:xhigh PASS 100% 120 in → 347 out (311 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-5.2:high PASS 100% 129 in → 288 out (254 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-5.2:low PASS 100% 131 in → 756 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
z-ai/glm-5.2:low__json_schema PASS 100% 131 in → 1,710 out (1,679 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
z-ai/glm-5.2:medium__json_schema PASS 100% 126 in → 260 out (199 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
z-ai/glm-5.2:none__json_schema PASS 100% 123 in → 34 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ]}
thinkingmachines/inkling-small:none FAIL 50% 131 in → 22 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b<|end_message|>
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
thinkingmachines/inkling:none FAIL 50% 128 in → 22 out (0 reasoning)
git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b<|end_message|>
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-fable-5:high FAIL 0% 170 in → 239 out (76 reasoning)
``` git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-haiku-4.5:medium FAIL 0% 179 in → 453 out (356 reasoning)
``` git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-opus-4.7:max FAIL 0% 174 in → 44 out (0 reasoning)
``` git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-opus-4.8:max FAIL 0% 175 in → 291 out (50 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-sonnet-5:high FAIL 0% 171 in → 41 out (0 reasoning)
``` git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-sonnet-5:low FAIL 0% 174 in → 41 out (0 reasoning)
``` git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-sonnet-5:none FAIL 0% 172 in → 41 out (0 reasoning)
``` git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
anthropic/claude-sonnet-5:xhigh FAIL 0% 170 in → 108 out (53 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
arcee-ai/trinity-mini:high FAIL 0% 121 in → 480 out (486 reasoning)
gitworktree add ../wt-a feature-a git worktree add ../wt-b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
arcee-ai/trinity-mini:high__json_schema FAIL 0% 117 in → 405 out (391 reasoning)
gitworktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "gitworktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-flash-0731:high__json_schema FAIL 0% 200 in → 897 out (827 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-flash-0731:max FAIL 0% 212 in → 14,386 out (14,362 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-flash-0731:max__json_schema FAIL 0% 202 in → 424 out (364 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-flash-0731:medium FAIL 0% 125 in → 261 out (237 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
deepseek/deepseek-v4-pro:high__json_schema FAIL 0% 320 in → 911 out (879 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-pro:medium FAIL 0% 126 in → 715 out (690 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-pro:none FAIL 0% 123 in → 24 out (0 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
deepseek/deepseek-v4-pro:none__json_schema FAIL 0% 122 in → 39 out (0 reasoning)
git worktree add -b feature-a ../wt-a main git worktree add -b feature-b ../wt-b main
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a main", "git worktree add -b feature-b ../wt-b main" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3-flash-preview:high FAIL 0% 148 in → 1,223 out (1,196 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.1-flash-lite-preview:high FAIL 0% 151 in → 624 out (597 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.1-flash-lite-preview:medium FAIL 0% 147 in → 321 out (294 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.1-flash-lite-preview:xhigh FAIL 0% 150 in → 821 out (794 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.1-pro-preview:medium__json_schema FAIL 0% 228 in → 566 out (535 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.1-pro-preview:xhigh__json_schema FAIL 0% 229 in → 613 out (582 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash-lite:high FAIL 0% 146 in → 1,342 out (1,315 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash-lite:max__json_schema FAIL 0% 226 in → 887 out (854 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash-lite:medium FAIL 0% 152 in → 331 out (304 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash-lite:xhigh FAIL 0% 148 in → 1,054 out (1,027 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash:high FAIL 0% 150 in → 1,210 out (1,183 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash:low FAIL 0% 148 in → 466 out (439 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash:medium FAIL 0% 150 in → 454 out (427 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash:medium__json_schema FAIL 0% 221 in → 1,079 out (1,033 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemini-3.5-flash:xhigh FAIL 0% 150 in → 1,445 out (1,418 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemma-4-26b-a4b-it:high FAIL 0% 168 in → 1,639 out (1,386 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemma-4-26b-a4b-it:low__json_schema FAIL 0% 167 in → 1,620 out (1,356 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
google/gemma-4-31b-it:none__json_schema FAIL 0% 165 in → 5 out (0 reasoning)
(empty output)
JSON Schema Structured Output
(raw) {"commands":[]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
ibm-granite/granite-4.1-8b FAIL 0% 139 in → 60 out
To create worktrees for the branches 'feature-a' at '../wt-a' and 'feature-b' at '../wt-b', use the following Git commands, one per line: ``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
liquid/lfm-2-24b-a2b FAIL 0% 137 in → 169 out
``` git worktree add /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_d5yf4kre/worktree_usage_f007 /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_d5yf4kre ``` ``` git worktree add /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_d5yf4kre/worktree_usage_f007_b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output: To create worktrees for both `feature-a` and `feature-b` in their respective directories, you can use the following Git commands: ```bash cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_r9_0qp_8/ git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ``` These commands will set up two separate worktree branches from the current branch (`main`) into the specified subdirectories.
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
minimax/minimax-m2.5:medium__json_schema FAIL 0% 149 in → 578 out (495 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a -b feature-a", "git worktree add ../wt-b -b feature-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
minimax/minimax-m2.5:xhigh__json_schema FAIL 0% 147 in → 1,336 out (1,253 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
minimax/minimax-m3:high FAIL 0% 287 in → 430 out (433 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
minimax/minimax-m3:xhigh FAIL 0% 285 in → 63 out (33 reasoning)
git worktree add -b feature-a ../wt-a HEAD git worktree add -b feature-b ../wt-b HEAD
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
mistralai/devstral-2512 FAIL 0% 145 in → 34 out
Here are the commands to create the worktrees: ``` git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
mistralai/mistral-medium-3-5:low__json_schema FAIL 0% 145 in → 1,816 out (1,825 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
mistralai/mistral-medium-3-5:xhigh__json_schema FAIL 0% 146 in → 907 out (901 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a -b feature-a", "git worktree add ../wt-b -b feature-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k2.6:low FAIL 0% 120 in → 5,005 out (5,390 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 2 (char 1)
moonshotai/kimi-k2.6:xhigh FAIL 0% 115 in → 3,798 out (4,057 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
moonshotai/kimi-k2.7-code:low FAIL 0% 124 in → 1,225 out (1,112 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_84jd3_7w/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_84jd3_7w/worktree_usage_f007 worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:high__json_schema FAIL 0% 193 in → 332 out (294 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:low FAIL 0% 193 in → 362 out (324 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:low__json_schema FAIL 0% 194 in → 831 out (793 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:max FAIL 0% 199 in → 221 out (183 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:max__json_schema FAIL 0% 194 in → 407 out (369 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:medium FAIL 0% 197 in → 250 out (212 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
moonshotai/kimi-k3:xhigh FAIL 0% 193 in → 449 out (411 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:high FAIL 0% 120 in → 1,364 out (1,333 reasoning)
git worktree add -b feature-a ../wt-a main git worktree add -b feature-b ../wt-b main
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:high__json_schema FAIL 0% 161 in → 556 out (516 reasoning)
git worktree add -b feature-a ../wt-a main git worktree add -b feature-b ../wt-b main
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a main","git worktree add -b feature-b ../wt-b main"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:low FAIL 0% 116 in → 65 out (36 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:low__json_schema FAIL 0% 162 in → 151 out (113 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:medium FAIL 0% 119 in → 263 out (234 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:medium__json_schema FAIL 0% 160 in → 250 out (210 reasoning)
git worktree add -b feature-a ../wt-a main git worktree add -b feature-b ../wt-b main
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a main","git worktree add -b feature-b ../wt-b main"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:none FAIL 0% 121 in → 27 out (0 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:xhigh FAIL 0% 122 in → 23,999 out (23,826 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_gx3fcngv/worktree_usage_f007 worktree add -b feature-a ../wt-a 2ba51ad24ea27e97125df3ea53e7b866b8d5e8f5 git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_gx3fcngv/worktree_usage_f007 worktree add -b feature-b ../wt-b 2ba51ad24ea27e97125df3ea53e7b866b8d5e8f5
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-mini:xhigh__json_schema FAIL 0% 171 in → 25,362 out (25,324 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4-nano:low FAIL 0% 122 in → 81 out (0 reasoning)
git worktree add -b feature-a ../wt-a e4548859b3b0d085fe7b9d29ca70a88c19a4b33d git worktree add -b feature-b ../wt-b e4548859b3b0d085fe7b9d29ca70a88c19a4b33d
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:high FAIL 0% 116 in → 418 out (303 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_zszouugb/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_zszouugb/worktree_usage_f007 worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:high__json_schema FAIL 0% 167 in → 260 out (222 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:low FAIL 0% 116 in → 154 out (33 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_7jzsjrby/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_7jzsjrby/worktree_usage_f007 worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:low__json_schema FAIL 0% 162 in → 88 out (50 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:medium FAIL 0% 120 in → 385 out (268 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_qwzyypel/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_qwzyypel/worktree_usage_f007 worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:medium__json_schema FAIL 0% 166 in → 383 out (251 reasoning)
git -C '/private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_q53q19uk/worktree_usage_f007' worktree add -b feature-a '../wt-a' main git -C '/private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_q53q19uk/worktree_usage_f007' worktree add -b feature-b '../wt-b' main
JSON Schema Structured Output
(raw) {"commands":["git -C '/private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_q53q19uk/worktree_usage_f007' worktree add -b feature-a '../wt-a' main","git -C '/private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_q53q19uk/worktree_usage_f007' worktree add -b feature-b '../wt-b' main"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:none FAIL 0% 120 in → 119 out (0 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_74w6euqf/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_74w6euqf/worktree_usage_f007 worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:none__json_schema FAIL 0% 164 in → 126 out (0 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_k2ct28p3/worktree_usage_f007 worktree add ../wt-a -b feature-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_k2ct28p3/worktree_usage_f007 worktree add ../wt-b -b feature-b
JSON Schema Structured Output
(raw) {"commands":["git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_k2ct28p3/worktree_usage_f007 worktree add ../wt-a -b feature-a","git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_k2ct28p3/worktree_usage_f007 worktree add ../wt-b -b feature-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:xhigh FAIL 0% 119 in → 547 out (516 reasoning)
git worktree add -b feature-a ../wt-a main git worktree add -b feature-b ../wt-b main
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.4:xhigh__json_schema FAIL 0% 164 in → 18,653 out (18,523 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_4bamcsmn/worktree_usage_f007 worktree add -b feature-a ../wt-a main git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_4bamcsmn/worktree_usage_f007 worktree add -b feature-b ../wt-b main
JSON Schema Structured Output
(raw) {"commands":["git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_4bamcsmn/worktree_usage_f007 worktree add -b feature-a ../wt-a main","git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_4bamcsmn/worktree_usage_f007 worktree add -b feature-b ../wt-b main"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.5:low FAIL 0% 119 in → 341 out (220 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_88mpe7d5/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_88mpe7d5/worktree_usage_f007 worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.5:medium__json_schema FAIL 0% 165 in → 464 out (426 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:high FAIL 0% 118 in → 114 out (85 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:high__json_schema FAIL 0% 161 in → 119 out (81 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:low FAIL 0% 117 in → 97 out (68 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:low__json_schema FAIL 0% 164 in → 87 out (47 reasoning)
git worktree add -b feature-a ../wt-a main git worktree add -b feature-b ../wt-b main
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a main","git worktree add -b feature-b ../wt-b main"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:max FAIL 0% 121 in → 315 out (286 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:medium FAIL 0% 114 in → 123 out (94 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:medium__json_schema FAIL 0% 159 in → 121 out (83 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:none FAIL 0% 120 in → 27 out (0 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:xhigh FAIL 0% 121 in → 173 out (144 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-luna:xhigh__json_schema FAIL 0% 163 in → 554 out (516 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:high FAIL 0% 115 in → 175 out (146 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:high__json_schema FAIL 0% 166 in → 156 out (118 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:low FAIL 0% 120 in → 90 out (61 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:low__json_schema FAIL 0% 164 in → 201 out (163 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:max FAIL 0% 120 in → 109 out (80 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:max__json_schema FAIL 0% 170 in → 379 out (341 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:medium FAIL 0% 124 in → 74 out (45 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:medium__json_schema FAIL 0% 163 in → 178 out (140 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:xhigh FAIL 0% 120 in → 176 out (147 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-sol:xhigh__json_schema FAIL 0% 161 in → 382 out (344 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:high__json_schema FAIL 0% 161 in → 95 out (57 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:low FAIL 0% 116 in → 85 out (56 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:low__json_schema FAIL 0% 163 in → 190 out (60 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_05d4u82g/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_05d4u82g/worktree_usage_f007 worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_05d4u82g/worktree_usage_f007 worktree add -b feature-a ../wt-a","git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_05d4u82g/worktree_usage_f007 worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:max FAIL 0% 125 in → 165 out (136 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:max__json_schema FAIL 0% 167 in → 219 out (181 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:medium FAIL 0% 116 in → 104 out (75 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:medium__json_schema FAIL 0% 161 in → 36 out (0 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:none FAIL 0% 115 in → 27 out (0 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:none__json_schema FAIL 0% 166 in → 36 out (0 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:xhigh FAIL 0% 116 in → 100 out (71 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-5.6-terra:xhigh__json_schema FAIL 0% 164 in → 251 out (127 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_gozdwult/worktree_usage_f007 worktree add -b feature-a ../wt-a git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_gozdwult/worktree_usage_f007 worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_gozdwult/worktree_usage_f007 worktree add -b feature-a ../wt-a","git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f007_gozdwult/worktree_usage_f007 worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-oss-120b:high__json_schema FAIL 0% 178 in → 1,384 out (1,360 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-oss-20b:high FAIL 0% 167 in → 713 out (740 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-oss-20b:medium__json_schema FAIL 0% 184 in → 291 out (271 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../wt-a -b feature-a", "git worktree add ../wt-b -b feature-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-oss-20b:xhigh FAIL 0% 179 in → 557 out (531 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
openai/gpt-oss-20b:xhigh__json_schema FAIL 0% 181 in → 1,434 out (1,470 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 0% 151 in → 29 out (0 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c911b205-bc2d-928c-81c7-439e91943475","request_id":"c911b205-bc2d-928c-81c7-439e91943475"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-ae79552b-958c-93cc-a528-fdaee7b27fa7","request_id":"ae79552b-958c-93cc-a528-fdaee7b27fa7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-154b2bff-f9a6-9863-a7b4-18cdaf3a9e17","request_id":"154b2bff-f9a6-9863-a7b4-18cdaf3a9e17"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-419c7dae-af35-94b7-892f-76683b14d7f9","request_id":"419c7dae-af35-94b7-892f-76683b14d7f9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9a19fd81-f93b-9c89-a48c-939a4a025493","request_id":"9a19fd81-f93b-9c89-a48c-939a4a025493"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-d5acf04d-953d-988f-b50a-6bfe110656a8"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-a3a91139-382b-97e6-88c1-bc1575644b81"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-df9ce7be-401e-93a0-bbb2-3204e39057d3"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-cba58fdf-9fdb-9bad-8dfc-d484f8141e97"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-f4282abf-9dbc-918b-9978-9fcf6f471530"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-71f3402e-1375-992c-8cec-ca530ba7226a"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
qwen/qwen3.7-max:high FAIL 0% 141 in → 1,422 out (1,394 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6c6c64f3-36df-9ba3-84f6-7e8c55a87e05","request_id":"6c6c64f3-36df-9ba3-84f6-7e8c55a87e05"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e97baf58-57dc-9afd-b5f4-c2ad963772f9","request_id":"e97baf58-57dc-9afd-b5f4-c2ad963772f9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
qwen/qwen3.7-max:medium FAIL 0% 145 in → 1,349 out (1,321 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-dd805e28-9776-98fd-bd9b-dd0f77567f55","request_id":"dd805e28-9776-98fd-bd9b-dd0f77567f55"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7544f893-6e05-9218-a655-4ec202521ec5","request_id":"7544f893-6e05-9218-a655-4ec202521ec5"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-001c3483-f5df-961b-b8d3-dec6de836d4c","request_id":"001c3483-f5df-961b-b8d3-dec6de836d4c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8db14760-e712-9764-a436-5a320ba1190c","request_id":"8db14760-e712-9764-a436-5a320ba1190c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-561c9bdc-49de-92ec-987e-05894e549ab0","request_id":"561c9bdc-49de-92ec-987e-05894e549ab0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-df384b78-4386-9222-b3af-fcc70f5f19c7","request_id":"df384b78-4386-9222-b3af-fcc70f5f19c7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-465bed2b-545f-9a3e-b3b4-bb2163870672","request_id":"465bed2b-545f-9a3e-b3b4-bb2163870672"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-966bd622-38a2-95ba-8587-72628cc0d302","request_id":"966bd622-38a2-95ba-8587-72628cc0d302"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
tencent/hy3:high FAIL 0% 125 in → 1,611 out (1,586 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
tencent/hy3:high__json_schema FAIL 0% 129 in → 2,545 out (2,431 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
tencent/hy3:low FAIL 0% 126 in → 1,747 out (1,722 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
tencent/hy3:medium FAIL 0% 126 in → 1,588 out (1,563 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
tencent/hy3:medium__json_schema FAIL 0% 127 in → 600 out (538 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
tencent/hy3:xhigh FAIL 0% 126 in → 1,923 out (1,898 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
tencent/hy3:xhigh__json_schema FAIL 0% 126 in → 1,485 out (1,381 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": ["git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b"] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b{ "commands": ["git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b"] }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b{ "commands": [ "git worktree add ../wt-a feature-a", "git worktree add ../wt-b feature-b" ] }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
thinkingmachines/inkling:xhigh__json_schema FAIL 0% 126 in → 408 out (366 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:high FAIL 0% 329 in → 931 out (904 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:high__json_schema FAIL 0% 411 in → 1,159 out (1,127 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:low FAIL 0% 329 in → 469 out (438 reasoning)
``` git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b ```
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:low__json_schema FAIL 0% 412 in → 542 out (500 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:max FAIL 0% 335 in → 1,093 out (1,066 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:max__json_schema FAIL 0% 408 in → 1,063 out (1,021 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:medium FAIL 0% 329 in → 715 out (688 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:medium__json_schema FAIL 0% 408 in → 1,397 out (1,355 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature-a ../wt-a", "git worktree add -b feature-b ../wt-b" ] }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:xhigh FAIL 0% 331 in → 781 out (754 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
x-ai/grok-4.5:xhigh__json_schema FAIL 0% 404 in → 672 out (640 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature-a ../wt-a","git worktree add -b feature-b ../wt-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-5.2:medium FAIL 0% 131 in → 92 out (67 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
z-ai/glm-5.2:none FAIL 0% 126 in → 32 out (0 reasoning)
git branch feature-a main git branch feature-b main git worktree add ../wt-a feature-a git worktree add ../wt-b feature-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
z-ai/glm-5.2:xhigh FAIL 0% 131 in → 1,560 out (1,535 reasoning)
git worktree add -b feature-a ../wt-a git worktree add -b feature-b ../wt-b
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]
z-ai/glm-5.2:xhigh__json_schema FAIL 0% 128 in → 167 out (125 reasoning)
git worktree add ../wt-a -b feature-a git worktree add ../wt-b -b feature-b
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../wt-a -b feature-a", "git worktree add ../wt-b -b feature-b"]}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': '../wt-a'}, {'type': 'dir_exists', 'path': '../wt-b'}, {'type': 'file_content', 'path': '../wt-a/a.txt', 'value': 'A'}, {'type': 'file_content', 'path': '../wt-b/b.txt', 'value': 'B'}]