Make changes in worktree and commit
Tests ability to make changes in a worktree and commit them. Evaluates worktree-as-workspace workflow.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'Hello' > hello.txt
  5. 05 git add hello.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 git checkout -b feature
  8. 08 echo 'Feature' > feature.txt
  9. 09 git add feature.txt
  10. 10 git commit -m 'Add feature'
  11. 11 git checkout main
Prompt
Create a git worktree for the 'feature' branch at '../feature-wt'. Then write 'Updated' to the file '../feature-wt/feature.txt'. Output ONLY the commands to run, one per line.
Expected
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt

Scoped model quality, cost, API time, and token usage for worktree_usage/f002.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 184 in → 78 out (32 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-fable-5:high__json_schema PASS 100% 414 in → 78 out (26 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-fable-5:low PASS 100% 187 in → 43 out (0 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-fable-5:low__json_schema PASS 100% 421 in → 50 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-fable-5:max PASS 100% 182 in → 371 out (72 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-fable-5:max__json_schema PASS 100% 425 in → 275 out (73 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-fable-5:medium PASS 100% 185 in → 43 out (0 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 415 in → 50 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-fable-5:xhigh PASS 100% 185 in → 195 out (65 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 419 in → 117 out (31 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-haiku-4.5:high PASS 100% 178 in → 245 out (169 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 364 in → 1,299 out (737 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-haiku-4.5:low PASS 100% 183 in → 259 out (185 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 363 in → 607 out (528 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-haiku-4.5:medium PASS 100% 181 in → 469 out (386 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 363 in → 1,858 out (783 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-haiku-4.5:none PASS 100% 153 in → 34 out (0 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 336 in → 39 out (0 reasoning)
git worktree add ../feature-wt feature bash -c 'echo Updated > ../feature-wt/feature.txt'
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "bash -c 'echo Updated > ../feature-wt/feature.txt'"]}
anthropic/claude-haiku-4.5:xhigh PASS 100% 179 in → 253 out (179 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 364 in → 816 out (605 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.6:high PASS 100% 154 in → 34 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 337 in → 92 out (36 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.6:low PASS 100% 153 in → 33 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 336 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.6:max PASS 100% 152 in → 114 out (55 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 334 in → 114 out (56 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.6:medium PASS 100% 153 in → 34 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 336 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.6:none PASS 100% 153 in → 33 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 335 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.6:xhigh PASS 100% 153 in → 34 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 335 in → 92 out (36 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.7:high PASS 100% 187 in → 40 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
anthropic/claude-opus-4.7:high__json_schema PASS 100% 418 in → 47 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.7:low PASS 100% 190 in → 38 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
anthropic/claude-opus-4.7:low__json_schema PASS 100% 426 in → 54 out (0 reasoning)
git worktree add ../feature-wt feature sh -c "echo Updated > ../feature-wt/feature.txt"
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","sh -c \"echo Updated > ../feature-wt/feature.txt\""]}
anthropic/claude-opus-4.7:max PASS 100% 187 in → 139 out (26 reasoning)
``` git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 424 in → 47 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.7:medium PASS 100% 184 in → 38 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 421 in → 54 out (0 reasoning)
git worktree add ../feature-wt feature sh -c 'echo Updated > ../feature-wt/feature.txt'
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","sh -c 'echo Updated > ../feature-wt/feature.txt'"]}
anthropic/claude-opus-4.7:none PASS 100% 191 in → 40 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
anthropic/claude-opus-4.7:none__json_schema PASS 100% 429 in → 57 out (0 reasoning)
git worktree add ../feature-wt feature sh -c "echo 'Updated' > ../feature-wt/feature.txt"
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","sh -c \"echo 'Updated' > ../feature-wt/feature.txt\""]}
anthropic/claude-opus-4.7:xhigh PASS 100% 193 in → 40 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 427 in → 54 out (0 reasoning)
git worktree add ../feature-wt feature sh -c "echo Updated > ../feature-wt/feature.txt"
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","sh -c \"echo Updated > ../feature-wt/feature.txt\""]}
anthropic/claude-opus-4.8:high PASS 100% 184 in → 39 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
anthropic/claude-opus-4.8:high__json_schema PASS 100% 416 in → 48 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.8:low PASS 100% 183 in → 38 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
anthropic/claude-opus-4.8:low__json_schema PASS 100% 420 in → 48 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.8:max PASS 100% 183 in → 188 out (37 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 420 in → 206 out (65 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.8:medium PASS 100% 187 in → 38 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 418 in → 48 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.8:none PASS 100% 192 in → 42 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 414 in → 44 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
anthropic/claude-opus-4.8:xhigh PASS 100% 188 in → 42 out (0 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 418 in → 48 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-4.6:high PASS 100% 151 in → 105 out (51 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 334 in → 202 out (134 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-4.6:low PASS 100% 157 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 335 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-4.6:max PASS 100% 148 in → 427 out (166 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 333 in → 281 out (172 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-4.6:medium PASS 100% 149 in → 88 out (36 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 334 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-4.6:none PASS 100% 150 in → 34 out (0 reasoning)
``` git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 338 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 153 in → 230 out (133 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 339 in → 165 out (92 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
anthropic/claude-sonnet-5:high PASS 100% 183 in → 150 out (45 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 415 in → 153 out (56 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
arcee-ai/trinity-large-thinking:high PASS 100% 130 in → 1,101 out (1,101 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > '../feature-wt/feature.txt'
arcee-ai/trinity-large-thinking:low PASS 100% 129 in → 824 out (812 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
arcee-ai/trinity-large-thinking:medium PASS 100% 127 in → 409 out (390 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
arcee-ai/trinity-large-thinking:xhigh PASS 100% 128 in → 629 out (614 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
arcee-ai/trinity-mini:high__json_schema PASS 100% 127 in → 425 out (431 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
arcee-ai/trinity-mini:xhigh PASS 100% 124 in → 1,064 out (1,116 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash-0731:high PASS 100% 206 in → 572 out (585 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash-0731:low PASS 100% 130 in → 235 out (211 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash-0731:medium PASS 100% 129 in → 314 out (301 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 127 in → 401 out (377 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash-0731:none PASS 100% 129 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 128 in → 33 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands":[ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 216 in → 1,060 out (1,037 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 124 in → 439 out (404 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash:high PASS 100% 130 in → 316 out (321 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 135 in → 957 out (919 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash:low PASS 100% 132 in → 337 out (315 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 130 in → 466 out (428 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash:medium PASS 100% 131 in → 299 out (298 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 129 in → 378 out (340 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash:none PASS 100% 132 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 130 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-flash:xhigh PASS 100% 209 in → 469 out (445 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 286 in → 1,605 out (1,721 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-pro:high PASS 100% 132 in → 532 out (508 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 134 in → 87 out (0 reasoning)
cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_a2dde30b/worktree_usage_f002 git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_a2dde30b/worktree_usage_f002", "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-pro:low PASS 100% 134 in → 440 out (417 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
deepseek/deepseek-v4-pro:medium PASS 100% 129 in → 473 out (451 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 332 in → 285 out (254 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
deepseek/deepseek-v4-pro:none PASS 100% 128 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 127 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
deepseek/deepseek-v4-pro:xhigh PASS 100% 210 in → 1,121 out (1,105 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 208 in → 831 out (793 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemini-3-flash-preview:high PASS 100% 152 in → 250 out (229 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
google/gemini-3-flash-preview:high__json_schema PASS 100% 235 in → 515 out (488 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3-flash-preview:low PASS 100% 159 in → 325 out (302 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3-flash-preview:low__json_schema PASS 100% 157 in → 1,320 out (1,295 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3-flash-preview:medium PASS 100% 154 in → 412 out (391 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
google/gemini-3-flash-preview:medium__json_schema PASS 100% 235 in → 857 out (832 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3-flash-preview:xhigh PASS 100% 158 in → 332 out (309 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 232 in → 659 out (632 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3.1-flash-lite-preview:high PASS 100% 155 in → 584 out (561 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 154 in → 695 out (668 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3.1-flash-lite-preview:low PASS 100% 154 in → 183 out (162 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 229 in → 140 out (111 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 155 in → 327 out (304 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 155 in → 229 out (201 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 152 in → 502 out (479 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 158 in → 1,206 out (1,181 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3.1-pro-preview:high PASS 100% 155 in → 392 out (369 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 234 in → 1,250 out (1,208 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemini-3.1-pro-preview:low PASS 100% 152 in → 371 out (348 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 231 in → 361 out (334 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemini-3.1-pro-preview:medium PASS 100% 157 in → 667 out (644 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 233 in → 500 out (474 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3.1-pro-preview:xhigh PASS 100% 155 in → 484 out (461 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 231 in → 518 out (491 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash-lite:high PASS 100% 156 in → 559 out (536 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 156 in → 790 out (763 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash-lite:low PASS 100% 153 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 154 in → 14 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
google/gemini-3.5-flash-lite:max PASS 100% 158 in → 1,276 out (1,253 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 233 in → 1,455 out (1,428 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash-lite:medium PASS 100% 159 in → 541 out (518 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 235 in → 450 out (423 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash-lite:xhigh PASS 100% 153 in → 634 out (611 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 154 in → 746 out (721 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash:high PASS 100% 154 in → 1,022 out (999 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3.5-flash:high__json_schema PASS 100% 235 in → 833 out (804 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash:low PASS 100% 156 in → 278 out (255 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3.5-flash:low__json_schema PASS 100% 235 in → 378 out (351 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemini-3.5-flash:medium PASS 100% 154 in → 672 out (649 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.5-flash:medium__json_schema PASS 100% 229 in → 578 out (536 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
google/gemini-3.5-flash:xhigh PASS 100% 156 in → 504 out (481 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 234 in → 1,273 out (1,231 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
google/gemini-3.6-flash:high PASS 100% 155 in → 528 out (505 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.6-flash:high__json_schema PASS 100% 157 in → 761 out (736 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3.6-flash:low PASS 100% 155 in → 230 out (209 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
google/gemini-3.6-flash:low__json_schema PASS 100% 231 in → 40 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
google/gemini-3.6-flash:max PASS 100% 158 in → 720 out (697 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.6-flash:max__json_schema PASS 100% 235 in → 733 out (708 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3.6-flash:medium PASS 100% 158 in → 588 out (565 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.6-flash:medium__json_schema PASS 100% 157 in → 546 out (521 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
google/gemini-3.6-flash:xhigh PASS 100% 158 in → 601 out (578 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 156 in → 589 out (562 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemma-4-26b-a4b-it:high PASS 100% 174 in → 524 out (424 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 169 in → 415 out (275 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemma-4-26b-a4b-it:low PASS 100% 172 in → 626 out (490 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 170 in → 318 out (243 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemma-4-26b-a4b-it:medium PASS 100% 172 in → 467 out (401 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemma-4-26b-a4b-it:none PASS 100% 168 in → 24 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 167 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 172 in → 559 out (427 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 172 in → 444 out (301 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemma-4-31b-it:high PASS 100% 171 in → 232 out (175 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemma-4-31b-it:high__json_schema PASS 100% 171 in → 417 out (274 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemma-4-31b-it:low PASS 100% 175 in → 220 out (164 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemma-4-31b-it:low__json_schema PASS 100% 170 in → 274 out (191 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
google/gemma-4-31b-it:medium PASS 100% 172 in → 243 out (219 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
google/gemma-4-31b-it:medium__json_schema PASS 100% 169 in → 254 out (169 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
google/gemma-4-31b-it:none PASS 100% 164 in → 24 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
google/gemma-4-31b-it:none__json_schema PASS 100% 170 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
google/gemma-4-31b-it:xhigh PASS 100% 172 in → 253 out (191 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
ibm-granite/granite-4.1-8b__json_schema PASS 100% 145 in → 37 out
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
inclusionai/ling-2.6-flash PASS 100% 163 in → 21 out
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
inclusionai/ling-2.6-flash__json_schema PASS 100% 169 in → 43 out
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
minimax/minimax-m2.5:high PASS 100% 152 in → 1,044 out (1,103 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
minimax/minimax-m2.5:high__json_schema PASS 100% 157 in → 974 out (987 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
minimax/minimax-m2.5:low PASS 100% 155 in → 321 out (304 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m2.5:medium PASS 100% 173 in → 391 out (367 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
minimax/minimax-m2.5:medium__json_schema PASS 100% 169 in → 940 out (902 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
minimax/minimax-m2.5:xhigh PASS 100% 159 in → 1,191 out (1,169 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 158 in → 909 out (826 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
minimax/minimax-m2.7:high PASS 100% 158 in → 459 out (457 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m2.7:high__json_schema PASS 100% 308 in → 433 out (395 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
minimax/minimax-m2.7:low PASS 100% 159 in → 377 out (354 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m2.7:low__json_schema PASS 100% 310 in → 460 out (422 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
minimax/minimax-m2.7:medium PASS 100% 159 in → 598 out (575 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m2.7:medium__json_schema PASS 100% 310 in → 563 out (534 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
minimax/minimax-m2.7:xhigh PASS 100% 154 in → 384 out (363 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
minimax/minimax-m3:high PASS 100% 291 in → 146 out (128 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m3:high__json_schema PASS 100% 278 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
minimax/minimax-m3:low PASS 100% 298 in → 269 out (223 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m3:medium PASS 100% 295 in → 219 out (188 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
minimax/minimax-m3:medium__json_schema PASS 100% 282 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
minimax/minimax-m3:xhigh PASS 100% 296 in → 989 out (1,001 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
mistralai/devstral-2512 PASS 100% 153 in → 28 out
```bash git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
mistralai/devstral-2512__json_schema PASS 100% 155 in → 37 out
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
mistralai/mistral-medium-3-5:high PASS 100% 164 in → 621 out (640 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 153 in → 1,282 out (1,215 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt"]}
mistralai/mistral-medium-3-5:low PASS 100% 167 in → 979 out (1,027 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 150 in → 1,279 out (1,338 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
mistralai/mistral-medium-3-5:medium PASS 100% 166 in → 1,717 out (1,707 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 151 in → 796 out (783 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
mistralai/mistral-medium-3-5:none PASS 100% 164 in → 28 out (0 reasoning)
```bash git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 157 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
mistralai/mistral-medium-3-5:xhigh PASS 100% 165 in → 975 out (1,052 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 155 in → 1,524 out (1,635 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
moonshotai/kimi-k2.6:high PASS 100% 126 in → 1,121 out (1,100 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
moonshotai/kimi-k2.6:high__json_schema PASS 100% 124 in → 1,543 out (1,406 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
moonshotai/kimi-k2.6:low PASS 100% 123 in → 2,514 out (2,759 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
moonshotai/kimi-k2.6:low__json_schema PASS 100% 124 in → 1,849 out (1,819 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"] }
moonshotai/kimi-k2.6:medium PASS 100% 130 in → 1,670 out (1,647 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
moonshotai/kimi-k2.6:xhigh PASS 100% 127 in → 1,563 out (1,635 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 126 in → 81 out (0 reasoning)
cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_uqp_bdyu/worktree_usage_f002 git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_uqp_bdyu/worktree_usage_f002", "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
moonshotai/kimi-k2.7-code:high PASS 100% 122 in → 479 out (456 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 126 in → 352 out (315 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "printf 'Updated' > ../feature-wt/feature.txt" ] }
moonshotai/kimi-k2.7-code:low PASS 100% 123 in → 435 out (412 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
moonshotai/kimi-k2.7-code:medium PASS 100% 128 in → 302 out (281 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 125 in → 257 out (220 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
moonshotai/kimi-k3:high PASS 100% 200 in → 184 out (148 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
moonshotai/kimi-k3:high__json_schema PASS 100% 204 in → 185 out (151 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
moonshotai/kimi-k3:low PASS 100% 203 in → 237 out (198 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
moonshotai/kimi-k3:low__json_schema PASS 100% 203 in → 177 out (141 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
moonshotai/kimi-k3:max PASS 100% 205 in → 184 out (148 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
moonshotai/kimi-k3:max__json_schema PASS 100% 201 in → 158 out (122 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
moonshotai/kimi-k3:medium PASS 100% 204 in → 159 out (125 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
moonshotai/kimi-k3:medium__json_schema PASS 100% 203 in → 201 out (165 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
moonshotai/kimi-k3:xhigh PASS 100% 200 in → 120 out (84 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 200 in → 197 out (158 reasoning)
``` git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt ```
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 167 in → 576 out (612 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 168 in → 282 out (270 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 166 in → 302 out (314 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 166 in → 542 out (505 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 167 in → 357 out (365 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 167 in → 299 out (288 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 165 in → 21 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 168 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 167 in → 272 out (279 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 163 in → 469 out (420 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 167 in → 79 out (53 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 164 in → 204 out (166 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 167 in → 165 out (142 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 169 in → 327 out (273 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 167 in → 119 out (106 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 171 in → 202 out (164 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 167 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 171 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 168 in → 157 out (152 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 167 in → 242 out (203 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
openai/gpt-5.4-mini:high PASS 100% 128 in → 336 out (308 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4-mini:high__json_schema PASS 100% 176 in → 596 out (516 reasoning)
cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_zb9dflfr/worktree_usage_f002 && git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_zb9dflfr/worktree_usage_f002 && git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
openai/gpt-5.4-mini:low PASS 100% 129 in → 119 out (91 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4-mini:low__json_schema PASS 100% 180 in → 68 out (31 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4-mini:medium PASS 100% 129 in → 347 out (232 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_8k3th4v9/worktree_usage_f002 worktree add ../feature-wt feature echo 'Updated' > /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_8k3th4v9/feature-wt/feature.txt
openai/gpt-5.4-mini:medium__json_schema PASS 100% 170 in → 143 out (106 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4-mini:none PASS 100% 131 in → 26 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4-mini:none__json_schema PASS 100% 171 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4-mini:xhigh PASS 100% 130 in → 544 out (516 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 129 in → 544 out (516 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4-nano:high PASS 100% 127 in → 177 out (151 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-5.4-nano:high__json_schema PASS 100% 175 in → 138 out (103 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
openai/gpt-5.4-nano:low PASS 100% 129 in → 85 out (59 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-5.4-nano:low__json_schema PASS 100% 171 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature printf "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf \"Updated\" > ../feature-wt/feature.txt"]}
openai/gpt-5.4-nano:medium PASS 100% 126 in → 153 out (127 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-5.4-nano:medium__json_schema PASS 100% 172 in → 167 out (127 reasoning)
git worktree add ../feature-wt feature printf '%s' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4-nano:none PASS 100% 133 in → 69 out (0 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_j4goxme3/worktree_usage_f002 worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-5.4-nano:xhigh PASS 100% 129 in → 451 out (423 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 173 in → 429 out (392 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4:low__json_schema PASS 100% 172 in → 147 out (110 reasoning)
git worktree add -B feature ../feature-wt echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add -B feature ../feature-wt","echo Updated > ../feature-wt/feature.txt"]}
openai/gpt-5.4:medium PASS 100% 127 in → 254 out (194 reasoning)
git rev-parse --verify refs/heads/feature >/dev/null 2>&1 && git worktree add ../feature-wt feature || git worktree add -b feature ../feature-wt printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.4:medium__json_schema PASS 100% 174 in → 289 out (227 reasoning)
git show-ref --verify --quiet refs/heads/feature && git worktree add ../feature-wt feature || git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git show-ref --verify --quiet refs/heads/feature && git worktree add ../feature-wt feature || git worktree add -b feature ../feature-wt","echo 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4:none__json_schema PASS 100% 172 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.4:xhigh PASS 100% 131 in → 575 out (516 reasoning)
if git show-ref --verify --quiet refs/heads/feature; then git worktree add ../feature-wt feature; else git worktree add -b feature ../feature-wt HEAD; fi printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.4:xhigh__json_schema PASS 100% 175 in → 15,279 out (14,962 reasoning)
if git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/worktree_usage_f002 show-ref --verify --quiet refs/heads/feature; then git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/worktree_usage_f002 worktree add /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/feature-wt feature; else git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/worktree_usage_f002 worktree add -b feature /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/feature-wt HEAD; fi printf 'Updated' > /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["if git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/worktree_usage_f002 show-ref --verify --quiet refs/heads/feature; then git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/worktree_usage_f002 worktree add /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/feature-wt feature; else git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/worktree_usage_f002 worktree add -b feature /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/feature-wt HEAD; fi","printf 'Updated' > /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_c6w5_g76/feature-wt/feature.txt"]}
openai/gpt-5.5:high PASS 100% 126 in → 133 out (104 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.5:high__json_schema PASS 100% 169 in → 131 out (94 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.5:low PASS 100% 124 in → 27 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.5:low__json_schema PASS 100% 177 in → 79 out (42 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.5:medium PASS 100% 123 in → 119 out (90 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.5:medium__json_schema PASS 100% 173 in → 107 out (70 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.5:none PASS 100% 126 in → 27 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.5:none__json_schema PASS 100% 171 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.5:xhigh PASS 100% 130 in → 544 out (516 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.5:xhigh__json_schema PASS 100% 176 in → 555 out (516 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-luna:high PASS 100% 130 in → 92 out (64 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-luna:low PASS 100% 128 in → 71 out (42 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-luna:low__json_schema PASS 100% 172 in → 81 out (39 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-luna:max PASS 100% 130 in → 214 out (185 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-luna:max__json_schema PASS 100% 175 in → 1,061 out (1,024 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-luna:medium PASS 100% 130 in → 101 out (72 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-luna:medium__json_schema PASS 100% 175 in → 125 out (86 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-luna:none PASS 100% 127 in → 27 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-luna:none__json_schema PASS 100% 171 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-luna:xhigh PASS 100% 129 in → 229 out (201 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 177 in → 391 out (352 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-sol:high PASS 100% 126 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-sol:high__json_schema PASS 100% 174 in → 205 out (163 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-sol:low PASS 100% 127 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-sol:low__json_schema PASS 100% 171 in → 110 out (70 reasoning)
git worktree add ../feature-wt feature printf '%s' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-sol:max PASS 100% 126 in → 162 out (130 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-sol:max__json_schema PASS 100% 169 in → 219 out (180 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-sol:medium PASS 100% 128 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-sol:medium__json_schema PASS 100% 175 in → 99 out (57 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-sol:none PASS 100% 127 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-sol:none__json_schema PASS 100% 167 in → 40 out (0 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-sol:xhigh PASS 100% 127 in → 105 out (77 reasoning)
git worktree add ../feature-wt feature printf 'Updated' > ../feature-wt/feature.txt
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 171 in → 292 out (250 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-terra:high PASS 100% 130 in → 169 out (140 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-terra:high__json_schema PASS 100% 176 in → 167 out (84 reasoning)
cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_omai_pxq/worktree_usage_f002 && git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_omai_pxq/worktree_usage_f002 && git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-terra:low PASS 100% 129 in → 109 out (80 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-terra:low__json_schema PASS 100% 175 in → 239 out (155 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_x50p1ylv/worktree_usage_f002 worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_x50p1ylv/worktree_usage_f002 worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-terra:max__json_schema PASS 100% 174 in → 537 out (498 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-terra:medium__json_schema PASS 100% 175 in → 135 out (93 reasoning)
git worktree add ../feature-wt feature printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-terra:none PASS 100% 121 in → 27 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
openai/gpt-5.6-terra:none__json_schema PASS 100% 178 in → 37 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 174 in → 158 out (119 reasoning)
git worktree add ../feature-wt feature printf 'Updated\n' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\\n' > ../feature-wt/feature.txt"]}
openai/gpt-oss-120b:high PASS 100% 187 in → 626 out (630 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
openai/gpt-oss-120b:high__json_schema PASS 100% 189 in → 842 out (885 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
openai/gpt-oss-120b:low PASS 100% 187 in → 64 out (39 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-oss-120b:low__json_schema PASS 100% 186 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
openai/gpt-oss-120b:medium PASS 100% 179 in → 348 out (339 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
openai/gpt-oss-120b:medium__json_schema PASS 100% 173 in → 314 out (295 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-oss-20b:high PASS 100% 192 in → 394 out (397 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-oss-20b:high__json_schema PASS 100% 301 in → 737 out (698 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
openai/gpt-oss-20b:low PASS 100% 185 in → 59 out (31 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
openai/gpt-oss-20b:low__json_schema PASS 100% 185 in → 51 out (18 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
openai/gpt-oss-20b:medium PASS 100% 187 in → 492 out (470 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
openai/gpt-oss-20b:medium__json_schema PASS 100% 187 in → 266 out (229 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
openai/gpt-oss-20b:xhigh PASS 100% 185 in → 671 out (713 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 173 in → 652 out (675 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
poolside/laguna-m.1:high PASS 100% 151 in → 373 out (350 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-m.1:high__json_schema PASS 100% 155 in → 368 out (327 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
poolside/laguna-m.1:low PASS 100% 153 in → 909 out (886 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-m.1:low__json_schema PASS 100% 156 in → 354 out (313 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
poolside/laguna-m.1:medium PASS 100% 152 in → 661 out (640 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-m.1:medium__json_schema PASS 100% 153 in → 582 out (547 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ]}
poolside/laguna-m.1:none PASS 100% 155 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-m.1:none__json_schema PASS 100% 153 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"] }
poolside/laguna-m.1:xhigh PASS 100% 155 in → 475 out (452 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-m.1:xhigh__json_schema PASS 100% 151 in → 758 out (732 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt"]}
poolside/laguna-xs-2.1:high PASS 100% 155 in → 426 out (405 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
poolside/laguna-xs-2.1:high__json_schema PASS 100% 156 in → 423 out (382 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
poolside/laguna-xs-2.1:low PASS 100% 155 in → 497 out (476 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
poolside/laguna-xs-2.1:low__json_schema PASS 100% 156 in → 457 out (429 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
poolside/laguna-xs-2.1:medium PASS 100% 155 in → 492 out (472 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 146 in → 723 out (695 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
poolside/laguna-xs-2.1:none PASS 100% 154 in → 44 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt </think>git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
poolside/laguna-xs-2.1:none__json_schema PASS 100% 156 in → 28 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
poolside/laguna-xs-2.1:xhigh PASS 100% 153 in → 361 out (339 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 152 in → 326 out (298 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
poolside/laguna-xs.2:high PASS 100% 187 in → 186 out (162 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-xs.2:high__json_schema PASS 100% 192 in → 540 out (499 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
poolside/laguna-xs.2:low PASS 100% 192 in → 197 out (173 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-xs.2:low__json_schema PASS 100% 185 in → 212 out (183 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt"]}
poolside/laguna-xs.2:medium PASS 100% 197 in → 303 out (279 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-xs.2:medium__json_schema PASS 100% 191 in → 197 out (169 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt"]}
poolside/laguna-xs.2:none PASS 100% 194 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
poolside/laguna-xs.2:none__json_schema PASS 100% 191 in → 28 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
poolside/laguna-xs.2:xhigh PASS 100% 193 in → 509 out (486 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 190 in → 251 out (225 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
qwen/qwen3.6-27b:high__json_schema PASS 100% 156 in → 1,111 out (1,040 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-27b:low PASS 100% 156 in → 1,603 out (1,576 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.6-27b:low__json_schema PASS 100% 159 in → 1,593 out (1,459 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-27b:medium PASS 100% 157 in → 1,479 out (1,412 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-27b:none PASS 100% 157 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-27b:none__json_schema PASS 100% 161 in → 42 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-27b:xhigh PASS 100% 158 in → 1,886 out (1,764 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 157 in → 1,703 out (1 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-35b-a3b:high PASS 100% 155 in → 1,231 out (1,104 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 156 in → 1,103 out (1,026 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-35b-a3b:low PASS 100% 152 in → 1,242 out (1,247 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
qwen/qwen3.6-35b-a3b:medium PASS 100% 158 in → 1,682 out (1,576 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 152 in → 719 out (635 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-35b-a3b:none PASS 100% 161 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-35b-a3b:none__json_schema PASS 100% 162 in → 42 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 156 in → 1,330 out (1,246 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-flash:low PASS 100% 154 in → 1,114 out (1,087 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.6-flash:medium PASS 100% 158 in → 1,125 out (1,098 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.6-flash:xhigh PASS 100% 157 in → 2,334 out (2,309 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
qwen/qwen3.7-flash:high PASS 100% 152 in → 3,927 out (3,900 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-flash:low PASS 100% 156 in → 953 out (926 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-flash:max PASS 100% 159 in → 1,155 out (1,128 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-flash:medium PASS 100% 155 in → 1,042 out (1,015 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-flash:none PASS 100% 152 in → 20 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
qwen/qwen3.7-flash:xhigh PASS 100% 154 in → 1,309 out (1,282 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.7-max:high PASS 100% 153 in → 704 out (677 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.7-max:low PASS 100% 156 in → 501 out (474 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-max:medium PASS 100% 156 in → 850 out (823 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.7-max:none PASS 100% 156 in → 20 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
qwen/qwen3.7-max:xhigh PASS 100% 153 in → 813 out (786 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-plus:high PASS 100% 159 in → 1,269 out (1,242 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-plus:low PASS 100% 158 in → 922 out (895 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.7-plus:medium PASS 100% 157 in → 1,158 out (1,131 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
qwen/qwen3.7-plus:none PASS 100% 159 in → 22 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
qwen/qwen3.7-plus:xhigh PASS 100% 158 in → 661 out (634 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
tencent/hy3:high PASS 100% 137 in → 4,169 out (4,145 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
tencent/hy3:high__json_schema PASS 100% 136 in → 2,405 out (2,321 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
tencent/hy3:low PASS 100% 134 in → 472 out (450 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
tencent/hy3:low__json_schema PASS 100% 130 in → 943 out (893 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": ["git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt"] }
tencent/hy3:medium PASS 100% 133 in → 999 out (977 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
tencent/hy3:medium__json_schema PASS 100% 136 in → 888 out (823 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
tencent/hy3:none PASS 100% 138 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
tencent/hy3:none__json_schema PASS 100% 135 in → 32 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
tencent/hy3:xhigh__json_schema PASS 100% 135 in → 4,069 out (3,964 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling-small:high PASS 100% 138 in → 354 out (300 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
thinkingmachines/inkling-small:high__json_schema PASS 100% 141 in → 300 out (258 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling-small:low PASS 100% 134 in → 52 out (27 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
thinkingmachines/inkling-small:low__json_schema PASS 100% 137 in → 65 out (25 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling-small:max PASS 100% 137 in → 427 out (363 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
thinkingmachines/inkling-small:max__json_schema PASS 100% 137 in → 344 out (304 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling-small:medium PASS 100% 140 in → 220 out (200 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
thinkingmachines/inkling-small:medium__json_schema PASS 100% 134 in → 207 out (167 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling-small:none PASS 100% 134 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
thinkingmachines/inkling-small:xhigh PASS 100% 135 in → 351 out (307 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 134 in → 440 out (400 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling:high PASS 100% 138 in → 256 out (228 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
thinkingmachines/inkling:high__json_schema PASS 100% 143 in → 305 out (272 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": ["git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt"] }
thinkingmachines/inkling:low PASS 100% 131 in → 54 out (27 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
thinkingmachines/inkling:max PASS 100% 133 in → 323 out (294 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
thinkingmachines/inkling:max__json_schema PASS 100% 138 in → 347 out (306 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
thinkingmachines/inkling:medium PASS 100% 138 in → 246 out (218 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
thinkingmachines/inkling:none PASS 100% 136 in → 26 out (0 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
thinkingmachines/inkling:none__json_schema PASS 100% 133 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ]}
thinkingmachines/inkling:xhigh PASS 100% 138 in → 336 out (306 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
thinkingmachines/inkling:xhigh__json_schema PASS 100% 136 in → 227 out (185 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
x-ai/grok-4.3:high PASS 100% 299 in → 345 out (326 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
x-ai/grok-4.3:high__json_schema PASS 100% 366 in → 1,642 out (1,619 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
x-ai/grok-4.3:low PASS 100% 307 in → 521 out (500 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.3:low__json_schema PASS 100% 373 in → 699 out (674 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
x-ai/grok-4.3:max PASS 100% 301 in → 569 out (548 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.3:max__json_schema PASS 100% 366 in → 1,322 out (1,297 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo \"Updated\" > ../feature-wt/feature.txt"]}
x-ai/grok-4.3:medium PASS 100% 306 in → 1,313 out (1,292 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.3:medium__json_schema PASS 100% 373 in → 1,026 out (991 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
x-ai/grok-4.3:none PASS 100% 306 in → 19 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
x-ai/grok-4.3:none__json_schema PASS 100% 365 in → 23 out (0 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo Updated > ../feature-wt/feature.txt"]}
x-ai/grok-4.3:xhigh PASS 100% 297 in → 1,105 out (1,084 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.3:xhigh__json_schema PASS 100% 365 in → 703 out (668 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
x-ai/grok-4.5:high PASS 100% 336 in → 378 out (356 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.5:high__json_schema PASS 100% 423 in → 558 out (531 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
x-ai/grok-4.5:low PASS 100% 338 in → 219 out (197 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.5:low__json_schema PASS 100% 417 in → 476 out (449 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo 'Updated' > ../feature-wt/feature.txt"]}
x-ai/grok-4.5:max PASS 100% 341 in → 351 out (331 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
x-ai/grok-4.5:max__json_schema PASS 100% 411 in → 349 out (312 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
x-ai/grok-4.5:medium PASS 100% 342 in → 394 out (374 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
x-ai/grok-4.5:medium__json_schema PASS 100% 417 in → 661 out (624 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
x-ai/grok-4.5:xhigh PASS 100% 337 in → 422 out (400 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
x-ai/grok-4.5:xhigh__json_schema PASS 100% 414 in → 408 out (371 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
z-ai/glm-4.7-flash:high PASS 100% 132 in → 532 out (552 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > '../feature-wt/feature.txt'
z-ai/glm-4.7-flash:low PASS 100% 131 in → 362 out (329 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
z-ai/glm-4.7-flash:low__json_schema PASS 100% 132 in → 521 out (480 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
z-ai/glm-4.7-flash:medium PASS 100% 129 in → 845 out (857 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 128 in → 581 out (541 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
z-ai/glm-4.7-flash:none__json_schema PASS 100% 131 in → 30 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
z-ai/glm-4.7-flash:xhigh PASS 100% 129 in → 449 out (430 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 130 in → 638 out (594 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
z-ai/glm-5.2:high PASS 100% 140 in → 277 out (256 reasoning)
git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
z-ai/glm-5.2:high__json_schema PASS 100% 138 in → 757 out (715 reasoning)
git branch feature 2>/dev/null || true git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git branch feature 2>/dev/null || true", "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt"]}
z-ai/glm-5.2:low PASS 100% 136 in → 310 out (286 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
z-ai/glm-5.2:low__json_schema PASS 100% 136 in → 292 out (254 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
z-ai/glm-5.2:medium PASS 100% 139 in → 567 out (539 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
z-ai/glm-5.2:medium__json_schema PASS 100% 142 in → 77 out (43 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
z-ai/glm-5.2:none__json_schema PASS 100% 137 in → 35 out (0 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
z-ai/glm-5.2:xhigh PASS 100% 140 in → 640 out (594 reasoning)
git worktree add ../feature-wt feature echo "Updated" > ../feature-wt/feature.txt
z-ai/glm-5.2:xhigh__json_schema PASS 100% 132 in → 372 out (298 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
anthropic/claude-sonnet-5:low FAIL 0% 182 in → 68 out (27 reasoning)
``` git worktree add ../feature-wt feature echo -n "Updated" > ../feature-wt/feature.txt ```
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:low__json_schema FAIL 0% 416 in → 57 out (0 reasoning)
git worktree add ../feature-wt feature sh -c "echo -n 'Updated' > ../feature-wt/feature.txt"
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","sh -c \"echo -n 'Updated' > ../feature-wt/feature.txt\""]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:medium FAIL 0% 184 in → 72 out (35 reasoning)
git worktree add ../feature-wt feature echo -n "Updated" > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:medium__json_schema FAIL 0% 412 in → 50 out (0 reasoning)
git worktree add ../feature-wt feature echo -n 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo -n 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:none FAIL 0% 182 in → 44 out (0 reasoning)
``` git worktree add ../feature-wt feature echo -n "Updated" > ../feature-wt/feature.txt ```
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:none__json_schema FAIL 0% 416 in → 49 out (0 reasoning)
git worktree add ../feature-wt feature echo -n 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo -n 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:xhigh FAIL 0% 182 in → 170 out (31 reasoning)
git worktree add ../feature-wt feature echo -n "Updated" > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
anthropic/claude-sonnet-5:xhigh__json_schema FAIL 0% 420 in → 89 out (26 reasoning)
git worktree add ../feature-wt feature echo -n 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","echo -n 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
arcee-ai/trinity-mini:high FAIL 0% 127 in → 940 out (992 reasoning)
git worktreeadd ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
arcee-ai/trinity-mini:low FAIL 0% 129 in → 732 out (782 reasoning)
git worktreeadd -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
arcee-ai/trinity-mini:low__json_schema FAIL 0% 126 in → 737 out (746 reasoning)
git worktree add ../feature-wt feature cd ../feature-wt echo "Updated" > feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "cd ../feature-wt", "echo \"Updated\" > feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
arcee-ai/trinity-mini:medium FAIL 0% 132 in → 906 out (981 reasoning)
git worktree add -b feature ../feature-wt echo "Updated" > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
arcee-ai/trinity-mini:medium__json_schema FAIL 0% 127 in → 798 out (777 reasoning)
git worktree add ../feature-wt feature cd ../feature-wt echo "Updated" > feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "cd ../feature-wt", "echo \"Updated\" > feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
arcee-ai/trinity-mini:xhigh__json_schema FAIL 0% 129 in → 371 out (363 reasoning)
gitworktree add ../feature-wt feature cd ../feature-wt echo "Updated" > feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "gitworktree add ../feature-wt feature", "cd ../feature-wt", "echo \"Updated\" > feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
deepseek/deepseek-v4-flash-0731:high__json_schema FAIL 0% 132 in → 1,160 out (1,160 reasoning)
git worktree add -b feature ../feature-wt echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "echo \"Updated\" > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
deepseek/deepseek-v4-flash-0731:low__json_schema FAIL 0% 133 in → 903 out (895 reasoning)
git worktree add -b feature ../feature-wt echo "Updated" > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": ["git worktree add -b feature ../feature-wt", "echo \"Updated\" > ../feature-wt/feature.txt"] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
deepseek/deepseek-v4-flash-0731:max FAIL 0% 209 in → 2,355 out (2,330 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
deepseek/deepseek-v4-flash-0731:max__json_schema FAIL 0% 222 in → 2,698 out (2,737 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "printf 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
deepseek/deepseek-v4-pro:low__json_schema FAIL 0% 132 in → 40 out (0 reasoning)
git worktree add ../feature-wt feature cd ../feature-wt echo 'Updated' > feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add ../feature-wt feature", "cd ../feature-wt", "echo 'Updated' > feature.txt" ]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git work{ "commands": [ "git worktree add ../feature-wt feature", "echo 'Updated' > ../feature-wt/feature.txt" ] }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
ibm-granite/granite-4.1-8b FAIL 0% 145 in → 151 out
To accomplish the task described, you need to follow these steps using Git commands. The instructions assume you are starting from the root of your Git repository where you want to create the worktree. 1. **Create a new worktree for the 'feature' branch at '../feature-wt':** ```bash git worktree add ../feature-wt feature ``` 2. **Write 'Updated' to the file '../feature-wt/feature.txt':** ```bash echo 'Updated' > ../feature-wt/feature.txt ``` These commands will set up the worktree and update the specified file as requested. No additional setup is required beyond having Git installed and being in the correct repository directory.
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
liquid/lfm-2-24b-a2b FAIL 0% 143 in → 130 out
```bash git worktree add /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_svf60dm0/worktree_usage_f002 main cd /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_svf60dm0/worktree_usage_f002 git checkout feature echo "Updated" > feature.txt ```
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
Invalid JSON. Output: ```bash git worktree add /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_n2365rze/worktree_usage_f002 /private/path/to/your/project/feature echo "Updated" > /private/path/to/your/project/feature/feature.txt ``` **Note:** Replace `/private/path/to/your/project` with the actual path where your repository resides. The `HEAD 97c5e...` line is just informational and not part of the commands you need to run.
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
minimax/minimax-m2.5:low__json_schema FAIL 0% 158 in → 1,215 out (1,257 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "echo 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
minimax/minimax-m3:low__json_schema FAIL 0% 284 in → 40 out (0 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "echo 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
minimax/minimax-m3:xhigh__json_schema FAIL 0% 284 in → 40 out (0 reasoning)
git worktree add -b feature ../feature-wt main echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt main", "echo 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
Invalid JSON. Output: git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 2 (char 1)
moonshotai/kimi-k2.7-code:low__json_schema FAIL 0% 218 in → 520 out (488 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add -b feature ../feature-wt", "printf 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
moonshotai/kimi-k2.7-code:medium__json_schema FAIL 0% 220 in → 237 out (205 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands": ["git worktree add -b feature ../feature-wt", "printf 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
moonshotai/kimi-k2.7-code:xhigh FAIL 0% 127 in → 475 out (450 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.4-nano:none__json_schema FAIL 0% 168 in → 36 out (0 reasoning)
git worktree add ../feature-wt feature printf 'Updated ' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add ../feature-wt feature","printf 'Updated\n' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.4:high FAIL 0% 128 in → 367 out (337 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.4:high__json_schema FAIL 0% 179 in → 639 out (516 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_f8umjt_9/worktree_usage_f002 worktree add -b feature ../feature-wt printf 'Updated ' > /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_f8umjt_9/feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_f8umjt_9/worktree_usage_f002 worktree add -b feature ../feature-wt","printf 'Updated\n' > /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f002_f8umjt_9/feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.4:low FAIL 0% 128 in → 223 out (193 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.4:none FAIL 0% 129 in → 29 out (0 reasoning)
git worktree add ../feature-wt -b feature printf 'Updated\n' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.6-luna:high__json_schema FAIL 0% 174 in → 560 out (516 reasoning)
git worktree add -b feature ../feature-wt printf '%s\n' 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature ../feature-wt","printf '%s\\n' 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.6-terra:max FAIL 0% 125 in → 820 out (786 reasoning)
git worktree add -b feature ../feature-wt printf '%s\n' 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.6-terra:medium FAIL 0% 127 in → 127 out (96 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated\n' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-5.6-terra:xhigh FAIL 0% 126 in → 186 out (155 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated\n' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-oss-120b:xhigh FAIL 0% 191 in → 395 out (428 reasoning)
git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
openai/gpt-oss-120b:xhigh__json_schema FAIL 0% 191 in → 788 out (750 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) {"commands":["git worktree add -b feature ../feature-wt","echo 'Updated' > ../feature-wt/feature.txt"]}
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
qwen/qwen3.6-27b:high FAIL 0% 160 in → 1,557 out (1,411 reasoning)
git worktree add -b feature ../feature-wt echo "Updated" > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
qwen/qwen3.6-27b:medium__json_schema FAIL 0% 154 in → 2,158 out (1,975 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "echo 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
qwen/qwen3.6-35b-a3b:low__json_schema FAIL 0% 158 in → 1,195 out (1,143 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "echo 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
qwen/qwen3.6-35b-a3b:xhigh__json_schema FAIL 0% 156 in → 2,241 out (2,071 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
(raw) { "commands": [ "git worktree add -b feature ../feature-wt", "echo 'Updated' > ../feature-wt/feature.txt" ] }
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
qwen/qwen3.6-flash:high FAIL 0% 155 in → 1,709 out (1,680 reasoning)
git worktree add -b feature ../feature-wt echo 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8ff1e3ba-f6ea-908f-a10c-b2274d86f95a","request_id":"8ff1e3ba-f6ea-908f-a10c-b2274d86f95a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a041376e-f0ad-9f13-a03e-b250f708490f","request_id":"a041376e-f0ad-9f13-a03e-b250f708490f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-2eaf0cd4-e6cc-9587-8a88-82c2a2df3cb8","request_id":"2eaf0cd4-e6cc-9587-8a88-82c2a2df3cb8"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
qwen/qwen3.6-flash:none FAIL 0% 157 in → 24 out (0 reasoning)
git worktree add ../feature-wt feature echo -n 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-591131c9-5c3c-9bb0-a9f9-a8b91d8efaf9","request_id":"591131c9-5c3c-9bb0-a9f9-a8b91d8efaf9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4ea5742f-c0ab-9ba4-b3ad-7a094360b371","request_id":"4ea5742f-c0ab-9ba4-b3ad-7a094360b371"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-bce908aa-056d-90da-88c1-f36c99409d7e"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-b6dccd60-85a0-9938-87ec-4b2d5c84d68c"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-f492a579-0c91-9ac3-87f0-bb727f8287da"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-e78b2230-afb4-9180-9bf7-bb2180a282fb"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-1d9fcf50-ce41-9265-9c8c-73dc23179926"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-21f13a52-94ba-9160-818e-cc2a6536449e"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-ab0ea489-b7bf-9380-bdde-45ea608b0031","request_id":"ab0ea489-b7bf-9380-bdde-45ea608b0031"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b582213e-9ffe-928e-9bde-36b18f0bdd63","request_id":"b582213e-9ffe-928e-9bde-36b18f0bdd63"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-357f9c7f-6352-9b79-a9ee-1b4444293be6","request_id":"357f9c7f-6352-9b79-a9ee-1b4444293be6"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-21cb0a0b-c99a-9c7d-961b-e25d8a0ade7d","request_id":"21cb0a0b-c99a-9c7d-961b-e25d8a0ade7d"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-2f3cb876-cb80-9c33-b59b-db9af1d49a36","request_id":"2f3cb876-cb80-9c33-b59b-db9af1d49a36"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-dbd0856c-8780-9b95-84ef-116fd552034b","request_id":"dbd0856c-8780-9b95-84ef-116fd552034b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-97a151a7-d4b6-93bf-8e1c-3134567e8564","request_id":"97a151a7-d4b6-93bf-8e1c-3134567e8564"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0b5faad8-5d2c-9273-8273-4257da6f3fdf","request_id":"0b5faad8-5d2c-9273-8273-4257da6f3fdf"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0f5eee8f-9b9b-90b2-ab9a-a3f59e0fc477","request_id":"0f5eee8f-9b9b-90b2-ab9a-a3f59e0fc477"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f119863e-60d5-9c99-a876-84d35c17f489","request_id":"f119863e-60d5-9c99-a876-84d35c17f489"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
tencent/hy3:xhigh FAIL 0% 137 in → 13,685 out (13,659 reasoning)
git worktree add -b feature ../feature-wt printf 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
Invalid JSON. Output: git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt{ "commands": [ "git worktree add ../feature-wt feature", "echo Updated > ../feature-wt/feature.txt" ] }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../feature-wt feature echo 'Updated' > ../feature-wt/feature.txt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree add ../feature-wt feature echo Updated > ../feature-wt/feature.txt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-4.7-flash:none FAIL 0% 130 in → 14 out (0 reasoning)
cd "../feature-wt" && echo "Updated" > feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]
z-ai/glm-5.2:none FAIL 0% 130 in → 25 out (0 reasoning)
git worktree add ../feature-wt feature echo -n 'Updated' > ../feature-wt/feature.txt
Failure: Failed assertions: [{'type': 'file_content', 'path': '../feature-wt/feature.txt', 'value': 'Updated'}]