Unlock a worktree
Tests ability to unlock a previously locked worktree. Evaluates worktree-unlock workflow.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'Hello' > hello.txt
  5. 05 git add hello.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 git checkout -b feature
  8. 08 echo 'Feature' > feature.txt
  9. 09 git add feature.txt
  10. 10 git commit -m 'Add feature'
  11. 11 git checkout main
  12. 12 git worktree add ../feature-wt feature
  13. 13 git worktree lock ../feature-wt
Prompt
Unlock the worktree at '../feature-wt'. Output ONLY the git command, nothing else.
Expected
git worktree unlock ../feature-wt

Scoped model quality, cost, API time, and token usage for worktree_usage/f011.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 259 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-fable-5:high__json_schema PASS 100% 469 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-fable-5:low PASS 100% 251 in → 16 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-fable-5:low__json_schema PASS 100% 468 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-fable-5:max PASS 100% 258 in → 69 out (35 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-fable-5:max__json_schema PASS 100% 472 in → 94 out (40 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-fable-5:medium PASS 100% 253 in → 16 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-fable-5:medium__json_schema PASS 100% 465 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-fable-5:xhigh PASS 100% 256 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 463 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-haiku-4.5:high PASS 100% 251 in → 183 out (130 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 406 in → 215 out (162 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-haiku-4.5:low PASS 100% 245 in → 138 out (109 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 403 in → 126 out (97 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-haiku-4.5:medium PASS 100% 249 in → 158 out (128 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 410 in → 138 out (103 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-haiku-4.5:none PASS 100% 222 in → 13 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 375 in → 18 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 243 in → 185 out (127 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 406 in → 183 out (129 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.6:high PASS 100% 206 in → 17 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 378 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.6:low PASS 100% 218 in → 13 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-opus-4.6:low__json_schema PASS 100% 371 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.6:max PASS 100% 211 in → 18 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 376 in → 62 out (27 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.6:medium PASS 100% 216 in → 17 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 379 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.6:none PASS 100% 220 in → 17 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 379 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.6:xhigh PASS 100% 221 in → 17 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 380 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.7:high PASS 100% 262 in → 18 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-opus-4.7:high__json_schema PASS 100% 474 in → 25 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.7:low PASS 100% 260 in → 18 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-opus-4.7:low__json_schema PASS 100% 474 in → 25 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.7:max PASS 100% 267 in → 23 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 471 in → 25 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.7:medium PASS 100% 261 in → 18 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 474 in → 25 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.7:none PASS 100% 270 in → 20 out (0 reasoning)
`git worktree unlock ../feature-wt`
anthropic/claude-opus-4.7:none__json_schema PASS 100% 481 in → 25 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.7:xhigh PASS 100% 258 in → 18 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 469 in → 25 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.8:high PASS 100% 256 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 472 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.8:low PASS 100% 259 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 466 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.8:max PASS 100% 250 in → 68 out (31 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 464 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.8:medium PASS 100% 253 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 463 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.8:none PASS 100% 255 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 464 in → 22 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-opus-4.8:xhigh PASS 100% 255 in → 16 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 460 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-4.6:high PASS 100% 213 in → 15 out (0 reasoning)
`git worktree unlock ../feature-wt`
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 373 in → 68 out (32 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-4.6:low PASS 100% 214 in → 15 out (0 reasoning)
`git worktree unlock ../feature-wt`
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 376 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-4.6:max PASS 100% 211 in → 68 out (32 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 383 in → 160 out (93 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-4.6:medium PASS 100% 221 in → 15 out (0 reasoning)
`git worktree unlock ../feature-wt`
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 374 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-4.6:none PASS 100% 215 in → 15 out (0 reasoning)
`git worktree unlock ../feature-wt`
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 369 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 211 in → 15 out (0 reasoning)
`git worktree unlock ../feature-wt`
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 374 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-5:high PASS 100% 259 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 456 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-5:low PASS 100% 251 in → 16 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-sonnet-5:low__json_schema PASS 100% 464 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-5:medium PASS 100% 251 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 464 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-5:none PASS 100% 249 in → 16 out (0 reasoning)
git worktree unlock ../feature-wt
anthropic/claude-sonnet-5:none__json_schema PASS 100% 467 in → 22 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
anthropic/claude-sonnet-5:xhigh PASS 100% 249 in → 20 out (0 reasoning)
``` git worktree unlock ../feature-wt ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 457 in → 23 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
arcee-ai/trinity-large-thinking:high PASS 100% 178 in → 382 out (376 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-large-thinking:low PASS 100% 180 in → 535 out (539 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-large-thinking:medium PASS 100% 183 in → 713 out (721 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-large-thinking:xhigh PASS 100% 176 in → 566 out (570 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-mini:high PASS 100% 173 in → 301 out (344 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-mini:high__json_schema PASS 100% 182 in → 425 out (473 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
arcee-ai/trinity-mini:low__json_schema PASS 100% 175 in → 277 out (298 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
arcee-ai/trinity-mini:medium PASS 100% 177 in → 282 out (310 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-mini:medium__json_schema PASS 100% 176 in → 314 out (336 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
arcee-ai/trinity-mini:xhigh PASS 100% 176 in → 478 out (520 reasoning)
git worktree unlock ../feature-wt
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 174 in → 315 out (343 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 267 in → 91 out (82 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 187 in → 75 out (71 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 193 in → 154 out (151 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 188 in → 256 out (253 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash-0731:max PASS 100% 263 in → 74 out (70 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 268 in → 111 out (102 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 187 in → 51 out (42 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 192 in → 166 out (202 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 190 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 192 in → 13 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 276 in → 111 out (108 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 185 in → 219 out (202 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash:high PASS 100% 190 in → 84 out (76 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 192 in → 134 out (120 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash:low PASS 100% 187 in → 183 out (173 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 188 in → 229 out (198 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash:medium PASS 100% 190 in → 207 out (197 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash:none PASS 100% 184 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 188 in → 16 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 269 in → 123 out (113 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 269 in → 143 out (126 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-pro:high PASS 100% 187 in → 161 out (139 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 378 in → 113 out (98 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
deepseek/deepseek-v4-pro:low PASS 100% 189 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 381 in → 52 out (37 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 186 in → 123 out (106 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
deepseek/deepseek-v4-pro:none PASS 100% 184 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 373 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
deepseek/deepseek-v4-pro:xhigh PASS 100% 268 in → 340 out (301 reasoning)
git worktree unlock ../feature-wt
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 452 in → 441 out (426 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3-flash-preview:high PASS 100% 239 in → 122 out (114 reasoning)
git worktree unlock ../feature-wt
google/gemini-3-flash-preview:high__json_schema PASS 100% 227 in → 132 out (120 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3-flash-preview:low PASS 100% 236 in → 87 out (79 reasoning)
git worktree unlock ../feature-wt
google/gemini-3-flash-preview:low__json_schema PASS 100% 296 in → 196 out (184 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3-flash-preview:medium PASS 100% 240 in → 94 out (86 reasoning)
git worktree unlock ../feature-wt
google/gemini-3-flash-preview:medium__json_schema PASS 100% 292 in → 132 out (114 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemini-3-flash-preview:xhigh PASS 100% 236 in → 77 out (69 reasoning)
git worktree unlock ../feature-wt
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 293 in → 185 out (173 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 230 in → 202 out (194 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 236 in → 218 out (205 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 229 in → 132 out (124 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 241 in → 154 out (136 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemini-3.1-flash-lite-preview:medium PASS 100% 235 in → 100 out (92 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 287 in → 112 out (94 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 230 in → 176 out (168 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 234 in → 206 out (193 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.1-pro-preview:high PASS 100% 232 in → 233 out (225 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 288 in → 108 out (96 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.1-pro-preview:low PASS 100% 227 in → 47 out (39 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 290 in → 73 out (55 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemini-3.1-pro-preview:medium PASS 100% 230 in → 234 out (226 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 288 in → 62 out (50 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 233 in → 167 out (159 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 287 in → 68 out (56 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.5-flash-lite:high PASS 100% 235 in → 267 out (259 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 235 in → 201 out (188 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash-lite:low PASS 100% 237 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 292 in → 13 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash-lite:max PASS 100% 230 in → 315 out (307 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 298 in → 211 out (198 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash-lite:medium PASS 100% 237 in → 111 out (103 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 238 in → 200 out (187 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 232 in → 201 out (193 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 232 in → 293 out (280 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash:high PASS 100% 236 in → 425 out (417 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash:high__json_schema PASS 100% 296 in → 304 out (292 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.5-flash:low PASS 100% 236 in → 79 out (71 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash:low__json_schema PASS 100% 287 in → 75 out (62 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash:medium PASS 100% 235 in → 324 out (316 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash:medium__json_schema PASS 100% 293 in → 102 out (89 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.5-flash:xhigh PASS 100% 234 in → 308 out (300 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 294 in → 218 out (205 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemini-3.6-flash:high PASS 100% 238 in → 221 out (213 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.6-flash:high__json_schema PASS 100% 231 in → 223 out (211 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.6-flash:low PASS 100% 240 in → 80 out (72 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.6-flash:low__json_schema PASS 100% 239 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.6-flash:max PASS 100% 236 in → 207 out (199 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.6-flash:max__json_schema PASS 100% 299 in → 245 out (233 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemini-3.6-flash:medium PASS 100% 232 in → 213 out (205 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.6-flash:medium__json_schema PASS 100% 292 in → 161 out (143 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemini-3.6-flash:xhigh PASS 100% 238 in → 266 out (258 reasoning)
git worktree unlock ../feature-wt
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 291 in → 230 out (218 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
google/gemma-4-26b-a4b-it:high PASS 100% 253 in → 612 out (495 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 252 in → 524 out (371 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemma-4-26b-a4b-it:low PASS 100% 254 in → 358 out (309 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 255 in → 419 out (288 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemma-4-26b-a4b-it:medium PASS 100% 251 in → 562 out (498 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 246 in → 536 out (441 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemma-4-26b-a4b-it:none PASS 100% 247 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 243 in → 19 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 250 in → 329 out (267 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 251 in → 182 out (135 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemma-4-31b-it:high PASS 100% 246 in → 163 out (114 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-31b-it:high__json_schema PASS 100% 245 in → 104 out (78 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemma-4-31b-it:low PASS 100% 255 in → 139 out (1 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-31b-it:low__json_schema PASS 100% 255 in → 166 out (125 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
google/gemma-4-31b-it:medium PASS 100% 251 in → 123 out (100 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-31b-it:medium__json_schema PASS 100% 245 in → 130 out (107 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
google/gemma-4-31b-it:none PASS 100% 244 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-31b-it:none__json_schema PASS 100% 250 in → 16 out (0 reasoning)
git worktree unlock '../feature-wt'
JSON Schema Structured Output
(raw) { "command": "git worktree unlock '../feature-wt'" }
google/gemma-4-31b-it:xhigh PASS 100% 251 in → 112 out (92 reasoning)
git worktree unlock ../feature-wt
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 254 in → 160 out (127 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
ibm-granite/granite-4.1-8b PASS 100% 207 in → 10 out
git worktree unlock '../feature-wt'
ibm-granite/granite-4.1-8b__json_schema PASS 100% 213 in → 16 out
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
inclusionai/ling-2.6-flash PASS 100% 245 in → 13 out
``` git worktree unlock ../feature-wt ```
inclusionai/ling-2.6-flash__json_schema PASS 100% 242 in → 19 out
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
minimax/minimax-m2.5:high PASS 100% 209 in → 110 out (108 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.5:high__json_schema PASS 100% 210 in → 184 out (145 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
minimax/minimax-m2.5:low PASS 100% 212 in → 210 out (181 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.5:low__json_schema PASS 100% 207 in → 137 out (121 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
minimax/minimax-m2.5:medium PASS 100% 206 in → 114 out (118 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.5:medium__json_schema PASS 100% 235 in → 165 out (160 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
minimax/minimax-m2.5:xhigh PASS 100% 209 in → 152 out (142 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 236 in → 138 out (119 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
minimax/minimax-m2.7:high PASS 100% 215 in → 301 out (329 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.7:low PASS 100% 212 in → 1,770 out (1,709 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.7:low__json_schema PASS 100% 341 in → 256 out (238 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
minimax/minimax-m2.7:medium PASS 100% 220 in → 380 out (385 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.7:medium__json_schema PASS 100% 347 in → 408 out (394 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
minimax/minimax-m2.7:xhigh PASS 100% 214 in → 136 out (134 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 340 in → 320 out (306 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
minimax/minimax-m3:high PASS 100% 350 in → 39 out (30 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m3:high__json_schema PASS 100% 332 in → 15 out (0 reasoning)
git worktree unlock '../feature-wt'
JSON Schema Structured Output
(raw) {"command": "git worktree unlock '../feature-wt'"}
minimax/minimax-m3:low PASS 100% 351 in → 25 out (14 reasoning)
git worktree unlock "../feature-wt"
minimax/minimax-m3:low__json_schema PASS 100% 336 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
minimax/minimax-m3:medium PASS 100% 349 in → 37 out (31 reasoning)
git worktree unlock ../feature-wt
minimax/minimax-m3:medium__json_schema PASS 100% 333 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
minimax/minimax-m3:xhigh PASS 100% 348 in → 35 out (27 reasoning)
git worktree unlock '../feature-wt'
minimax/minimax-m3:xhigh__json_schema PASS 100% 300 in → 17 out (16 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
mistralai/devstral-2512 PASS 100% 236 in → 14 out
```bash git worktree unlock ../feature-wt ```
mistralai/devstral-2512__json_schema PASS 100% 238 in → 14 out
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
mistralai/mistral-medium-3-5:high PASS 100% 248 in → 396 out (350 reasoning)
git worktree unlock ../feature-wt
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 233 in → 277 out (274 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
mistralai/mistral-medium-3-5:low PASS 100% 247 in → 310 out (309 reasoning)
git worktree unlock ../feature-wt
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 232 in → 186 out (191 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
mistralai/mistral-medium-3-5:medium PASS 100% 248 in → 258 out (260 reasoning)
git worktree unlock ../feature-wt
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 236 in → 240 out (246 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
mistralai/mistral-medium-3-5:none PASS 100% 246 in → 10 out (0 reasoning)
git worktree unlock '../feature-wt'
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 240 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 251 in → 433 out (416 reasoning)
git worktree unlock ../feature-wt
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 231 in → 97 out (84 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
moonshotai/kimi-k2.6:high PASS 100% 179 in → 160 out (151 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.6:high__json_schema PASS 100% 175 in → 330 out (141 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
moonshotai/kimi-k2.6:low PASS 100% 176 in → 171 out (161 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.6:low__json_schema PASS 100% 174 in → 211 out (205 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
moonshotai/kimi-k2.6:medium PASS 100% 169 in → 318 out (308 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.6:xhigh PASS 100% 176 in → 325 out (316 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 174 in → 500 out (463 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
moonshotai/kimi-k2.7-code:high PASS 100% 177 in → 96 out (86 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.7-code:low PASS 100% 179 in → 81 out (71 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 180 in → 86 out (71 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
moonshotai/kimi-k2.7-code:medium PASS 100% 174 in → 102 out (92 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.7-code:xhigh PASS 100% 172 in → 115 out (105 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 257 in → 102 out (88 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
moonshotai/kimi-k3:high PASS 100% 257 in → 113 out (90 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:high__json_schema PASS 100% 257 in → 93 out (70 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:low PASS 100% 254 in → 78 out (55 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:low__json_schema PASS 100% 251 in → 63 out (40 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:max PASS 100% 250 in → 60 out (37 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:max__json_schema PASS 100% 255 in → 62 out (39 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:medium PASS 100% 253 in → 82 out (59 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:medium__json_schema PASS 100% 251 in → 63 out (40 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:xhigh PASS 100% 249 in → 64 out (41 reasoning)
git worktree unlock ../feature-wt
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 253 in → 93 out (70 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 256 in → 216 out (180 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 250 in → 130 out (124 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 250 in → 127 out (112 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 246 in → 196 out (212 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 250 in → 653 out (662 reasoning)
git worktree unlock '../feature-wt'
JSON Schema Structured Output
(raw) { "command": "git worktree unlock '../feature-wt'" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 252 in → 60 out (0 reasoning)
git worktree unlock /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_j48bzyr3/feature-wt
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 241 in → 334 out (349 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 250 in → 207 out (209 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 251 in → 327 out (283 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 249 in → 77 out (59 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 247 in → 150 out (131 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 240 in → 96 out (78 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 248 in → 91 out (90 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 242 in → 95 out (77 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 244 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 248 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 249 in → 60 out (54 reasoning)
git worktree unlock ../feature-wt
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 252 in → 85 out (66 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
openai/gpt-5.4-mini:high PASS 100% 180 in → 156 out (142 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-mini:high__json_schema PASS 100% 219 in → 179 out (157 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4-mini:low PASS 100% 185 in → 46 out (32 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-mini:low__json_schema PASS 100% 223 in → 55 out (33 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4-mini:medium PASS 100% 183 in → 113 out (99 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-mini:medium__json_schema PASS 100% 217 in → 99 out (77 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4-mini:none PASS 100% 183 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-mini:xhigh PASS 100% 186 in → 255 out (241 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 214 in → 538 out (516 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4-nano:high PASS 100% 177 in → 45 out (31 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-nano:high__json_schema PASS 100% 224 in → 111 out (89 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4-nano:low__json_schema PASS 100% 214 in → 26 out (0 reasoning)
git -C ../feature-wt worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git -C ../feature-wt worktree unlock ../feature-wt"}
openai/gpt-5.4-nano:medium PASS 100% 186 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-nano:medium__json_schema PASS 100% 214 in → 26 out (0 reasoning)
git -C ../feature-wt worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git -C ../feature-wt worktree unlock ../feature-wt"}
openai/gpt-5.4-nano:none PASS 100% 179 in → 18 out (0 reasoning)
git -C ../feature-wt worktree unlock ../feature-wt
openai/gpt-5.4-nano:xhigh PASS 100% 178 in → 153 out (139 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 216 in → 613 out (591 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4:high PASS 100% 181 in → 83 out (69 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4:high__json_schema PASS 100% 213 in → 82 out (60 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4:low PASS 100% 179 in → 47 out (33 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4:low__json_schema PASS 100% 214 in → 57 out (35 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4:medium PASS 100% 180 in → 66 out (52 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4:medium__json_schema PASS 100% 221 in → 61 out (39 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4:none PASS 100% 184 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4:none__json_schema PASS 100% 215 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.4:xhigh PASS 100% 186 in → 121 out (107 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.4:xhigh__json_schema PASS 100% 216 in → 269 out (247 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.5:high PASS 100% 184 in → 55 out (41 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.5:high__json_schema PASS 100% 212 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.5:low PASS 100% 182 in → 45 out (31 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.5:low__json_schema PASS 100% 218 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.5:medium PASS 100% 187 in → 49 out (35 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.5:medium__json_schema PASS 100% 222 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.5:none PASS 100% 176 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.5:none__json_schema PASS 100% 213 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.5:xhigh PASS 100% 178 in → 55 out (41 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.5:xhigh__json_schema PASS 100% 215 in → 181 out (159 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-luna:high PASS 100% 175 in → 42 out (28 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-luna:high__json_schema PASS 100% 211 in → 61 out (39 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-luna:low PASS 100% 180 in → 42 out (28 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-luna:low__json_schema PASS 100% 214 in → 86 out (64 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-luna:max PASS 100% 181 in → 78 out (64 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-luna:max__json_schema PASS 100% 212 in → 171 out (149 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-luna:medium PASS 100% 176 in → 58 out (44 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-luna:medium__json_schema PASS 100% 215 in → 73 out (51 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-luna:none PASS 100% 178 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-luna:none__json_schema PASS 100% 216 in → 21 out (0 reasoning)
git worktree unlock '../feature-wt'
JSON Schema Structured Output
(raw) {"command":"git worktree unlock '../feature-wt'"}
openai/gpt-5.6-luna:xhigh PASS 100% 184 in → 50 out (36 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 224 in → 230 out (208 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-sol:high PASS 100% 179 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-sol:high__json_schema PASS 100% 212 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-sol:low PASS 100% 182 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-sol:low__json_schema PASS 100% 209 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-sol:max PASS 100% 194 in → 49 out (35 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-sol:max__json_schema PASS 100% 212 in → 161 out (139 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-sol:medium PASS 100% 181 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-sol:medium__json_schema PASS 100% 221 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-sol:none PASS 100% 181 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-sol:none__json_schema PASS 100% 214 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-sol:xhigh PASS 100% 188 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 213 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-terra:high PASS 100% 175 in → 43 out (29 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-terra:high__json_schema PASS 100% 218 in → 77 out (55 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-terra:low PASS 100% 186 in → 38 out (24 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-terra:low__json_schema PASS 100% 217 in → 73 out (51 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-terra:max PASS 100% 185 in → 81 out (67 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-terra:max__json_schema PASS 100% 213 in → 70 out (48 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-terra:medium PASS 100% 183 in → 34 out (20 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-terra:medium__json_schema PASS 100% 208 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-terra:none PASS 100% 181 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-terra:none__json_schema PASS 100% 221 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-5.6-terra:xhigh PASS 100% 185 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 215 in → 20 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-120b:high PASS 100% 239 in → 100 out (92 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-120b:high__json_schema PASS 100% 239 in → 234 out (234 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-120b:low PASS 100% 238 in → 50 out (33 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-120b:low__json_schema PASS 100% 253 in → 42 out (22 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-120b:medium PASS 100% 243 in → 133 out (120 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-120b:medium__json_schema PASS 100% 244 in → 139 out (117 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-120b:xhigh PASS 100% 228 in → 245 out (250 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 232 in → 819 out (905 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-20b:high PASS 100% 241 in → 248 out (256 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-20b:high__json_schema PASS 100% 245 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-20b:low PASS 100% 247 in → 36 out (21 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-20b:low__json_schema PASS 100% 239 in → 31 out (9 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-20b:medium PASS 100% 241 in → 186 out (187 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-20b:medium__json_schema PASS 100% 242 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
openai/gpt-oss-20b:xhigh PASS 100% 244 in → 117 out (109 reasoning)
git worktree unlock ../feature-wt
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 245 in → 504 out (535 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
poolside/laguna-m.1:high PASS 100% 224 in → 429 out (418 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-m.1:high__json_schema PASS 100% 226 in → 140 out (125 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-m.1:low PASS 100% 233 in → 154 out (143 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-m.1:low__json_schema PASS 100% 226 in → 186 out (171 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-m.1:medium PASS 100% 232 in → 191 out (180 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-m.1:medium__json_schema PASS 100% 226 in → 527 out (512 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-m.1:xhigh PASS 100% 226 in → 235 out (224 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-m.1:xhigh__json_schema PASS 100% 231 in → 351 out (332 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
poolside/laguna-xs-2.1:high PASS 100% 222 in → 303 out (294 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs-2.1:high__json_schema PASS 100% 225 in → 158 out (143 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-xs-2.1:low PASS 100% 227 in → 440 out (431 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs-2.1:low__json_schema PASS 100% 224 in → 255 out (235 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
poolside/laguna-xs-2.1:medium PASS 100% 229 in → 285 out (276 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 223 in → 283 out (268 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-xs-2.1:none PASS 100% 227 in → 9 out (0 reasoning)
git worktree unlock '../feature-wt'
poolside/laguna-xs-2.1:none__json_schema PASS 100% 224 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-xs-2.1:xhigh PASS 100% 222 in → 482 out (473 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 223 in → 259 out (244 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-xs.2:high PASS 100% 260 in → 66 out (55 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs.2:high__json_schema PASS 100% 265 in → 139 out (119 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
poolside/laguna-xs.2:low PASS 100% 261 in → 280 out (269 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs.2:low__json_schema PASS 100% 262 in → 122 out (107 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-xs.2:medium PASS 100% 261 in → 112 out (101 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs.2:medium__json_schema PASS 100% 258 in → 72 out (52 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
poolside/laguna-xs.2:none__json_schema PASS 100% 258 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
poolside/laguna-xs.2:xhigh PASS 100% 264 in → 124 out (113 reasoning)
git worktree unlock ../feature-wt
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 260 in → 126 out (111 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
qwen/qwen3.6-27b:high PASS 100% 234 in → 977 out (860 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-27b:high__json_schema PASS 100% 232 in → 764 out (744 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-27b:low PASS 100% 228 in → 492 out (511 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-27b:low__json_schema PASS 100% 236 in → 19 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-27b:medium PASS 100% 229 in → 654 out (636 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-27b:medium__json_schema PASS 100% 226 in → 19 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-27b:none PASS 100% 225 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-27b:none__json_schema PASS 100% 237 in → 19 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-27b:xhigh PASS 100% 229 in → 259 out (265 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 233 in → 870 out (821 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-35b-a3b:high PASS 100% 228 in → 867 out (815 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 233 in → 720 out (683 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-35b-a3b:low PASS 100% 224 in → 733 out (672 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 231 in → 306 out (273 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 232 in → 256 out (244 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 232 in → 940 out (883 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
qwen/qwen3.6-35b-a3b:none PASS 100% 232 in → 9 out (0 reasoning)
git worktree unlock '../feature-wt'
qwen/qwen3.6-35b-a3b:none__json_schema PASS 100% 226 in → 13 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 236 in → 571 out (565 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 227 in → 716 out (680 reasoning)
git worktree unlock '../feature-wt'
JSON Schema Structured Output
(raw) { "command": "git worktree unlock '../feature-wt'" }
qwen/qwen3.6-flash:high PASS 100% 224 in → 857 out (844 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-flash:low PASS 100% 235 in → 853 out (840 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-flash:medium PASS 100% 234 in → 301 out (288 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-flash:none PASS 100% 238 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.6-flash:xhigh PASS 100% 238 in → 264 out (251 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-flash:high PASS 100% 230 in → 751 out (738 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-flash:low PASS 100% 229 in → 446 out (433 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-flash:max PASS 100% 228 in → 940 out (927 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-flash:medium PASS 100% 226 in → 408 out (395 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-flash:none PASS 100% 238 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-flash:xhigh PASS 100% 233 in → 656 out (643 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-max:high PASS 100% 231 in → 73 out (60 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-max:low PASS 100% 230 in → 120 out (107 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-max:medium PASS 100% 233 in → 115 out (102 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-max:none PASS 100% 236 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-max:xhigh PASS 100% 228 in → 98 out (85 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-plus:high PASS 100% 233 in → 163 out (150 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-plus:low PASS 100% 225 in → 225 out (212 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-plus:medium PASS 100% 230 in → 108 out (95 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-plus:none PASS 100% 233 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
qwen/qwen3.7-plus:xhigh PASS 100% 231 in → 395 out (382 reasoning)
git worktree unlock ../feature-wt
tencent/hy3:high PASS 100% 198 in → 148 out (138 reasoning)
git worktree unlock ../feature-wt
tencent/hy3:high__json_schema PASS 100% 202 in → 166 out (150 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
tencent/hy3:low PASS 100% 192 in → 340 out (330 reasoning)
git worktree unlock ../feature-wt
tencent/hy3:low__json_schema PASS 100% 193 in → 303 out (286 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
tencent/hy3:medium PASS 100% 191 in → 315 out (301 reasoning)
git worktree unlock ../feature-wt
tencent/hy3:medium__json_schema PASS 100% 196 in → 381 out (313 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
tencent/hy3:none PASS 100% 196 in → 9 out (0 reasoning)
git worktree unlock ../feature-wt
tencent/hy3:none__json_schema PASS 100% 200 in → 17 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
tencent/hy3:xhigh PASS 100% 191 in → 285 out (275 reasoning)
git worktree unlock ../feature-wt
tencent/hy3:xhigh__json_schema PASS 100% 194 in → 258 out (252 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
thinkingmachines/inkling-small:high PASS 100% 188 in → 34 out (22 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling-small:high__json_schema PASS 100% 186 in → 109 out (89 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
thinkingmachines/inkling-small:low PASS 100% 187 in → 51 out (37 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling-small:low__json_schema PASS 100% 192 in → 68 out (50 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
thinkingmachines/inkling-small:max PASS 100% 195 in → 154 out (149 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling-small:max__json_schema PASS 100% 194 in → 111 out (90 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
thinkingmachines/inkling-small:medium PASS 100% 189 in → 46 out (32 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling-small:medium__json_schema PASS 100% 194 in → 85 out (64 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
thinkingmachines/inkling-small:xhigh PASS 100% 187 in → 136 out (122 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 188 in → 125 out (104 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
thinkingmachines/inkling:high PASS 100% 191 in → 116 out (102 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling:high__json_schema PASS 100% 193 in → 129 out (109 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
thinkingmachines/inkling:low PASS 100% 188 in → 46 out (32 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling:max PASS 100% 187 in → 137 out (123 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling:max__json_schema PASS 100% 192 in → 100 out (83 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
thinkingmachines/inkling:medium PASS 100% 186 in → 28 out (14 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling:medium__json_schema PASS 100% 184 in → 119 out (101 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
thinkingmachines/inkling:none PASS 100% 184 in → 11 out (0 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling:xhigh PASS 100% 188 in → 114 out (106 reasoning)
git worktree unlock ../feature-wt
thinkingmachines/inkling:xhigh__json_schema PASS 100% 194 in → 114 out (97 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.3:high PASS 100% 355 in → 136 out (128 reasoning)
git worktree unlock ../feature-wt
x-ai/grok-4.3:high__json_schema PASS 100% 402 in → 282 out (270 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.3:low PASS 100% 350 in → 155 out (147 reasoning)
git worktree unlock ../feature-wt
x-ai/grok-4.3:low__json_schema PASS 100% 412 in → 244 out (228 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
x-ai/grok-4.3:max PASS 100% 345 in → 207 out (199 reasoning)
git worktree unlock ../feature-wt
x-ai/grok-4.3:max__json_schema PASS 100% 411 in → 307 out (295 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.3:medium PASS 100% 357 in → 141 out (133 reasoning)
git worktree unlock ../feature-wt
x-ai/grok-4.3:medium__json_schema PASS 100% 417 in → 285 out (273 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.3:none PASS 100% 346 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
x-ai/grok-4.3:none__json_schema PASS 100% 413 in → 12 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.3:xhigh PASS 100% 355 in → 126 out (118 reasoning)
git worktree unlock ../feature-wt
x-ai/grok-4.3:xhigh__json_schema PASS 100% 407 in → 413 out (401 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.5:high PASS 100% 391 in → 121 out (109 reasoning)
``` git worktree unlock ../feature-wt ```
x-ai/grok-4.5:high__json_schema PASS 100% 463 in → 467 out (454 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.5:low PASS 100% 392 in → 95 out (83 reasoning)
``` git worktree unlock ../feature-wt ```
x-ai/grok-4.5:low__json_schema PASS 100% 462 in → 170 out (157 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.5:max PASS 100% 395 in → 187 out (175 reasoning)
``` git worktree unlock ../feature-wt ```
x-ai/grok-4.5:max__json_schema PASS 100% 471 in → 260 out (247 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.5:medium PASS 100% 395 in → 189 out (177 reasoning)
``` git worktree unlock ../feature-wt ```
x-ai/grok-4.5:medium__json_schema PASS 100% 456 in → 125 out (112 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
x-ai/grok-4.5:xhigh PASS 100% 398 in → 103 out (91 reasoning)
``` git worktree unlock ../feature-wt ```
x-ai/grok-4.5:xhigh__json_schema PASS 100% 457 in → 271 out (258 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
z-ai/glm-4.7-flash:high PASS 100% 194 in → 268 out (260 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-4.7-flash:low PASS 100% 196 in → 467 out (446 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-4.7-flash:low__json_schema PASS 100% 193 in → 323 out (300 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
z-ai/glm-4.7-flash:medium PASS 100% 197 in → 370 out (340 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-4.7-flash:none PASS 100% 191 in → 8 out (0 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-4.7-flash:none__json_schema PASS 100% 196 in → 14 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
z-ai/glm-4.7-flash:xhigh PASS 100% 188 in → 344 out (341 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 193 in → 310 out (274 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command": "git worktree unlock ../feature-wt"}
z-ai/glm-5.2:high PASS 100% 201 in → 69 out (65 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-5.2:high__json_schema PASS 100% 372 in → 85 out (67 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
z-ai/glm-5.2:low PASS 100% 201 in → 204 out (194 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-5.2:low__json_schema PASS 100% 200 in → 205 out (184 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree unlock ../feature-wt" }
z-ai/glm-5.2:medium PASS 100% 207 in → 195 out (185 reasoning)
git worktree unlock ../feature-wt
z-ai/glm-5.2:none__json_schema PASS 100% 367 in → 13 out (0 reasoning)
git worktree unlock ../feature-wt
JSON Schema Structured Output
(raw) {"command":"git worktree unlock ../feature-wt"}
z-ai/glm-5.2:xhigh PASS 100% 201 in → 140 out (144 reasoning)
git worktree unlock ../feature-wt
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
arcee-ai/trinity-mini:low FAIL 0% 179 in → 377 out (388 reasoning)
git worktreeunlock ../feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
deepseek/deepseek-v4-pro:medium FAIL 0% 185 in → 110 out (105 reasoning)
git worktree unlock../feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
liquid/lfm-2-24b-a2b FAIL 0% 213 in → 109 out
git worktree add -f /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_k7e_5n_y/worktree_usage_f011 /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_k7e_5n_y/feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
Invalid JSON. Output: git worktree add -f /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_lsa1uxal/worktree_usage_f011 feature-wt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree unlock ../feature-wt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 2 (char 1)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
nvidia/nemotron-3-nano-30b-a3b:low FAIL 0% 249 in → 293 out (296 reasoning)
gitworktree unlock ../feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
nvidia/nemotron-3-nano-30b-a3b:none__json_schema FAIL 0% 251 in → 17 out (0 reasoning)
git worktree lock ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree lock ../feature-wt" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
openai/gpt-5.4-mini:none__json_schema FAIL 0% 216 in → 60 out (0 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_k70liw9l/feature-wt worktree unlock
JSON Schema Structured Output
(raw) {"command":"git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_k70liw9l/feature-wt worktree unlock"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
openai/gpt-5.4-nano:low FAIL 0% 181 in → 17 out (0 reasoning)
git -C feature-wt worktree unlock ../feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
openai/gpt-5.4-nano:none__json_schema FAIL 0% 213 in → 95 out (0 reasoning)
git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_fsp191hj/ worktree unlock /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_fsp191hj/feature-wt
JSON Schema Structured Output
(raw) {"command":"git -C /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_fsp191hj/ worktree unlock /private/var/folders/kg/gb4597jx5jv6b7tngk181jr40000gn/T/worktree_usage_f011_fsp191hj/feature-wt"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
poolside/laguna-m.1:none FAIL 0% 227 in → 12 out (0 reasoning)
git worktree remove --force ../feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
poolside/laguna-m.1:none__json_schema FAIL 0% 223 in → 21 out (0 reasoning)
git worktree remove --force ../feature-wt
JSON Schema Structured Output
(raw) { "command": "git worktree remove --force ../feature-wt" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
poolside/laguna-xs.2:none FAIL 0% 266 in → 12 out (0 reasoning)
git worktree remove --force ../feature-wt
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-91b11ef7-a555-99e5-9a30-6d6486735fb6","request_id":"91b11ef7-a555-99e5-9a30-6d6486735fb6"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5a333b1f-383e-9586-9a95-6c650d3d6e28","request_id":"5a333b1f-383e-9586-9a95-6c650d3d6e28"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-050806f1-811b-9800-996d-121e3e5ab7d7","request_id":"050806f1-811b-9800-996d-121e3e5ab7d7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d6962f9c-466c-9b0b-9cb9-053b52dcfb47","request_id":"d6962f9c-466c-9b0b-9cb9-053b52dcfb47"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7f3d5761-1224-9958-9d92-447202877d05","request_id":"7f3d5761-1224-9958-9d92-447202877d05"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-5a21c2bd-2950-9d35-b497-54dc9c8f92ef"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-7dff1fbe-87fe-9a84-9e64-d3bca939558d"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-409f09a7-05c8-9816-90f9-e21b728e14d9"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-2aa0b9f6-42bf-9ecb-9954-375e4e4b9e37"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-37cc9b4f-bbde-98e6-bd99-6003a0402c27"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-a94ed4c0-4d0c-9070-b290-f536d7c549e5"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-2f38fb0a-73e8-97f7-987e-fbb9c1deac55","request_id":"2f38fb0a-73e8-97f7-987e-fbb9c1deac55"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7a6664df-f266-99e3-8ec3-16742247792b","request_id":"7a6664df-f266-99e3-8ec3-16742247792b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5c042a6b-28cc-9df7-8a9d-3ad3290c5375","request_id":"5c042a6b-28cc-9df7-8a9d-3ad3290c5375"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-efa5a3b7-dd4b-9981-a5c1-82382403acb9","request_id":"efa5a3b7-dd4b-9981-a5c1-82382403acb9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b414f169-91f3-9afa-be44-8b00c86611c0","request_id":"b414f169-91f3-9afa-be44-8b00c86611c0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-38add378-b05f-94a3-bf9d-05130c5fbca7","request_id":"38add378-b05f-94a3-bf9d-05130c5fbca7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-922fab08-af0d-9e86-a215-19b0c81041a9","request_id":"922fab08-af0d-9e86-a215-19b0c81041a9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-62a09a01-bd25-94cc-af86-aa663423d368","request_id":"62a09a01-bd25-94cc-af86-aa663423d368"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-df08cc46-b4a1-996e-875d-f4ee9b9a8641","request_id":"df08cc46-b4a1-996e-875d-f4ee9b9a8641"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-78754bb4-dfd9-97e9-a993-76160833edcb","request_id":"78754bb4-dfd9-97e9-a993-76160833edcb"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
thinkingmachines/inkling-small:none FAIL 0% 193 in → 11 out (0 reasoning)
git worktree unlock ../feature-wt<|end_message|>
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
Invalid JSON. Output: git worktree unlock ../feature-wt{ "command": "git worktree unlock ../feature-wt" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree unlock ../feature-wt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: { "command":"git worktree unlock ../feature-wtgit worktree unlock ../feature-wt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Unterminated string starting at: line 4 column 11 (char 14)
Failure: Failed to parse structured JSON response: Unterminated string starting at: line 4 column 11 (char 14)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git worktree unlock ../feature-wt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
z-ai/glm-5.2:none FAIL 0% 195 in → 9 out (0 reasoning)
git worktree unlock ../
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git worktree list --porcelain', 'not_contains': 'locked'}]
Invalid JSON. Output: git worktree unlock ../feature-wt
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)