Rename a tag via delete and create
Tests ability to rename a tag via delete-and-create workflow. Evaluates tag-rename as a composite operation.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'App' > app.txt
  5. 05 git add app.txt
  6. 06 git commit -m 'Release v1'
  7. 07 git tag v1.0-rc
Prompt
Rename the tag 'v1.0-rc' to 'v1.0' by moving the tag to the same commit. Output ONLY the git commands, one per line, nothing else.
Expected
git tag v1.0 v1.0-rc && git tag -d v1.0-rc

Scoped model quality, cost, API time, and token usage for tag_management/f008.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 89 in → 147 out (73 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-fable-5:high__json_schema PASS 100% 299 in → 88 out (48 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-fable-5:low PASS 100% 91 in → 40 out (4 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-fable-5:low__json_schema PASS 100% 300 in → 60 out (32 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-fable-5:max PASS 100% 90 in → 366 out (61 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-fable-5:max__json_schema PASS 100% 299 in → 394 out (45 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-fable-5:medium PASS 100% 90 in → 71 out (48 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-fable-5:medium__json_schema PASS 100% 300 in → 80 out (37 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-fable-5:xhigh PASS 100% 87 in → 122 out (47 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 300 in → 135 out (53 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-haiku-4.5:high PASS 100% 101 in → 623 out (416 reasoning)
``` git tag v1.0 aa37223 git tag -d v1.0-rc ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 266 in → 3,017 out (1,123 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-haiku-4.5:low PASS 100% 103 in → 503 out (409 reasoning)
``` git tag -d v1.0-rc git tag v1.0 6d3b439 ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 265 in → 1,207 out (705 reasoning)
git tag -d v1.0-rc && git tag v1.0 fecab5b
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc && git tag v1.0 fecab5b"}
anthropic/claude-haiku-4.5:medium PASS 100% 101 in → 3,547 out (1,360 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 265 in → 1,727 out (892 reasoning)
git tag -d v1.0-rc && git tag v1.0 5ffce04
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc && git tag v1.0 5ffce04"}
anthropic/claude-haiku-4.5:none PASS 100% 72 in → 29 out (0 reasoning)
``` git tag -d v1.0-rc git tag v1.0 60f47eb ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 234 in → 29 out (0 reasoning)
git tag -d v1.0-rc && git tag v1.0 386fb59
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc && git tag v1.0 386fb59"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 101 in → 832 out (471 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 265 in → 1,158 out (840 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-opus-4.6:high PASS 100% 73 in → 121 out (48 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.6:high__json_schema PASS 100% 234 in → 122 out (75 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.6:low PASS 100% 72 in → 32 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 236 in → 58 out (12 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.6:max PASS 100% 74 in → 127 out (49 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 236 in → 239 out (135 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.6:medium PASS 100% 72 in → 32 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 236 in → 65 out (18 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.6:none PASS 100% 73 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.6:none__json_schema PASS 100% 234 in → 31 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.6:xhigh PASS 100% 74 in → 121 out (48 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 236 in → 190 out (135 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.7:high PASS 100% 93 in → 32 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.7:high__json_schema PASS 100% 307 in → 41 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-opus-4.7:low PASS 100% 95 in → 32 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.7:low__json_schema PASS 100% 303 in → 41 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-opus-4.7:max PASS 100% 93 in → 498 out (56 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 305 in → 39 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.7:medium PASS 100% 95 in → 32 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 306 in → 41 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-opus-4.7:none PASS 100% 94 in → 32 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.7:none__json_schema PASS 100% 307 in → 39 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.7:xhigh PASS 100% 95 in → 32 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 304 in → 41 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-opus-4.8:high PASS 100% 89 in → 108 out (40 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 297 in → 37 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.8:low PASS 100% 90 in → 62 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin :refs/tags/v1.0-rc ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 300 in → 37 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.8:max PASS 100% 90 in → 310 out (47 reasoning)
``` git tag v1.0 3023bd0 git tag -d v1.0-rc ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 299 in → 295 out (68 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-opus-4.8:medium PASS 100% 89 in → 78 out (36 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 299 in → 37 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.8:none PASS 100% 90 in → 34 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 299 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-opus-4.8:xhigh PASS 100% 90 in → 247 out (50 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 301 in → 247 out (66 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-sonnet-4.6:high PASS 100% 73 in → 134 out (67 reasoning)
``` git tag v1.0 0001ff8 git tag -d v1.0-rc ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 237 in → 230 out (147 reasoning)
git tag v1.0 4a1b452 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 4a1b452\ngit tag -d v1.0-rc"}
anthropic/claude-sonnet-4.6:low PASS 100% 75 in → 45 out (5 reasoning)
git tag v1.0 f4a5bf4 git tag -d v1.0-rc
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 236 in → 64 out (21 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-sonnet-4.6:max PASS 100% 72 in → 283 out (194 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 236 in → 344 out (158 reasoning)
git tag v1.0 d74de4f git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 d74de4f\ngit tag -d v1.0-rc"}
anthropic/claude-sonnet-4.6:medium PASS 100% 72 in → 86 out (31 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 235 in → 184 out (113 reasoning)
git tag v1.0 b680965 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 b680965\ngit tag -d v1.0-rc"}
anthropic/claude-sonnet-4.6:none PASS 100% 73 in → 49 out (0 reasoning)
``` git tag v1.0 af3e150 git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 235 in → 34 out (0 reasoning)
git tag v1.0 v1.0-rc^{} && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc^{} && git tag -d v1.0-rc"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 73 in → 85 out (32 reasoning)
``` git tag v1.0 d131a60 git tag -d v1.0-rc ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 235 in → 234 out (134 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-sonnet-5:high PASS 100% 89 in → 34 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 300 in → 37 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-sonnet-5:low PASS 100% 89 in → 30 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-sonnet-5:low__json_schema PASS 100% 300 in → 37 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-sonnet-5:medium PASS 100% 91 in → 34 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 299 in → 95 out (48 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
anthropic/claude-sonnet-5:none PASS 100% 89 in → 34 out (0 reasoning)
``` git tag v1.0 v1.0-rc git tag -d v1.0-rc ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 301 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
anthropic/claude-sonnet-5:xhigh PASS 100% 88 in → 123 out (40 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 297 in → 108 out (53 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
arcee-ai/trinity-large-thinking:high PASS 100% 67 in → 1,767 out (1,767 reasoning)
git tag -d v1.0-rc git tag v1.0 4e00942
arcee-ai/trinity-large-thinking:low PASS 100% 67 in → 573 out (557 reasoning)
git tag -d v1.0-rc git tag v1.0 19314b2
arcee-ai/trinity-large-thinking:medium PASS 100% 67 in → 301 out (282 reasoning)
git tag -d v1.0-rc git tag v1.0 6797ea3
arcee-ai/trinity-large-thinking:xhigh PASS 100% 68 in → 772 out (756 reasoning)
git tag -d v1.0-rc git tag v1.0 9d46d74
arcee-ai/trinity-mini:medium PASS 100% 66 in → 523 out (532 reasoning)
git tag -d v1.0-rc git tag v1.0
arcee-ai/trinity-mini:medium__json_schema PASS 100% 67 in → 369 out (326 reasoning)
git tag -d v1.0-rc git tag v1.0 4ca7341
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\ngit tag v1.0 4ca7341" }
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 66 in → 1,141 out (1,165 reasoning)
git tag -d v1.0-rc git tag v1.0
JSON Schema Structured Output
(raw) { "command":"git tag -d v1.0-rc\ngit tag v1.0" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 64 in → 290 out (212 reasoning)
git tag v1.0 03996b5 git tag -d v1.0-rc
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 143 in → 5,617 out (4,611 reasoning)
git tag v1.0 v1.0-rc^{commit} git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc^{commit}\ngit tag -d v1.0-rc"}
deepseek/deepseek-v4-flash-0731:low PASS 100% 66 in → 329 out (273 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-flash-0731:max PASS 100% 156 in → 8,133 out (8,110 reasoning)
git tag v1.0 4598d42 git tag -d v1.0-rc
deepseek/deepseek-v4-flash-0731:medium PASS 100% 64 in → 565 out (419 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 64 in → 1,153 out (954 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 64 in → 24 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 73 in → 36 out (0 reasoning)
git tag -f v1.0 v1.0-rc^{} && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -f v1.0 v1.0-rc^{} && git tag -d v1.0-rc" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 143 in → 6,192 out (5,044 reasoning)
git tag v1.0 9563b48 git tag -d v1.0-rc
deepseek/deepseek-v4-flash:low PASS 100% 65 in → 475 out (373 reasoning)
git tag v1.0 $(git rev-parse v1.0-rc) git tag -d v1.0-rc
deepseek/deepseek-v4-flash:medium PASS 100% 63 in → 388 out (363 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 65 in → 148 out (115 reasoning)
git tag -d v1.0-rc git tag v1.0 b2845b3
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\n git tag v1.0 b2845b3" }
deepseek/deepseek-v4-flash:none PASS 100% 66 in → 26 out (0 reasoning)
git tag -f v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-flash:xhigh PASS 100% 66 in → 1,067 out (892 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
deepseek/deepseek-v4-pro:high PASS 100% 63 in → 21 out (0 reasoning)
git tag v1.0 cf68933 git tag -d v1.0-rc
deepseek/deepseek-v4-pro:low PASS 100% 67 in → 357 out (332 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-pro:medium PASS 100% 66 in → 519 out (421 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
deepseek/deepseek-v4-pro:xhigh PASS 100% 142 in → 1,091 out (1,065 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
google/gemini-3-flash-preview:high PASS 100% 64 in → 1,406 out (1,363 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
google/gemini-3-flash-preview:high__json_schema PASS 100% 65 in → 1,612 out (1,585 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3-flash-preview:low PASS 100% 63 in → 597 out (554 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
google/gemini-3-flash-preview:low__json_schema PASS 100% 121 in → 980 out (952 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3-flash-preview:medium PASS 100% 63 in → 803 out (780 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3-flash-preview:medium__json_schema PASS 100% 121 in → 642 out (614 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3-flash-preview:xhigh PASS 100% 64 in → 577 out (554 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 64 in → 960 out (933 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 64 in → 1,593 out (1,569 reasoning)
git tag v1.0 8ecb09c git tag -d v1.0-rc
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 64 in → 1,052 out (1,020 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc" }
google/gemini-3.1-flash-lite-preview:low PASS 100% 62 in → 191 out (149 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin :v1.0-rc git push origin v1.0
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 120 in → 164 out (113 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc && git push origin :refs/tags/v1.0-rc && git push origin v1.0
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc && git push origin :refs/tags/v1.0-rc && git push origin v1.0"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 64 in → 680 out (657 reasoning)
git tag v1.0 dbe2e3a git tag -d v1.0-rc
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 65 in → 1,037 out (1,014 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.1-pro-preview:high PASS 100% 62 in → 289 out (266 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 121 in → 797 out (769 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.1-pro-preview:low PASS 100% 63 in → 387 out (364 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 121 in → 397 out (369 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.1-pro-preview:medium PASS 100% 64 in → 555 out (532 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 121 in → 597 out (569 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 64 in → 852 out (829 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 121 in → 716 out (689 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash-lite:high PASS 100% 63 in → 1,144 out (1,121 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 64 in → 795 out (767 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash-lite:low PASS 100% 64 in → 47 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 03512cc git push origin :refs/tags/v1.0-rc git push origin v1.0
google/gemini-3.5-flash-lite:max PASS 100% 61 in → 1,242 out (1,219 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 65 in → 895 out (868 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash-lite:medium PASS 100% 64 in → 650 out (627 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 64 in → 520 out (490 reasoning)
git tag -f v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -f v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 65 in → 1,418 out (1,395 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 119 in → 587 out (559 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash:high PASS 100% 62 in → 1,001 out (978 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash:high__json_schema PASS 100% 120 in → 590 out (563 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash:low PASS 100% 64 in → 391 out (368 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash:low__json_schema PASS 100% 121 in → 393 out (365 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash:medium PASS 100% 63 in → 826 out (803 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash:medium__json_schema PASS 100% 121 in → 563 out (535 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.5-flash:xhigh PASS 100% 63 in → 872 out (849 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 121 in → 422 out (394 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.6-flash:high PASS 100% 65 in → 422 out (399 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.6-flash:high__json_schema PASS 100% 63 in → 703 out (676 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.6-flash:low PASS 100% 65 in → 270 out (247 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.6-flash:low__json_schema PASS 100% 63 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.6-flash:max PASS 100% 64 in → 509 out (486 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.6-flash:max__json_schema PASS 100% 119 in → 577 out (550 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemini-3.6-flash:medium PASS 100% 65 in → 308 out (285 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.6-flash:medium__json_schema PASS 100% 120 in → 362 out (329 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc" }
google/gemini-3.6-flash:xhigh PASS 100% 65 in → 408 out (385 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 121 in → 358 out (331 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
google/gemma-4-26b-a4b-it:high PASS 100% 79 in → 1,896 out (1,476 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 78 in → 980 out (711 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
google/gemma-4-26b-a4b-it:low PASS 100% 80 in → 794 out (615 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 80 in → 906 out (679 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
google/gemma-4-26b-a4b-it:medium PASS 100% 79 in → 559 out (535 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 80 in → 622 out (433 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
google/gemma-4-26b-a4b-it:none PASS 100% 77 in → 24 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-26b-a4b-it:xhigh PASS 100% 79 in → 1,173 out (1,074 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 78 in → 2,023 out (1,519 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin --delete v1.0-rc" }
google/gemma-4-31b-it:high PASS 100% 79 in → 523 out (395 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-31b-it:high__json_schema PASS 100% 79 in → 362 out (256 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
google/gemma-4-31b-it:low PASS 100% 80 in → 456 out (281 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-31b-it:medium PASS 100% 78 in → 628 out (434 reasoning)
git tag v1.0 ac50cfa git tag -d v1.0-rc
google/gemma-4-31b-it:medium__json_schema PASS 100% 79 in → 253 out (143 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
google/gemma-4-31b-it:none PASS 100% 75 in → 23 out (0 reasoning)
git tag v1.0 f40ada3 git tag -d v1.0-rc
google/gemma-4-31b-it:xhigh PASS 100% 82 in → 492 out (468 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 79 in → 635 out (448 reasoning)
git tag v1.0 5a8ae2c git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 5a8ae2c\ngit tag -d v1.0-rc" }
ibm-granite/granite-4.1-8b PASS 100% 71 in → 18 out
git tag v1.0 git tag -d v1.0-rc
liquid/lfm-2-24b-a2b PASS 100% 71 in → 37 out
git tag -l v1.0-rc git tag -d v1.0-rc git tag v1.0 git push origin v1.0
minimax/minimax-m2.5:high PASS 100% 98 in → 622 out (601 reasoning)
git tag v1.0 f8acef4 git tag -d v1.0-rc
minimax/minimax-m2.5:medium PASS 100% 96 in → 786 out (738 reasoning)
git tag -d v1.0-rc git tag v1.0 fb07782
minimax/minimax-m2.5:xhigh PASS 100% 97 in → 379 out (322 reasoning)
git tag -d v1.0-rc git tag -a v1.0 -m "Release v1" 9bde146
minimax/minimax-m2.7:high PASS 100% 100 in → 690 out (665 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
minimax/minimax-m2.7:high__json_schema PASS 100% 228 in → 1,228 out (1,178 reasoning)
git tag v1.0 2dad657 git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 2dad657\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin --delete v1.0-rc"}
minimax/minimax-m2.7:low PASS 100% 96 in → 1,242 out (1,194 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin :refs/tags/v1.0-rc
minimax/minimax-m2.7:low__json_schema PASS 100% 231 in → 918 out (883 reasoning)
git tag v1.0 1af8a1b git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 1af8a1b\ngit tag -d v1.0-rc" }
minimax/minimax-m2.7:medium__json_schema PASS 100% 230 in → 786 out (735 reasoning)
git tag v1.0 ccf1d67 git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 ccf1d67\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin --delete v1.0-rc"}
minimax/minimax-m2.7:xhigh PASS 100% 97 in → 725 out (636 reasoning)
git tag v1.0 87017f9 git tag -d v1.0-rc
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 230 in → 910 out (850 reasoning)
git tag -a v1.0 4e950d8 -m 'Release v1' git tag -d v1.0-rc git push origin --delete v1.0-rc git push origin v1.0
JSON Schema Structured Output
(raw) {"command":"git tag -a v1.0 4e950d8 -m 'Release v1'\ngit tag -d v1.0-rc\ngit push origin --delete v1.0-rc\ngit push origin v1.0"}
minimax/minimax-m3:high PASS 100% 235 in → 159 out (110 reasoning)
git tag -d v1.0-rc git tag v1.0 d04bcfa
minimax/minimax-m3:high__json_schema PASS 100% 184 in → 32 out (31 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc" }
minimax/minimax-m3:low PASS 100% 235 in → 135 out (98 reasoning)
git tag v1.0 ffe6f44 git tag -d v1.0-rc
minimax/minimax-m3:medium PASS 100% 236 in → 129 out (73 reasoning)
git tag -d v1.0-rc git tag v1.0 8f7eca1 git push origin :refs/tags/v1.0-rc git push origin refs/tags/v1.0
minimax/minimax-m3:medium__json_schema PASS 100% 222 in → 30 out (0 reasoning)
git tag v1.0 ce622d8 && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 ce622d8 && git tag -d v1.0-rc" }
minimax/minimax-m3:xhigh PASS 100% 234 in → 101 out (61 reasoning)
git tag -d v1.0-rc git tag v1.0 744eddf
minimax/minimax-m3:xhigh__json_schema PASS 100% 222 in → 29 out (0 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
mistralai/devstral-2512 PASS 100% 66 in → 31 out
```git git tag v1.0 19a3675 git tag -d v1.0-rc ```
mistralai/mistral-medium-3-5:high PASS 100% 76 in → 2,748 out (2,595 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 65 in → 1,180 out (1,066 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin --delete v1.0-rc"}
mistralai/mistral-medium-3-5:low PASS 100% 78 in → 740 out (619 reasoning)
git tag v1.0 f12b858 git tag -d v1.0-rc
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 65 in → 2,225 out (2,130 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin :refs/tags/v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin :refs/tags/v1.0-rc"}
mistralai/mistral-medium-3-5:medium PASS 100% 77 in → 2,361 out (2,139 reasoning)
git tag v1.0 00ee093 git tag -d v1.0-rc
mistralai/mistral-medium-3-5:none PASS 100% 78 in → 31 out (0 reasoning)
```git git tag v1.0 0f38258 git tag -d v1.0-rc ```
mistralai/mistral-medium-3-5:xhigh PASS 100% 76 in → 2,862 out (2,764 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.6:high PASS 100% 65 in → 2,114 out (2,091 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.6:low PASS 100% 66 in → 729 out (706 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.6:low__json_schema PASS 100% 65 in → 1,464 out (1,462 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
moonshotai/kimi-k2.6:medium PASS 100% 66 in → 730 out (707 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 65 in → 2,036 out (1,892 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
moonshotai/kimi-k2.6:xhigh PASS 100% 66 in → 1,462 out (1,439 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 64 in → 33 out (0 reasoning)
git tag v1.0 7373700 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 7373700\ngit tag -d v1.0-rc" }
moonshotai/kimi-k2.7-code:high PASS 100% 66 in → 317 out (294 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 65 in → 448 out (417 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
moonshotai/kimi-k2.7-code:low PASS 100% 64 in → 950 out (927 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.7-code:medium PASS 100% 66 in → 342 out (307 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 149 in → 409 out (379 reasoning)
git tag v1.0 4793d5b git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 4793d5b\ngit tag -d v1.0-rc"}
moonshotai/kimi-k2.7-code:xhigh PASS 100% 64 in → 663 out (640 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 148 in → 445 out (416 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
moonshotai/kimi-k3:high PASS 100% 144 in → 328 out (292 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:high__json_schema PASS 100% 143 in → 143 out (107 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:low PASS 100% 142 in → 171 out (135 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:low__json_schema PASS 100% 143 in → 199 out (163 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:max PASS 100% 142 in → 296 out (260 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:max__json_schema PASS 100% 142 in → 202 out (166 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:medium PASS 100% 141 in → 295 out (259 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:medium__json_schema PASS 100% 237 in → 469 out (429 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
moonshotai/kimi-k3:xhigh PASS 100% 141 in → 154 out (118 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 143 in → 151 out (115 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 77 in → 604 out (480 reasoning)
git tag --delete v1.0-rc git tag v1.0 8dbeea1
JSON Schema Structured Output
(raw) { "command": "git tag --delete v1.0-rc\ngit tag v1.0 8dbeea1" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 78 in → 537 out (419 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin :v1.0-rc v1.0
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 77 in → 504 out (378 reasoning)
git tag -d v1.0-rc git tag v1.0 cfa69cc
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 79 in → 218 out (149 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 78 in → 205 out (179 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 79 in → 452 out (371 reasoning)
git tag v1.0 364865f git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin v1.0
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 364865f\ngit tag -d v1.0-rc\ngit push origin :refs/tags/v1.0-rc\ngit push origin v1.0" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 79 in → 272 out (238 reasoning)
git tag -d v1.0-rc git tag v1.0 59436e2
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 78 in → 728 out (669 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin --tags
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc^{}\ngit tag -d v1.0-rc\ngit push origin :refs/tags/v1.0-rc\ngit push origin --tags" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 79 in → 47 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 19a3675 git push origin :refs/tags/v1.0-rc git push origin v1.0
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 78 in → 227 out (187 reasoning)
git tag -d v1.0-rc git tag v1.0 5082ffb
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 79 in → 460 out (396 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin v1.0
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc\ngit push origin :refs/tags/v1.0-rc\ngit push origin v1.0" }
openai/gpt-5.4-mini:high PASS 100% 65 in → 306 out (277 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.4-mini:low PASS 100% 66 in → 136 out (107 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.4-mini:low__json_schema PASS 100% 99 in → 132 out (94 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.4-mini:medium PASS 100% 66 in → 267 out (238 reasoning)
git tag v1.0 513f9c2 git tag -d v1.0-rc
openai/gpt-5.4-mini:medium__json_schema PASS 100% 101 in → 152 out (115 reasoning)
git tag v1.0 ed4ba9f git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 ed4ba9f\ngit tag -d v1.0-rc"}
openai/gpt-5.4-mini:none PASS 100% 66 in → 27 out (0 reasoning)
git tag v1.0 2b2954c git tag -d v1.0-rc
openai/gpt-5.4-mini:none__json_schema PASS 100% 101 in → 34 out (0 reasoning)
git tag v1.0 c6344f8 && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 c6344f8 && git tag -d v1.0-rc"}
openai/gpt-5.4-mini:xhigh PASS 100% 66 in → 28,240 out (28,211 reasoning)
git tag v1.0 3c6ff01 git tag -d v1.0-rc
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 100 in → 24,337 out (24,300 reasoning)
git tag v1.0 93d1769 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 93d1769\ngit tag -d v1.0-rc"}
openai/gpt-5.4-nano:high PASS 100% 68 in → 28 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 a6a2a6a
openai/gpt-5.4-nano:low PASS 100% 65 in → 28 out (0 reasoning)
git tag -f v1.0 0bbabcf git tag -d v1.0-rc
openai/gpt-5.4-nano:low__json_schema PASS 100% 100 in → 46 out (0 reasoning)
git tag -d v1.0 || true git tag v1.0 852e61c git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag -d v1.0 || true\ngit tag v1.0 852e61c\ngit tag -d v1.0-rc"}
openai/gpt-5.4-nano:medium PASS 100% 65 in → 50 out (0 reasoning)
git tag -f v1.0 42244c8 git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin v1.0
openai/gpt-5.4-nano:none PASS 100% 67 in → 28 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 69b0f6f
openai/gpt-5.4-nano:xhigh PASS 100% 65 in → 857 out (827 reasoning)
git tag -f v1.0 2155ff3 git tag -d v1.0-rc
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 101 in → 3,492 out (3,452 reasoning)
git tag -f v1.0 2f0245d git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag -f v1.0 2f0245d\ngit tag -d v1.0-rc"}
openai/gpt-5.4:high PASS 100% 64 in → 206 out (177 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.4:high__json_schema PASS 100% 101 in → 554 out (516 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.4:low PASS 100% 67 in → 66 out (36 reasoning)
git tag v1.0 2f0abf8 git tag -d v1.0-rc
openai/gpt-5.4:low__json_schema PASS 100% 100 in → 74 out (37 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.4:medium PASS 100% 65 in → 160 out (131 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.4:medium__json_schema PASS 100% 101 in → 114 out (76 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.4:none PASS 100% 66 in → 35 out (0 reasoning)
git tag -a v1.0 e8ef20f -m "v1.0" git tag -d v1.0-rc
openai/gpt-5.4:none__json_schema PASS 100% 101 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.4:xhigh PASS 100% 66 in → 2,099 out (2,070 reasoning)
git tag v1.0 5c00c34 git tag -d v1.0-rc
openai/gpt-5.4:xhigh__json_schema PASS 100% 100 in → 552 out (516 reasoning)
git tag v1.0 981cc2c git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 981cc2c\ngit tag -d v1.0-rc"}
openai/gpt-5.5:high PASS 100% 65 in → 545 out (516 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.5:high__json_schema PASS 100% 99 in → 162 out (125 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
openai/gpt-5.5:low PASS 100% 66 in → 83 out (54 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.5:low__json_schema PASS 100% 99 in → 122 out (84 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.5:medium PASS 100% 65 in → 175 out (146 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.5:medium__json_schema PASS 100% 101 in → 320 out (283 reasoning)
git tag v1.0 2d0bacf && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 2d0bacf && git tag -d v1.0-rc"}
openai/gpt-5.5:none PASS 100% 66 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.5:none__json_schema PASS 100% 101 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.5:xhigh PASS 100% 65 in → 546 out (516 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
openai/gpt-5.5:xhigh__json_schema PASS 100% 102 in → 556 out (516 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc^{}\ngit tag -d v1.0-rc"}
openai/gpt-5.6-luna:high PASS 100% 67 in → 287 out (258 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-luna:high__json_schema PASS 100% 101 in → 201 out (163 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-luna:low PASS 100% 66 in → 157 out (128 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-luna:low__json_schema PASS 100% 101 in → 106 out (68 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-luna:max PASS 100% 64 in → 314 out (287 reasoning)
git tag v1.0 687ece9 git tag -d v1.0-rc
openai/gpt-5.6-luna:max__json_schema PASS 100% 102 in → 1,406 out (1,367 reasoning)
git tag v1.0 5a05b6c git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 5a05b6c\ngit tag -d v1.0-rc"}
openai/gpt-5.6-luna:medium PASS 100% 66 in → 263 out (234 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-luna:medium__json_schema PASS 100% 100 in → 241 out (205 reasoning)
git tag v1.0 c25a935 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 c25a935\ngit tag -d v1.0-rc"}
openai/gpt-5.6-luna:none PASS 100% 66 in → 26 out (0 reasoning)
git tag v1.0 b4c205f git tag -d v1.0-rc
openai/gpt-5.6-luna:none__json_schema PASS 100% 99 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-luna:xhigh PASS 100% 64 in → 543 out (516 reasoning)
git tag v1.0 841487d git tag -d v1.0-rc
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 100 in → 401 out (363 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-sol:high PASS 100% 66 in → 272 out (243 reasoning)
git tag v1.0 59a98c2 git tag -d v1.0-rc
openai/gpt-5.6-sol:high__json_schema PASS 100% 100 in → 394 out (357 reasoning)
git tag v1.0 3555c36 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 3555c36\ngit tag -d v1.0-rc"}
openai/gpt-5.6-sol:low PASS 100% 68 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-sol:low__json_schema PASS 100% 101 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-sol:max PASS 100% 65 in → 545 out (516 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-sol:max__json_schema PASS 100% 99 in → 554 out (516 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-sol:medium PASS 100% 67 in → 210 out (181 reasoning)
git tag v1.0 e7e1a14 git tag -d v1.0-rc
openai/gpt-5.6-sol:medium__json_schema PASS 100% 101 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-sol:none PASS 100% 67 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-sol:none__json_schema PASS 100% 99 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-sol:xhigh PASS 100% 66 in → 292 out (264 reasoning)
git tag v1.0 bd6fd5c git tag -d v1.0-rc
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 100 in → 215 out (177 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-terra:high PASS 100% 67 in → 191 out (161 reasoning)
git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
openai/gpt-5.6-terra:high__json_schema PASS 100% 101 in → 163 out (126 reasoning)
git tag v1.0 e75f03f git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 e75f03f\ngit tag -d v1.0-rc"}
openai/gpt-5.6-terra:low PASS 100% 66 in → 113 out (84 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-terra:low__json_schema PASS 100% 100 in → 103 out (65 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-terra:max PASS 100% 64 in → 1,060 out (1,034 reasoning)
git tag v1.0 cedf159 git tag -d v1.0-rc
openai/gpt-5.6-terra:medium PASS 100% 64 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-terra:medium__json_schema PASS 100% 99 in → 98 out (61 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc && git tag -d v1.0-rc"}
openai/gpt-5.6-terra:none PASS 100% 66 in → 27 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-5.6-terra:none__json_schema PASS 100% 100 in → 36 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
openai/gpt-5.6-terra:xhigh PASS 100% 64 in → 446 out (420 reasoning)
git tag v1.0 af16279 git tag -d v1.0-rc
openai/gpt-oss-120b:high PASS 100% 125 in → 904 out (873 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-oss-120b:low PASS 100% 129 in → 132 out (94 reasoning)
git tag -d v1.0-rc git tag v1.0 07ee7c5
openai/gpt-oss-120b:medium PASS 100% 128 in → 341 out (250 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
openai/gpt-oss-120b:xhigh PASS 100% 124 in → 1,748 out (1,573 reasoning)
git tag -d v1.0-rc git tag -a v1.0 5209c59 -m "Release v1" git push origin :refs/tags/v1.0-rc git push origin v1.0
openai/gpt-oss-20b:high PASS 100% 131 in → 640 out (554 reasoning)
git tag -d v1.0-rc git tag v1.0 3adf383
openai/gpt-oss-20b:low PASS 100% 124 in → 142 out (72 reasoning)
git tag -f v1.0 v1.0-rc git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin v1.0
openai/gpt-oss-20b:medium PASS 100% 128 in → 267 out (195 reasoning)
git tag -d v1.0-rc git tag v1.0 b26db95
openai/gpt-oss-20b:xhigh PASS 100% 127 in → 489 out (460 reasoning)
git tag -d v1.0-rc git tag v1.0 a6d0633
poolside/laguna-m.1:high PASS 100% 74 in → 583 out (558 reasoning)
git tag -d v1.0-rc git tag v1.0 436fce0
poolside/laguna-m.1:high__json_schema PASS 100% 75 in → 1,175 out (1,140 reasoning)
git tag -d v1.0-rc && git tag v1.0 88f46be
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc && git tag v1.0 88f46be" }
poolside/laguna-m.1:low PASS 100% 76 in → 2,048 out (2,002 reasoning)
git tag v1.0 81b5525 git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
poolside/laguna-m.1:low__json_schema PASS 100% 76 in → 778 out (740 reasoning)
git tag v1.0 299325b git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 299325b\ngit tag -d v1.0-rc\n" }
poolside/laguna-m.1:medium PASS 100% 76 in → 615 out (588 reasoning)
git tag v1.0 7c17e44 git tag -d v1.0-rc
poolside/laguna-m.1:none PASS 100% 74 in → 42 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 0cea67b git push origin :v1.0-rc git push origin v1.0
poolside/laguna-m.1:none__json_schema PASS 100% 74 in → 42 out (0 reasoning)
git tag -d v1.0-rc && git tag -a v1.0 c49bdb6 -m 'Release v1.0'
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc && git tag -a v1.0 c49bdb6 -m 'Release v1.0'" }
poolside/laguna-m.1:xhigh PASS 100% 76 in → 1,098 out (1,056 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin :v1.0-rc git push origin v1.0
poolside/laguna-m.1:xhigh__json_schema PASS 100% 76 in → 1,070 out (1,006 reasoning)
git tag v1.0 $(git rev-parse v1.0-rc) git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
JSON Schema Structured Output
(raw) { "command" : "git tag v1.0 $(git rev-parse v1.0-rc)\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin --delete v1.0-rc" }
poolside/laguna-xs-2.1:high PASS 100% 76 in → 525 out (501 reasoning)
git tag -d v1.0-rc git tag v1.0 d9c1369
poolside/laguna-xs-2.1:low PASS 100% 75 in → 535 out (512 reasoning)
git tag -d v1.0-rc git tag v1.0 de26917
poolside/laguna-xs-2.1:low__json_schema PASS 100% 74 in → 450 out (415 reasoning)
git tag -d v1.0-rc git tag v1.0 0a65eda
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\ngit tag v1.0 0a65eda" }
poolside/laguna-xs-2.1:medium PASS 100% 76 in → 477 out (452 reasoning)
git tag v1.0 5265281 git tag -d v1.0-rc
poolside/laguna-xs-2.1:none PASS 100% 75 in → 23 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 f4ed5c6
poolside/laguna-xs-2.1:none__json_schema PASS 100% 76 in → 34 out (0 reasoning)
git tag -d v1.0-rc && git tag v1.0 b7d395a
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc && git tag v1.0 b7d395a" }
poolside/laguna-xs-2.1:xhigh PASS 100% 76 in → 377 out (352 reasoning)
git tag v1.0 07b1646 git tag -d v1.0-rc
poolside/laguna-xs.2:high PASS 100% 113 in → 196 out (169 reasoning)
git tag -d v1.0-rc git tag v1.0 27884d7
poolside/laguna-xs.2:low PASS 100% 113 in → 186 out (159 reasoning)
git tag -d v1.0-rc git tag v1.0 070770a
poolside/laguna-xs.2:medium PASS 100% 112 in → 537 out (490 reasoning)
git tag -d v1.0-rc git tag v1.0 148bc76 git push origin :refs/tags/v1.0-rc git push origin refs/tags/v1.0
poolside/laguna-xs.2:medium__json_schema PASS 100% 113 in → 159 out (129 reasoning)
git tag -d v1.0-rc && git tag v1.0 d71c576
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc && git tag v1.0 d71c576"}
poolside/laguna-xs.2:none PASS 100% 110 in → 23 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 fef60fe
poolside/laguna-xs.2:none__json_schema PASS 100% 111 in → 28 out (0 reasoning)
git tag -d v1.0-rc && git tag v1.0 afa8f50
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc && git tag v1.0 afa8f50"}
poolside/laguna-xs.2:xhigh PASS 100% 112 in → 228 out (209 reasoning)
git tag -d v1.0-rc git tag v1.0
qwen/qwen3.6-27b:high PASS 100% 74 in → 2,262 out (1,732 reasoning)
git tag v1.0 15f821e git tag -d v1.0-rc
qwen/qwen3.6-27b:high__json_schema PASS 100% 73 in → 2,012 out (1,584 reasoning)
git tag v1.0 3d5fa5f git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 3d5fa5f\ngit tag -d v1.0-rc" }
qwen/qwen3.6-27b:low PASS 100% 73 in → 1,090 out (882 reasoning)
git tag -d v1.0-rc git tag v1.0 42ba2b2
qwen/qwen3.6-27b:low__json_schema PASS 100% 72 in → 1,354 out (1,117 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
qwen/qwen3.6-27b:medium PASS 100% 72 in → 1,628 out (1,353 reasoning)
git tag -d v1.0-rc git tag v1.0 f06ada5
qwen/qwen3.6-27b:medium__json_schema PASS 100% 72 in → 1,558 out (1,343 reasoning)
git tag v1.0 51cdb4f git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 51cdb4f\ngit tag -d v1.0-rc" }
qwen/qwen3.6-27b:none PASS 100% 76 in → 47 out (0 reasoning)
git tag v1.0 76957a7 git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin v1.0
qwen/qwen3.6-27b:xhigh PASS 100% 73 in → 1,614 out (1,586 reasoning)
git tag -d v1.0-rc git tag v1.0 eab58c8
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 73 in → 1,644 out (1,337 reasoning)
git tag v1.0 c1105de git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 c1105de\ngit tag -d v1.0-rc" }
qwen/qwen3.6-35b-a3b:high PASS 100% 74 in → 2,427 out (1,853 reasoning)
git tag -d v1.0-rc git tag v1.0 b08e805
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 73 in → 1,926 out (1,482 reasoning)
git tag -d v1.0-rc && git tag v1.0 5454fb6
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc && git tag v1.0 5454fb6" }
qwen/qwen3.6-35b-a3b:low PASS 100% 73 in → 1,507 out (1,192 reasoning)
git tag -d v1.0-rc git tag v1.0 9dd2086
qwen/qwen3.6-35b-a3b:medium PASS 100% 74 in → 1,805 out (1,336 reasoning)
git tag -d v1.0-rc git tag v1.0 c33917e
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 74 in → 1,685 out (1,287 reasoning)
git tag -d v1.0-rc git tag v1.0 0e4b67f
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\ngit tag v1.0 0e4b67f" }
qwen/qwen3.6-35b-a3b:none PASS 100% 76 in → 45 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 91d7712 git push origin --delete v1.0-rc git push origin v1.0
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 71 in → 1,215 out (962 reasoning)
git tag -d v1.0-rc git tag v1.0 be9ecdf
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 72 in → 2,703 out (2,026 reasoning)
git tag -f v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -f v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
qwen/qwen3.6-flash:high PASS 100% 73 in → 3,334 out (3,304 reasoning)
git tag -f v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.6-flash:low PASS 100% 73 in → 2,470 out (2,442 reasoning)
git tag v1.0 cd7a69b git tag -d v1.0-rc
qwen/qwen3.6-flash:medium PASS 100% 74 in → 2,119 out (2,089 reasoning)
git tag -d v1.0-rc git tag v1.0 446f564
qwen/qwen3.6-flash:none PASS 100% 75 in → 24 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 32cfe11
qwen/qwen3.6-flash:xhigh PASS 100% 74 in → 2,272 out (2,243 reasoning)
git tag -d v1.0-rc git tag v1.0 b835b76
qwen/qwen3.7-flash:high PASS 100% 74 in → 1,624 out (1,595 reasoning)
git tag -d v1.0-rc git tag v1.0 b45720d
qwen/qwen3.7-flash:low PASS 100% 73 in → 1,325 out (1,296 reasoning)
git tag -d v1.0-rc git tag v1.0 31c55df
qwen/qwen3.7-flash:max PASS 100% 73 in → 1,933 out (1,905 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-flash:medium PASS 100% 74 in → 1,479 out (1,450 reasoning)
git tag -d v1.0-rc git tag v1.0 e35c19f
qwen/qwen3.7-flash:none PASS 100% 76 in → 24 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 c3c95b1
qwen/qwen3.7-flash:xhigh PASS 100% 74 in → 1,511 out (1,481 reasoning)
git tag -d v1.0-rc git tag v1.0 062d7f4
qwen/qwen3.7-max:high PASS 100% 73 in → 2,227 out (2,199 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-max:low PASS 100% 72 in → 1,424 out (1,396 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-max:medium PASS 100% 74 in → 1,302 out (1,274 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-max:none PASS 100% 74 in → 46 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin refs/tags/v1.0
qwen/qwen3.7-max:xhigh PASS 100% 73 in → 507 out (479 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-plus:high PASS 100% 73 in → 1,211 out (1,183 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-plus:low PASS 100% 74 in → 1,105 out (1,077 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-plus:medium PASS 100% 74 in → 1,153 out (1,125 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-plus:none PASS 100% 74 in → 23 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
qwen/qwen3.7-plus:xhigh PASS 100% 74 in → 1,428 out (1,400 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
tencent/hy3:high PASS 100% 73 in → 1,144 out (1,119 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
tencent/hy3:high__json_schema PASS 100% 71 in → 1,616 out (1,297 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
tencent/hy3:low PASS 100% 71 in → 1,075 out (1,052 reasoning)
git tag v1.0 025560d git tag -d v1.0-rc
tencent/hy3:low__json_schema PASS 100% 72 in → 677 out (525 reasoning)
git tag -d v1.0-rc git tag v1.0 86af60e
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\ngit tag v1.0 86af60e" }
tencent/hy3:medium PASS 100% 74 in → 987 out (962 reasoning)
git tag v1.0 cf1a6a9 git tag -d v1.0-rc
tencent/hy3:medium__json_schema PASS 100% 74 in → 546 out (400 reasoning)
git tag v1.0 6d9feb4 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 6d9feb4\ngit tag -d v1.0-rc" }
tencent/hy3:none PASS 100% 76 in → 46 out (0 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git push origin v1.0
tencent/hy3:xhigh PASS 100% 74 in → 1,459 out (1,433 reasoning)
git tag v1.0 40f2d8a git tag -d v1.0-rc
tencent/hy3:xhigh__json_schema PASS 100% 72 in → 1,983 out (1,710 reasoning)
git tag v1.0 6892ffd git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 6892ffd\ngit tag -d v1.0-rc" }
thinkingmachines/inkling-small:high PASS 100% 73 in → 211 out (182 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
thinkingmachines/inkling-small:low PASS 100% 73 in → 136 out (84 reasoning)
git tag -d v1.0-rc git tag v1.0 ea06776
thinkingmachines/inkling-small:max PASS 100% 75 in → 327 out (299 reasoning)
git tag v1.0 d738f2c git tag -d v1.0-rc
thinkingmachines/inkling-small:max__json_schema PASS 100% 75 in → 273 out (235 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\n\ngit tag -d v1.0-rc" }
thinkingmachines/inkling-small:medium PASS 100% 74 in → 245 out (216 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
thinkingmachines/inkling-small:medium__json_schema PASS 100% 73 in → 416 out (379 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
thinkingmachines/inkling-small:none PASS 100% 72 in → 25 out (0 reasoning)
git tag v1.0 291e95b git tag -d v1.0-rc
thinkingmachines/inkling-small:xhigh PASS 100% 75 in → 255 out (226 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
thinkingmachines/inkling:high PASS 100% 75 in → 257 out (175 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
thinkingmachines/inkling:low PASS 100% 74 in → 52 out (15 reasoning)
git tag -d v1.0-rc git tag v1.0 c6ae290
thinkingmachines/inkling:max PASS 100% 75 in → 250 out (171 reasoning)
git tag -d v1.0-rc git tag v1.0 0ebda3e
thinkingmachines/inkling:medium PASS 100% 75 in → 88 out (58 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
thinkingmachines/inkling:medium__json_schema PASS 100% 74 in → 182 out (145 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
thinkingmachines/inkling:xhigh PASS 100% 76 in → 360 out (330 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
thinkingmachines/inkling:xhigh__json_schema PASS 100% 73 in → 230 out (195 reasoning)
git tag -d v1.0-rc git tag v1.0 65aa937
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\ngit tag v1.0 65aa937" }
x-ai/grok-4.3:high PASS 100% 244 in → 813 out (792 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.3:high__json_schema PASS 100% 300 in → 2,269 out (2,242 reasoning)
git tag v1.0 f6b6f09 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 f6b6f09\ngit tag -d v1.0-rc"}
x-ai/grok-4.3:low PASS 100% 247 in → 428 out (409 reasoning)
git tag v1.0 abda988 git tag -d v1.0-rc
x-ai/grok-4.3:low__json_schema PASS 100% 304 in → 645 out (615 reasoning)
git tag v1.0 7aeafc1 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 7aeafc1\ngit tag -d v1.0-rc" }
x-ai/grok-4.3:max PASS 100% 242 in → 509 out (488 reasoning)
git tag v1.0 8f41660 git tag -d v1.0-rc
x-ai/grok-4.3:max__json_schema PASS 100% 300 in → 1,703 out (1,672 reasoning)
git tag -d v1.0-rc git tag v1.0 c0a4a11
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\ngit tag v1.0 c0a4a11" }
x-ai/grok-4.3:medium PASS 100% 247 in → 1,359 out (1,339 reasoning)
git tag v1.0 9858069 git tag -d v1.0-rc
x-ai/grok-4.3:medium__json_schema PASS 100% 305 in → 1,244 out (1,213 reasoning)
git tag v1.0 51e8c79 git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 51e8c79\ngit tag -d v1.0-rc" }
x-ai/grok-4.3:none PASS 100% 239 in → 20 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 285567e
x-ai/grok-4.3:xhigh PASS 100% 243 in → 332 out (311 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.3:xhigh__json_schema PASS 100% 297 in → 3,085 out (3,056 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc" }
x-ai/grok-4.5:high PASS 100% 265 in → 449 out (426 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.5:high__json_schema PASS 100% 334 in → 632 out (603 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
x-ai/grok-4.5:low PASS 100% 264 in → 264 out (241 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.5:low__json_schema PASS 100% 331 in → 446 out (417 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
x-ai/grok-4.5:max PASS 100% 264 in → 458 out (435 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.5:max__json_schema PASS 100% 333 in → 861 out (832 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
x-ai/grok-4.5:medium PASS 100% 265 in → 383 out (360 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.5:medium__json_schema PASS 100% 332 in → 692 out (663 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
x-ai/grok-4.5:xhigh PASS 100% 265 in → 479 out (456 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
x-ai/grok-4.5:xhigh__json_schema PASS 100% 332 in → 725 out (696 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc"}
z-ai/glm-4.7-flash:high PASS 100% 65 in → 1,228 out (1,184 reasoning)
git tag -d v1.0-rc git tag v1.0 6c73620
z-ai/glm-4.7-flash:medium PASS 100% 66 in → 958 out (877 reasoning)
git tag -d v1.0-rc git tag v1.0 61f3d6b
z-ai/glm-4.7-flash:none PASS 100% 64 in → 22 out (0 reasoning)
git tag v1.0 9b34cff git tag -d v1.0-rc
z-ai/glm-4.7-flash:xhigh PASS 100% 64 in → 1,473 out (1,449 reasoning)
git tag -d v1.0-rc git tag v1.0 64040ac
z-ai/glm-5.2:high PASS 100% 72 in → 582 out (558 reasoning)
git tag -d v1.0-rc git tag v1.0 4bb398a
z-ai/glm-5.2:high__json_schema PASS 100% 74 in → 1,192 out (1,139 reasoning)
git tag v1.0 v1.0-rc && git tag -d v1.0-rc && git push origin v1.0 && git push origin :refs/tags/v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc && git tag -d v1.0-rc && git push origin v1.0 && git push origin :refs/tags/v1.0-rc" }
z-ai/glm-5.2:low PASS 100% 72 in → 639 out (615 reasoning)
git tag v1.0 04534f3 git tag -d v1.0-rc
z-ai/glm-5.2:medium PASS 100% 73 in → 527 out (412 reasoning)
git tag v1.0 8659e5e git tag -d v1.0-rc
z-ai/glm-5.2:medium__json_schema PASS 100% 73 in → 601 out (566 reasoning)
git tag v1.0 b22f7d6 && git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 b22f7d6 && git tag -d v1.0-rc" }
z-ai/glm-5.2:none PASS 100% 65 in → 43 out (0 reasoning)
git tag v1.0 a89d174 git tag -d v1.0-rc git push origin v1.0 git push origin :refs/tags/v1.0-rc
z-ai/glm-5.2:xhigh PASS 100% 71 in → 1,381 out (1,336 reasoning)
git tag v1.0 v1.0-rc git tag -d v1.0-rc git push origin v1.0 git push origin --delete v1.0-rc
z-ai/glm-5.2:xhigh__json_schema PASS 100% 66 in → 518 out (377 reasoning)
git tag v1.0 f5b003e git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 f5b003e\ngit tag -d v1.0-rc" }
arcee-ai/trinity-mini:low FAIL 66.7% 67 in → 659 out (637 reasoning)
git tag -f v1.0 v1.0-rc
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-flash-0731:low__json_schema FAIL 66.7% 65 in → 272 out (259 reasoning)
git tag v1.0 42b40e0
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 42b40e0" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-flash-0731:max__json_schema FAIL 66.7% 219 in → 438 out (324 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-flash-0731:xhigh__json_schema FAIL 66.7% 64 in → 188 out (141 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-flash:high__json_schema FAIL 66.7% 66 in → 948 out (923 reasoning)
git tag -f v1.0 v1.0-rc^{}
JSON Schema Structured Output
(raw) { "command": "git tag -f v1.0 v1.0-rc^{}" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-flash:none__json_schema FAIL 66.7% 66 in → 22 out (0 reasoning)
git tag -f v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -f v1.0 v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-flash:xhigh__json_schema FAIL 66.7% 142 in → 1,478 out (1,456 reasoning)
git tag v1.0 v1.0-rc^{}
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc^{}" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-pro:low__json_schema FAIL 66.7% 67 in → 576 out (554 reasoning)
git tag -f v1.0 v1.0-rc^{}
JSON Schema Structured Output
(raw) {"command":"git tag -f v1.0 v1.0-rc^{}"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-pro:none FAIL 66.7% 64 in → 42 out (0 reasoning)
git tag v1.0 9598aac git push origin v1.0 git push origin --delete v1.0-rc git tag -d v1.0-rc
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
deepseek/deepseek-v4-pro:xhigh__json_schema FAIL 66.7% 65 in → 56 out (53 reasoning)
git tag v1.0 46c8caa -f -m 'v1.0' && git push origin :refs/tags/v1.0-rc && git push origin v1.0
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 46c8caa -f -m 'v1.0' && git push origin :refs/tags/v1.0-rc && git push origin v1.0" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
google/gemini-3.1-flash-lite-preview:medium__json_schema FAIL 66.7% 64 in → 500 out (483 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
google/gemini-3.1-flash-lite-preview:xhigh__json_schema FAIL 66.7% 65 in → 1,429 out (1,412 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
google/gemini-3.5-flash-lite:low__json_schema FAIL 66.7% 121 in → 47 out (0 reasoning)
git tag -f v1.0 v1.0-rc && git push origin :refs/tags/v1.0-rc && git push origin v1.0
JSON Schema Structured Output
(raw) { "command": "git tag -f v1.0 v1.0-rc && git push origin :refs/tags/v1.0-rc && git push origin v1.0" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
google/gemma-4-31b-it:none__json_schema FAIL 66.7% 75 in → 18 out (0 reasoning)
git tag v1.0 17eaddd
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 17eaddd"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
ibm-granite/granite-4.1-8b__json_schema FAIL 66.7% 72 in → 19 out
git tag v1.0 2425e03
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 2425e03" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
minimax/minimax-m2.5:high__json_schema FAIL 66.7% 97 in → 1,033 out (1,015 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
mistralai/devstral-2512__json_schema FAIL 66.7% 66 in → 19 out
git tag v1.0 b8b1572
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 b8b1572"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
mistralai/mistral-medium-3-5:medium__json_schema FAIL 66.7% 66 in → 2,948 out (2,926 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
mistralai/mistral-medium-3-5:xhigh__json_schema FAIL 66.7% 65 in → 2,436 out (2,293 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
nvidia/nemotron-3-nano-30b-a3b:low__json_schema FAIL 66.7% 78 in → 804 out (717 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\n" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
nvidia/nemotron-3-nano-30b-a3b:none FAIL 66.7% 79 in → 22 out (0 reasoning)
git tag -f v1.0 v1.0-rc git push --force --tags
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema FAIL 66.7% 79 in → 415 out (335 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
nvidia/nemotron-3-super-120b-a12b:high__json_schema FAIL 66.7% 78 in → 393 out (363 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\n" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-5.4-mini:high__json_schema FAIL 66.7% 99 in → 272 out (246 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-5.4-nano:high__json_schema FAIL 66.7% 100 in → 116 out (89 reasoning)
git tag -f v1.0 517f1cc
JSON Schema Structured Output
(raw) {"command":"git tag -f v1.0 517f1cc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-5.4-nano:medium__json_schema FAIL 66.7% 101 in → 25 out (0 reasoning)
git tag -f v1.0 e5afbcf
JSON Schema Structured Output
(raw) {"command":"git tag -f v1.0 e5afbcf"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-5.4-nano:none__json_schema FAIL 66.7% 101 in → 26 out (0 reasoning)
git tag -f v1.0 31f33f1
JSON Schema Structured Output
(raw) {"command":"git tag -f v1.0 31f33f1"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-5.6-terra:max__json_schema FAIL 66.7% 100 in → 542 out (516 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-5.6-terra:xhigh__json_schema FAIL 66.7% 99 in → 321 out (293 reasoning)
git tag v1.0 v1.0-rc^{}
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 v1.0-rc^{}"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-oss-120b:high__json_schema FAIL 66.7% 128 in → 549 out (488 reasoning)
git tag v1.0 8dfa51a
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 8dfa51a"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
openai/gpt-oss-120b:xhigh__json_schema FAIL 66.7% 112 in → 754 out (640 reasoning)
git tag v1.0 be97175
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 be97175"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
poolside/laguna-m.1:medium__json_schema FAIL 66.7% 76 in → 642 out (616 reasoning)
git tag v1.0 3a29c67
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 3a29c67" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
poolside/laguna-xs-2.1:medium__json_schema FAIL 66.7% 76 in → 760 out (739 reasoning)
git tag v1.0 1982f7f
JSON Schema Structured Output
(raw) {"command": "git tag v1.0 1982f7f"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
tencent/hy3:none__json_schema FAIL 66.7% 74 in → 22 out (0 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc\n" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
thinkingmachines/inkling-small:low__json_schema FAIL 66.7% 74 in → 126 out (102 reasoning)
git tag v1.0 3632ca7
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 3632ca7" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
thinkingmachines/inkling-small:xhigh__json_schema FAIL 66.7% 76 in → 334 out (308 reasoning)
git tag v1.0 6e3ddf4
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 6e3ddf4" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
thinkingmachines/inkling:high__json_schema FAIL 66.7% 74 in → 363 out (338 reasoning)
git tag v1.0 v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
x-ai/grok-4.3:none__json_schema FAIL 66.7% 297 in → 16 out (0 reasoning)
git tag v1.0 cee6b4e
JSON Schema Structured Output
(raw) {"command":"git tag v1.0 cee6b4e"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
z-ai/glm-4.7-flash:none__json_schema FAIL 66.7% 64 in → 20 out (0 reasoning)
git tag v1.0 3b2fade
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 3b2fade" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
z-ai/glm-5.2:low__json_schema FAIL 66.7% 71 in → 898 out (721 reasoning)
git tag v1.0 95f5aac
JSON Schema Structured Output
(raw) { "command": "git tag v1.0 95f5aac\n" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}]
arcee-ai/trinity-mini:high FAIL 33.3% 68 in → 514 out (497 reasoning)
gittag -f v1.0 v1.0-rc
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
arcee-ai/trinity-mini:high__json_schema FAIL 33.3% 67 in → 1,044 out (1,057 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command":"git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
arcee-ai/trinity-mini:low__json_schema FAIL 33.3% 67 in → 664 out (654 reasoning)
gittag -d v1.0-rc git tag v1.0 07df4fe
JSON Schema Structured Output
(raw) { "command": "gittag -d v1.0-rc\ngit tag v1.0 07df4fe" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
arcee-ai/trinity-mini:xhigh FAIL 33.3% 66 in → 737 out (721 reasoning)
git tag -f v1.0v1.0-rc
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
deepseek/deepseek-v4-flash:high FAIL 33.3% 66 in → 440 out (376 reasoning)
gittag -d v1.0-rc git tag v1.0 0c3f22e git push origin :refs/tags/v1.0-rc git push origin v1.0
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
google/gemma-4-26b-a4b-it:none__json_schema FAIL 33.3% 76 in → 21 out (0 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
inclusionai/ling-2.6-flash FAIL 33.3% 85 in → 18 out
``` git tag mv v1.0-rc v1.0 ```
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
inclusionai/ling-2.6-flash__json_schema FAIL 33.3% 85 in → 43 out
git tag -d v1.0-rc git tag v1.0 v1.0-rc^{} 6192227^0
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc\n git tag v1.0 v1.0-rc^{} 6192227^0"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
minimax/minimax-m2.5:low FAIL 33.3% 98 in → 669 out (581 reasoning)
git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git tag v1.0 ecf7e02 git push origin v1.0
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
minimax/minimax-m2.5:low__json_schema FAIL 33.3% 97 in → 391 out (361 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
minimax/minimax-m2.5:medium__json_schema FAIL 33.3% 104 in → 171 out (138 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
minimax/minimax-m2.5:xhigh__json_schema FAIL 33.3% 103 in → 941 out (887 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
minimax/minimax-m2.7:medium FAIL 33.3% 97 in → 439 out (366 reasoning)
git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git tag v1.0 1ce90b1 git push origin v1.0
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
nvidia/nemotron-3-nano-30b-a3b:high FAIL 33.3% 78 in → 899 out (748 reasoning)
git tag -d v1.0-rc git tag -a v1.0 61dd8d8
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
nvidia/nemotron-3-nano-30b-a3b:low FAIL 33.3% 79 in → 262 out (198 reasoning)
gittag -d v1.0-rc git tag v1.0 $(git rev-parse v1.0-rc)
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema FAIL 33.3% 79 in → 334 out (230 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
nvidia/nemotron-3-nano-30b-a3b:none__json_schema FAIL 33.3% 77 in → 20 out (0 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\n" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
nvidia/nemotron-3-super-120b-a12b:none__json_schema FAIL 33.3% 79 in → 16 out (0 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
openai/gpt-oss-120b:low__json_schema FAIL 33.3% 126 in → 9 out (0 reasoning)
analysis
JSON Schema Structured Output
(raw) { "command": "analysis" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
openai/gpt-oss-120b:medium__json_schema FAIL 33.3% 123 in → 356 out (298 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
openai/gpt-oss-20b:high__json_schema FAIL 33.3% 128 in → 1,291 out (1,092 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
openai/gpt-oss-20b:low__json_schema FAIL 33.3% 125 in → 5 out (0 reasoning)
analysis
JSON Schema Structured Output
(raw) {"command":"analysis"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
openai/gpt-oss-20b:medium__json_schema FAIL 33.3% 123 in → 425 out (372 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
openai/gpt-oss-20b:xhigh__json_schema FAIL 33.3% 114 in → 1,836 out (1,589 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command":"git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
poolside/laguna-xs-2.1:high__json_schema FAIL 33.3% 74 in → 529 out (512 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc\n"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
poolside/laguna-xs-2.1:xhigh__json_schema FAIL 33.3% 76 in → 354 out (332 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc\n" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
poolside/laguna-xs.2:high__json_schema FAIL 33.3% 113 in → 449 out (434 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
poolside/laguna-xs.2:low__json_schema FAIL 33.3% 111 in → 235 out (219 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
poolside/laguna-xs.2:xhigh__json_schema FAIL 33.3% 111 in → 206 out (190 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
qwen/qwen3.6-35b-a3b:low__json_schema FAIL 33.3% 72 in → 1,429 out (1,170 reasoning)
output_raw_text
JSON Schema Structured Output
(raw) { "command": "output_raw_text" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v1.0-rc'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
thinkingmachines/inkling-small:high__json_schema FAIL 33.3% 76 in → 174 out (152 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) { "command": "git tag -d v1.0-rc" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
thinkingmachines/inkling:none FAIL 33.3% 75 in → 26 out (0 reasoning)
git tag -d v1.0-rc git tag v1.0 1b5d090<|end_message|>
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
z-ai/glm-4.7-flash:low FAIL 33.3% 66 in → 999 out (843 reasoning)
git tag -d v1.0-rc git push origin :refs/tags/v1.0-rc git tag v1.0 07a7a3d git push origin v1.0
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
z-ai/glm-4.7-flash:medium__json_schema FAIL 33.3% 66 in → 1,483 out (1,343 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc\n"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
z-ai/glm-4.7-flash:xhigh__json_schema FAIL 33.3% 64 in → 1,432 out (1,341 reasoning)
git tag -d v1.0-rc
JSON Schema Structured Output
(raw) {"command": "git tag -d v1.0-rc\n"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'contains': 'v1.0'}, {'type': 'git_output', 'command': 'git log v1.0 --oneline -1', 'contains': 'Release v1'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag v1.0 v1.0-rc^{} git tag -d v1.0-rc
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git tag v{ "command": "git tag v1.0 v1.0-rc\ngit tag -d v1.0-rc\ngit push origin v1.0\ngit push origin --delete v1.0-rc" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -l v1.0-rc git tag -l v1.0 git tag v1.0 -m "Rename v1.0-rc to v1.0" git push origin v1.0 git tag -d v1.0-rc
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git tag v1.0 v1.0-rc git tag -d v1.0-rc
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1b967e35-5491-90dd-ad2e-b7098c38b309","request_id":"1b967e35-5491-90dd-ad2e-b7098c38b309"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-fbc41559-1939-9f70-a89a-043a08c148c3","request_id":"fbc41559-1939-9f70-a89a-043a08c148c3"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-10ddb14c-5ea3-9efc-a2c4-bf77e21481c5","request_id":"10ddb14c-5ea3-9efc-a2c4-bf77e21481c5"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a23a485f-de31-9cb1-8748-01afe3ef12c6","request_id":"a23a485f-de31-9cb1-8748-01afe3ef12c6"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-38091801-ab41-915d-a647-5f3f020405aa","request_id":"38091801-ab41-915d-a647-5f3f020405aa"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-139dcbba-aeb1-9c76-bb03-a9f21f9fdb47"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-09969154-f9b6-9cbe-a1f7-b34e31a21c21"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-9f216150-3298-9114-9961-dee44d233383"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-64d84fa8-5ff3-95e6-bcbc-b2a546da0641"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ba84ce2c-142c-94b8-afc5-e98f7e4fa608"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-762e3129-904a-989b-afe8-43d5e734dd34"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-07594204-3687-97b4-a8a2-240348cd4a39","request_id":"07594204-3687-97b4-a8a2-240348cd4a39"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5e828e62-f7d9-9cab-99ae-0b2b92840d11","request_id":"5e828e62-f7d9-9cab-99ae-0b2b92840d11"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-bc7fab75-0f20-993b-819e-2c9381e9b8ac","request_id":"bc7fab75-0f20-993b-819e-2c9381e9b8ac"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-48106b91-566a-918d-999e-60d16fab8fe1","request_id":"48106b91-566a-918d-999e-60d16fab8fe1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1c88091d-e3c2-9eac-bf19-a5cce1043a9f","request_id":"1c88091d-e3c2-9eac-bf19-a5cce1043a9f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-48fe69b9-92cd-95cf-a915-5a4afe060179","request_id":"48fe69b9-92cd-95cf-a915-5a4afe060179"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8c413fe5-7cc9-9c79-ae97-92f8f7f968a1","request_id":"8c413fe5-7cc9-9c79-ae97-92f8f7f968a1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-49e4fe9d-067e-9f46-8b5c-d29f4f747743","request_id":"49e4fe9d-067e-9f46-8b5c-d29f4f747743"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3d42a66d-6447-94e8-90f4-feec87b8500c","request_id":"3d42a66d-6447-94e8-90f4-feec87b8500c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4c312162-f29c-9d30-88c3-6a512eaae1d3","request_id":"4c312162-f29c-9d30-88c3-6a512eaae1d3"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git tag v1.0 2906697 git tag -d v1.0-rc{"command": "git tag v1.0 2906697\n\ngit tag -d v1.0-rc"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag v1.0 ba2d87a git tag -d v1.0-rc
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v1.0-rc git tag v1.0 cd2f001
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v1.0-rc git tag v1.0 5987caa{ "command": "git tag -d v1.0-rc && git tag v1.0 5987caa" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v1.0-rc git tag v1.0 5cadce3
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.