Delete a tag
Tests ability to delete a tag. Evaluates tag-deletion command knowledge.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'App' > app.txt
  5. 05 git add app.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 git tag v0.1-beta
  8. 08 echo 'Release' > release.txt
  9. 09 git add release.txt
  10. 10 git commit -m 'Release v1.0'
  11. 11 git tag v1.0
Prompt
Delete the 'v0.1-beta' tag. Output ONLY the git command, nothing else.
Expected
git tag -d v0.1-beta

Scoped model quality, cost, API time, and token usage for tag_management/f003.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 85 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-fable-5:high__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-fable-5:low PASS 100% 82 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-fable-5:low__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-fable-5:max PASS 100% 83 in → 79 out (18 reasoning)
git tag -d v0.1-beta
anthropic/claude-fable-5:max__json_schema PASS 100% 294 in → 88 out (41 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-fable-5:medium PASS 100% 83 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-fable-5:xhigh PASS 100% 83 in → 35 out (13 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-haiku-4.5:high PASS 100% 94 in → 225 out (213 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 261 in → 182 out (150 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-haiku-4.5:low PASS 100% 97 in → 242 out (211 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 259 in → 256 out (221 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-haiku-4.5:medium PASS 100% 95 in → 222 out (206 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 260 in → 234 out (200 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-haiku-4.5:none PASS 100% 65 in → 17 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 228 in → 18 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 96 in → 299 out (267 reasoning)
git tag -d v0.1-beta
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 260 in → 188 out (147 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-opus-4.6:high PASS 100% 68 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 229 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.6:low PASS 100% 68 in → 13 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-opus-4.6:low__json_schema PASS 100% 229 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.6:max PASS 100% 66 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 229 in → 69 out (28 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.6:medium PASS 100% 66 in → 13 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 234 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.6:none PASS 100% 68 in → 17 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 231 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.6:xhigh PASS 100% 67 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 227 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.7:high PASS 100% 87 in → 18 out (0 reasoning)
`git tag -d v0.1-beta`
anthropic/claude-opus-4.7:high__json_schema PASS 100% 299 in → 23 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.7:low PASS 100% 89 in → 16 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-opus-4.7:low__json_schema PASS 100% 300 in → 23 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.7:max PASS 100% 86 in → 21 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 295 in → 23 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.7:medium PASS 100% 90 in → 18 out (0 reasoning)
`git tag -d v0.1-beta`
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 299 in → 23 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.7:none PASS 100% 90 in → 21 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.7:none__json_schema PASS 100% 300 in → 23 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.7:xhigh PASS 100% 92 in → 21 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 300 in → 23 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.8:high PASS 100% 86 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-opus-4.8:high__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.8:low PASS 100% 85 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-opus-4.8:low__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.8:max PASS 100% 85 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-opus-4.8:medium PASS 100% 85 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.8:none PASS 100% 84 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 294 in → 20 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-opus-4.8:xhigh PASS 100% 83 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-4.6:high PASS 100% 66 in → 17 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 230 in → 49 out (12 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-4.6:low PASS 100% 68 in → 17 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 229 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-4.6:max PASS 100% 68 in → 70 out (29 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 230 in → 70 out (29 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-4.6:medium PASS 100% 71 in → 17 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 229 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-4.6:none PASS 100% 65 in → 17 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 229 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 66 in → 51 out (14 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 231 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-5:high PASS 100% 87 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-sonnet-5:high__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-5:low PASS 100% 84 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-sonnet-5:low__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-5:medium PASS 100% 86 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
anthropic/claude-sonnet-5:none PASS 100% 81 in → 18 out (0 reasoning)
``` git tag -d v0.1-beta ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 294 in → 20 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
anthropic/claude-sonnet-5:xhigh PASS 100% 85 in → 14 out (0 reasoning)
git tag -d v0.1-beta
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 296 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
arcee-ai/trinity-large-thinking:high PASS 100% 61 in → 282 out (270 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-large-thinking:low PASS 100% 61 in → 1,530 out (1,522 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-large-thinking:medium PASS 100% 64 in → 996 out (987 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-large-thinking:xhigh PASS 100% 65 in → 665 out (654 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-mini:high__json_schema PASS 100% 62 in → 300 out (282 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
arcee-ai/trinity-mini:low PASS 100% 62 in → 284 out (279 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-mini:low__json_schema PASS 100% 61 in → 288 out (283 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
arcee-ai/trinity-mini:medium PASS 100% 61 in → 322 out (315 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-mini:medium__json_schema PASS 100% 63 in → 737 out (773 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
arcee-ai/trinity-mini:xhigh PASS 100% 60 in → 373 out (369 reasoning)
git tag -d v0.1-beta
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 63 in → 263 out (251 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 57 in → 154 out (122 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 57 in → 198 out (183 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 58 in → 96 out (81 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 60 in → 199 out (170 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-flash-0731:max PASS 100% 136 in → 48 out (33 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 149 in → 131 out (94 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
deepseek/deepseek-v4-flash-0731:medium PASS 100% 56 in → 72 out (61 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 57 in → 180 out (152 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
deepseek/deepseek-v4-flash-0731:none PASS 100% 57 in → 10 out (0 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 55 in → 18 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 61 in → 84 out (74 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 137 in → 115 out (98 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
deepseek/deepseek-v4-flash:high PASS 100% 58 in → 247 out (237 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash:low PASS 100% 59 in → 79 out (68 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 57 in → 100 out (81 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-flash:medium PASS 100% 55 in → 81 out (66 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 59 in → 231 out (198 reasoning)
git tag -d v0.1-beta && git push origin :refs/tags/v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta && git push origin :refs/tags/v0.1-beta" }
deepseek/deepseek-v4-flash:none PASS 100% 56 in → 10 out (0 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 59 in → 18 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 136 in → 59 out (48 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 138 in → 164 out (146 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-pro:high PASS 100% 57 in → 96 out (86 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 244 in → 104 out (88 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
deepseek/deepseek-v4-pro:low PASS 100% 55 in → 164 out (135 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 244 in → 189 out (173 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
deepseek/deepseek-v4-pro:medium PASS 100% 57 in → 138 out (127 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 60 in → 369 out (350 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
deepseek/deepseek-v4-pro:none PASS 100% 54 in → 10 out (0 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-pro:xhigh PASS 100% 137 in → 471 out (460 reasoning)
git tag -d v0.1-beta
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 136 in → 449 out (435 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemini-3-flash-preview:high PASS 100% 59 in → 143 out (133 reasoning)
git tag -d v0.1-beta
google/gemini-3-flash-preview:high__json_schema PASS 100% 118 in → 309 out (289 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemini-3-flash-preview:low PASS 100% 60 in → 213 out (203 reasoning)
git tag -d v0.1-beta
google/gemini-3-flash-preview:low__json_schema PASS 100% 118 in → 187 out (173 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3-flash-preview:medium PASS 100% 62 in → 296 out (286 reasoning)
git tag -d v0.1-beta
google/gemini-3-flash-preview:medium__json_schema PASS 100% 116 in → 322 out (308 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3-flash-preview:xhigh PASS 100% 62 in → 229 out (219 reasoning)
git tag -d v0.1-beta
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 117 in → 192 out (178 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 63 in → 367 out (357 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 63 in → 247 out (232 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 60 in → 140 out (130 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 61 in → 144 out (124 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemini-3.1-flash-lite-preview:medium PASS 100% 61 in → 205 out (195 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 118 in → 172 out (157 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 61 in → 268 out (258 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 60 in → 238 out (223 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.1-pro-preview:high PASS 100% 59 in → 302 out (292 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 118 in → 288 out (275 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.1-pro-preview:low PASS 100% 61 in → 112 out (102 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 118 in → 64 out (50 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.1-pro-preview:medium PASS 100% 60 in → 196 out (186 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 117 in → 268 out (254 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 62 in → 316 out (306 reasoning)
git tag -d v0.1-beta
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 118 in → 148 out (134 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.5-flash-lite:high PASS 100% 61 in → 356 out (346 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 118 in → 573 out (559 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.5-flash-lite:low PASS 100% 63 in → 10 out (0 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 117 in → 15 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.5-flash-lite:max PASS 100% 62 in → 449 out (439 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 115 in → 278 out (263 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.5-flash-lite:medium PASS 100% 63 in → 226 out (216 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 62 in → 326 out (311 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 61 in → 419 out (409 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 117 in → 334 out (319 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.5-flash:high PASS 100% 59 in → 383 out (373 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash:high__json_schema PASS 100% 116 in → 409 out (394 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.5-flash:low PASS 100% 60 in → 263 out (253 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash:low__json_schema PASS 100% 119 in → 84 out (69 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
google/gemini-3.5-flash:medium PASS 100% 62 in → 276 out (266 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash:medium__json_schema PASS 100% 116 in → 331 out (317 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.5-flash:xhigh PASS 100% 59 in → 302 out (292 reasoning)
git tag -d v0.1-beta
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 117 in → 198 out (184 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.6-flash:high PASS 100% 62 in → 199 out (189 reasoning)
git tag -d v0.1-beta
google/gemini-3.6-flash:high__json_schema PASS 100% 117 in → 316 out (302 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.6-flash:low PASS 100% 63 in → 152 out (142 reasoning)
git tag -d v0.1-beta
google/gemini-3.6-flash:low__json_schema PASS 100% 60 in → 14 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.6-flash:max PASS 100% 60 in → 244 out (234 reasoning)
git tag -d v0.1-beta
google/gemini-3.6-flash:max__json_schema PASS 100% 119 in → 207 out (193 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.6-flash:medium PASS 100% 63 in → 219 out (209 reasoning)
git tag -d v0.1-beta
google/gemini-3.6-flash:medium__json_schema PASS 100% 119 in → 275 out (261 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemini-3.6-flash:xhigh PASS 100% 60 in → 204 out (194 reasoning)
git tag -d v0.1-beta
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 117 in → 193 out (179 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
google/gemma-4-26b-a4b-it:high PASS 100% 77 in → 273 out (262 reasoning)
git tag -d v0.1-beta
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 77 in → 272 out (202 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-26b-a4b-it:low PASS 100% 76 in → 465 out (388 reasoning)
git tag -d v0.1-beta
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 77 in → 313 out (256 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-26b-a4b-it:medium PASS 100% 77 in → 227 out (197 reasoning)
git tag -d v0.1-beta
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 78 in → 324 out (255 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-26b-a4b-it:none PASS 100% 75 in → 11 out (0 reasoning)
git tag -d v0.1-beta
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 74 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 76 in → 270 out (259 reasoning)
git tag -d v0.1-beta
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 77 in → 291 out (229 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-31b-it:high PASS 100% 73 in → 196 out (148 reasoning)
git tag -d v0.1-beta
google/gemma-4-31b-it:high__json_schema PASS 100% 77 in → 195 out (151 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-31b-it:low PASS 100% 78 in → 256 out (202 reasoning)
git tag -d v0.1-beta
google/gemma-4-31b-it:low__json_schema PASS 100% 77 in → 183 out (130 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-31b-it:medium PASS 100% 75 in → 177 out (146 reasoning)
git tag -d v0.1-beta
google/gemma-4-31b-it:medium__json_schema PASS 100% 76 in → 223 out (173 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-31b-it:none PASS 100% 72 in → 11 out (0 reasoning)
git tag -d v0.1-beta
google/gemma-4-31b-it:none__json_schema PASS 100% 74 in → 18 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
google/gemma-4-31b-it:xhigh PASS 100% 78 in → 165 out (131 reasoning)
git tag -d v0.1-beta
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 76 in → 176 out (126 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
ibm-granite/granite-4.1-8b PASS 100% 68 in → 10 out
git tag -d v0.1-beta
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
inclusionai/ling-2.6-flash PASS 100% 77 in → 10 out
git tag -d v0.1-beta
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
liquid/lfm-2-24b-a2b PASS 100% 65 in → 11 out
git tag -d v0.1-beta
minimax/minimax-m2.5:high PASS 100% 91 in → 594 out (551 reasoning)
git tag -d v0.1-beta && git push origin :refs/tags/v0.1-beta
minimax/minimax-m2.5:high__json_schema PASS 100% 91 in → 163 out (148 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m2.5:low PASS 100% 92 in → 235 out (225 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.5:low__json_schema PASS 100% 89 in → 135 out (119 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m2.5:medium PASS 100% 98 in → 128 out (116 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.5:medium__json_schema PASS 100% 90 in → 177 out (159 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m2.5:xhigh PASS 100% 90 in → 254 out (249 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 90 in → 623 out (606 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m2.7:high PASS 100% 94 in → 119 out (109 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.7:high__json_schema PASS 100% 224 in → 384 out (369 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
minimax/minimax-m2.7:low PASS 100% 91 in → 229 out (219 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.7:medium PASS 100% 93 in → 312 out (302 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.7:medium__json_schema PASS 100% 223 in → 555 out (536 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m2.7:xhigh PASS 100% 92 in → 546 out (536 reasoning)
git tag -d v0.1-beta
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 222 in → 402 out (383 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m3:high PASS 100% 230 in → 28 out (16 reasoning)
git tag -d v0.1-beta
minimax/minimax-m3:high__json_schema PASS 100% 215 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m3:low PASS 100% 226 in → 33 out (20 reasoning)
git tag -d v0.1-beta
minimax/minimax-m3:medium PASS 100% 229 in → 37 out (28 reasoning)
git tag -d v0.1-beta
minimax/minimax-m3:medium__json_schema PASS 100% 215 in → 18 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
minimax/minimax-m3:xhigh PASS 100% 227 in → 33 out (22 reasoning)
git tag -d v0.1-beta
minimax/minimax-m3:xhigh__json_schema PASS 100% 216 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
mistralai/devstral-2512 PASS 100% 59 in → 15 out
```bash git tag -d v0.1-beta ```
mistralai/devstral-2512__json_schema PASS 100% 59 in → 15 out
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
mistralai/mistral-medium-3-5:high PASS 100% 71 in → 193 out (176 reasoning)
git tag -d v0.1-beta
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 58 in → 348 out (327 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
mistralai/mistral-medium-3-5:low PASS 100% 72 in → 144 out (119 reasoning)
git tag -d v0.1-beta
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 60 in → 270 out (257 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
mistralai/mistral-medium-3-5:medium PASS 100% 73 in → 248 out (227 reasoning)
git tag -d v0.1-beta
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 58 in → 210 out (201 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
mistralai/mistral-medium-3-5:none PASS 100% 71 in → 10 out (0 reasoning)
git tag -d v0.1-beta
mistralai/mistral-medium-3-5:xhigh PASS 100% 71 in → 201 out (189 reasoning)
git tag -d v0.1-beta
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 61 in → 255 out (256 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
moonshotai/kimi-k2.6:high PASS 100% 58 in → 581 out (570 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.6:high__json_schema PASS 100% 59 in → 786 out (620 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
moonshotai/kimi-k2.6:low PASS 100% 62 in → 593 out (636 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.6:medium PASS 100% 61 in → 255 out (244 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 243 in → 506 out (491 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
moonshotai/kimi-k2.6:xhigh PASS 100% 61 in → 419 out (447 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.7-code:high PASS 100% 63 in → 107 out (98 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 61 in → 139 out (121 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
moonshotai/kimi-k2.7-code:low PASS 100% 59 in → 114 out (103 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.7-code:medium PASS 100% 60 in → 157 out (146 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 60 in → 132 out (114 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 60 in → 121 out (110 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 59 in → 70 out (51 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
moonshotai/kimi-k3:high PASS 100% 137 in → 94 out (67 reasoning)
``` git tag -d v0.1-beta ```
moonshotai/kimi-k3:high__json_schema PASS 100% 234 in → 73 out (44 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
moonshotai/kimi-k3:low PASS 100% 137 in → 65 out (41 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k3:low__json_schema PASS 100% 138 in → 150 out (126 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k3:max PASS 100% 137 in → 80 out (56 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k3:max__json_schema PASS 100% 235 in → 82 out (53 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
moonshotai/kimi-k3:medium PASS 100% 137 in → 95 out (71 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k3:medium__json_schema PASS 100% 135 in → 82 out (58 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k3:xhigh PASS 100% 137 in → 134 out (110 reasoning)
git tag -d v0.1-beta
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 140 in → 86 out (59 reasoning)
``` git tag -d v0.1-beta ```
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 73 in → 1,383 out (1,422 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 74 in → 95 out (66 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 71 in → 137 out (116 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 74 in → 209 out (179 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 71 in → 240 out (209 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 72 in → 52 out (28 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 73 in → 10 out (0 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 73 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 74 in → 174 out (153 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 74 in → 117 out (98 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 74 in → 112 out (100 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 73 in → 153 out (133 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 74 in → 82 out (69 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 74 in → 620 out (596 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 73 in → 116 out (104 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 74 in → 142 out (122 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 71 in → 10 out (0 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 72 in → 18 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 72 in → 47 out (35 reasoning)
git tag -d v0.1-beta
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 74 in → 131 out (111 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
openai/gpt-5.4-mini:high PASS 100% 57 in → 140 out (125 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-mini:high__json_schema PASS 100% 94 in → 78 out (55 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-mini:low PASS 100% 56 in → 40 out (25 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-mini:low__json_schema PASS 100% 94 in → 65 out (42 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-mini:medium PASS 100% 58 in → 104 out (89 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-mini:medium__json_schema PASS 100% 94 in → 80 out (57 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-mini:none PASS 100% 57 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-mini:none__json_schema PASS 100% 92 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-mini:xhigh PASS 100% 58 in → 154 out (139 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 95 in → 186 out (163 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-nano:high PASS 100% 57 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-nano:high__json_schema PASS 100% 94 in → 34 out (0 reasoning)
git tag -d v0.1-beta && git push origin :refs/tags/v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta && git push origin :refs/tags/v0.1-beta"}
openai/gpt-5.4-nano:low PASS 100% 59 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-nano:low__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-nano:medium PASS 100% 59 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-nano:medium__json_schema PASS 100% 94 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-nano:none PASS 100% 58 in → 26 out (0 reasoning)
git tag -d v0.1-beta && git push origin :refs/tags/v0.1-beta
openai/gpt-5.4-nano:none__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4-nano:xhigh PASS 100% 58 in → 131 out (116 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 93 in → 401 out (378 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4:high PASS 100% 60 in → 74 out (59 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4:high__json_schema PASS 100% 94 in → 75 out (52 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4:low PASS 100% 60 in → 50 out (35 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4:low__json_schema PASS 100% 91 in → 47 out (24 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4:medium PASS 100% 58 in → 77 out (62 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4:medium__json_schema PASS 100% 93 in → 78 out (55 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4:none PASS 100% 58 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4:none__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.4:xhigh PASS 100% 57 in → 140 out (125 reasoning)
git tag -d v0.1-beta
openai/gpt-5.4:xhigh__json_schema PASS 100% 92 in → 539 out (516 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.5:high PASS 100% 60 in → 70 out (55 reasoning)
git tag -d v0.1-beta
openai/gpt-5.5:high__json_schema PASS 100% 94 in → 93 out (70 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.5:low PASS 100% 60 in → 50 out (35 reasoning)
git tag -d v0.1-beta
openai/gpt-5.5:low__json_schema PASS 100% 94 in → 22 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
openai/gpt-5.5:medium PASS 100% 59 in → 48 out (33 reasoning)
git tag -d v0.1-beta
openai/gpt-5.5:medium__json_schema PASS 100% 95 in → 86 out (63 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.5:none PASS 100% 57 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.5:none__json_schema PASS 100% 94 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.5:xhigh PASS 100% 59 in → 115 out (100 reasoning)
git tag -d v0.1-beta
openai/gpt-5.5:xhigh__json_schema PASS 100% 93 in → 124 out (101 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-luna:high PASS 100% 59 in → 69 out (54 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-luna:high__json_schema PASS 100% 92 in → 107 out (84 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-luna:low PASS 100% 58 in → 39 out (24 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-luna:low__json_schema PASS 100% 93 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-luna:max PASS 100% 57 in → 80 out (65 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-luna:max__json_schema PASS 100% 92 in → 100 out (77 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-luna:medium PASS 100% 59 in → 39 out (24 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-luna:medium__json_schema PASS 100% 94 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-luna:none PASS 100% 57 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-luna:none__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-luna:xhigh PASS 100% 57 in → 66 out (51 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 92 in → 55 out (32 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-sol:high PASS 100% 57 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-sol:high__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-sol:low PASS 100% 58 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-sol:low__json_schema PASS 100% 93 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-sol:max PASS 100% 58 in → 44 out (29 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-sol:max__json_schema PASS 100% 98 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-sol:medium PASS 100% 60 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-sol:medium__json_schema PASS 100% 93 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-sol:none PASS 100% 59 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-sol:none__json_schema PASS 100% 92 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-sol:xhigh PASS 100% 57 in → 65 out (50 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-terra:high PASS 100% 59 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-terra:high__json_schema PASS 100% 94 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-terra:low PASS 100% 57 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-terra:low__json_schema PASS 100% 92 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-terra:max PASS 100% 60 in → 49 out (34 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-terra:max__json_schema PASS 100% 94 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-terra:medium PASS 100% 59 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-terra:medium__json_schema PASS 100% 95 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-terra:none PASS 100% 59 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-terra:none__json_schema PASS 100% 94 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-5.6-terra:xhigh PASS 100% 61 in → 13 out (0 reasoning)
git tag -d v0.1-beta
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 92 in → 21 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-oss-120b:high PASS 100% 120 in → 545 out (581 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-120b:high__json_schema PASS 100% 120 in → 230 out (210 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-oss-120b:low PASS 100% 120 in → 45 out (21 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-120b:low__json_schema PASS 100% 104 in → 64 out (37 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-oss-120b:medium PASS 100% 109 in → 159 out (126 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-120b:medium__json_schema PASS 100% 120 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
openai/gpt-oss-120b:xhigh PASS 100% 118 in → 150 out (128 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 120 in → 851 out (838 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-oss-20b:high PASS 100% 120 in → 80 out (61 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-20b:high__json_schema PASS 100% 117 in → 226 out (187 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
openai/gpt-oss-20b:low PASS 100% 105 in → 50 out (24 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-20b:low__json_schema PASS 100% 107 in → 48 out (16 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
openai/gpt-oss-20b:medium PASS 100% 117 in → 73 out (53 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-20b:medium__json_schema PASS 100% 106 in → 88 out (56 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
openai/gpt-oss-20b:xhigh PASS 100% 121 in → 320 out (283 reasoning)
git tag -d v0.1-beta
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 120 in → 303 out (279 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-m.1:high PASS 100% 74 in → 403 out (392 reasoning)
git tag -d v0.1-beta
poolside/laguna-m.1:high__json_schema PASS 100% 73 in → 155 out (140 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
poolside/laguna-m.1:low PASS 100% 72 in → 767 out (757 reasoning)
git tag -d v0.1-beta
poolside/laguna-m.1:low__json_schema PASS 100% 72 in → 1,309 out (1,295 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
poolside/laguna-m.1:medium PASS 100% 73 in → 670 out (649 reasoning)
git tag -d v0.1-beta && git push origin --delete v0.1-beta
poolside/laguna-m.1:medium__json_schema PASS 100% 72 in → 564 out (545 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-m.1:none PASS 100% 74 in → 11 out (0 reasoning)
git tag -d v0.1-beta
poolside/laguna-m.1:xhigh PASS 100% 74 in → 1,163 out (1,143 reasoning)
git tag -d v0.1-beta && git push origin --delete v0.1-beta
poolside/laguna-m.1:xhigh__json_schema PASS 100% 73 in → 381 out (360 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-xs-2.1:high PASS 100% 71 in → 300 out (290 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs-2.1:high__json_schema PASS 100% 73 in → 337 out (316 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-xs-2.1:low PASS 100% 71 in → 308 out (298 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs-2.1:low__json_schema PASS 100% 70 in → 354 out (338 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
poolside/laguna-xs-2.1:medium PASS 100% 72 in → 291 out (281 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 72 in → 383 out (364 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-xs-2.1:none PASS 100% 72 in → 9 out (0 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs-2.1:none__json_schema PASS 100% 73 in → 20 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-xs-2.1:xhigh PASS 100% 73 in → 322 out (313 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 73 in → 729 out (710 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-xs.2:high PASS 100% 110 in → 156 out (144 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs.2:high__json_schema PASS 100% 109 in → 246 out (225 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
poolside/laguna-xs.2:low PASS 100% 109 in → 152 out (140 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs.2:low__json_schema PASS 100% 108 in → 166 out (151 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
poolside/laguna-xs.2:medium__json_schema PASS 100% 110 in → 175 out (159 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
poolside/laguna-xs.2:xhigh PASS 100% 107 in → 312 out (300 reasoning)
git tag -d v0.1-beta
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 109 in → 96 out (80 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
qwen/qwen3.6-27b:high PASS 100% 67 in → 1,244 out (1,138 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-27b:high__json_schema PASS 100% 69 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-27b:low PASS 100% 70 in → 869 out (767 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-27b:low__json_schema PASS 100% 68 in → 769 out (1 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-27b:medium PASS 100% 70 in → 986 out (972 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-27b:medium__json_schema PASS 100% 70 in → 1,043 out (891 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-27b:none PASS 100% 70 in → 10 out (0 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-27b:xhigh PASS 100% 68 in → 958 out (859 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 69 in → 1,072 out (912 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-35b-a3b:high PASS 100% 68 in → 966 out (857 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 68 in → 323 out (259 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-35b-a3b:low PASS 100% 68 in → 911 out (797 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 66 in → 843 out (758 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 69 in → 585 out (522 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 70 in → 849 out (746 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-35b-a3b:none PASS 100% 69 in → 10 out (0 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 69 in → 992 out (978 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 70 in → 309 out (238 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
qwen/qwen3.6-flash:high PASS 100% 69 in → 653 out (639 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-flash:low PASS 100% 68 in → 1,154 out (1,140 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-flash:medium PASS 100% 70 in → 801 out (787 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-flash:none PASS 100% 72 in → 9 out (0 reasoning)
git tag -d v0.1-beta
qwen/qwen3.6-flash:xhigh PASS 100% 69 in → 679 out (665 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-flash:high PASS 100% 68 in → 603 out (589 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-flash:low PASS 100% 70 in → 738 out (724 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-flash:max PASS 100% 70 in → 305 out (291 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-flash:medium PASS 100% 70 in → 1,173 out (1,159 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-flash:none PASS 100% 72 in → 9 out (0 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-flash:xhigh PASS 100% 70 in → 911 out (897 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-max:high PASS 100% 67 in → 181 out (167 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-max:low PASS 100% 70 in → 520 out (506 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-max:medium PASS 100% 70 in → 95 out (81 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-max:none PASS 100% 69 in → 9 out (0 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-max:xhigh PASS 100% 69 in → 459 out (445 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-plus:high PASS 100% 69 in → 564 out (550 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-plus:low PASS 100% 69 in → 534 out (520 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-plus:medium PASS 100% 70 in → 620 out (606 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-plus:none PASS 100% 71 in → 9 out (0 reasoning)
git tag -d v0.1-beta
qwen/qwen3.7-plus:xhigh PASS 100% 70 in → 451 out (437 reasoning)
git tag -d v0.1-beta
tencent/hy3:high PASS 100% 64 in → 575 out (526 reasoning)
git tag -d v0.1-beta
tencent/hy3:high__json_schema PASS 100% 66 in → 706 out (646 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
tencent/hy3:low PASS 100% 63 in → 303 out (292 reasoning)
git tag -d v0.1-beta
tencent/hy3:low__json_schema PASS 100% 62 in → 314 out (277 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
tencent/hy3:medium PASS 100% 67 in → 410 out (399 reasoning)
git tag -d v0.1-beta
tencent/hy3:medium__json_schema PASS 100% 65 in → 397 out (347 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
tencent/hy3:none PASS 100% 67 in → 10 out (0 reasoning)
git tag -d v0.1-beta
tencent/hy3:none__json_schema PASS 100% 66 in → 17 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
tencent/hy3:xhigh PASS 100% 63 in → 454 out (443 reasoning)
git tag -d v0.1-beta
tencent/hy3:xhigh__json_schema PASS 100% 65 in → 654 out (621 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling-small:high PASS 100% 67 in → 137 out (112 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling-small:high__json_schema PASS 100% 67 in → 147 out (126 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling-small:low PASS 100% 69 in → 61 out (36 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling-small:low__json_schema PASS 100% 69 in → 46 out (24 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling-small:max PASS 100% 70 in → 100 out (81 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling-small:max__json_schema PASS 100% 69 in → 96 out (77 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
thinkingmachines/inkling-small:medium PASS 100% 70 in → 74 out (56 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling-small:medium__json_schema PASS 100% 68 in → 77 out (55 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling-small:none PASS 100% 68 in → 12 out (0 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling-small:xhigh PASS 100% 69 in → 130 out (115 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 69 in → 127 out (108 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
thinkingmachines/inkling:high PASS 100% 69 in → 98 out (82 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling:high__json_schema PASS 100% 68 in → 90 out (69 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling:low PASS 100% 68 in → 39 out (20 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling:low__json_schema PASS 100% 65 in → 35 out (13 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling:max PASS 100% 66 in → 58 out (41 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling:medium PASS 100% 69 in → 72 out (56 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling:none PASS 100% 64 in → 12 out (0 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling:none__json_schema PASS 100% 68 in → 19 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
thinkingmachines/inkling:xhigh PASS 100% 70 in → 95 out (80 reasoning)
git tag -d v0.1-beta
thinkingmachines/inkling:xhigh__json_schema PASS 100% 66 in → 101 out (79 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
x-ai/grok-4.3:high PASS 100% 237 in → 285 out (276 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.3:high__json_schema PASS 100% 293 in → 310 out (297 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
x-ai/grok-4.3:low PASS 100% 245 in → 174 out (165 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.3:low__json_schema PASS 100% 299 in → 271 out (258 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
x-ai/grok-4.3:max PASS 100% 236 in → 121 out (112 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.3:max__json_schema PASS 100% 292 in → 361 out (344 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
x-ai/grok-4.3:medium PASS 100% 245 in → 734 out (725 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.3:medium__json_schema PASS 100% 299 in → 278 out (265 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
x-ai/grok-4.3:none PASS 100% 236 in → 9 out (0 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.3:none__json_schema PASS 100% 291 in → 13 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
x-ai/grok-4.3:xhigh PASS 100% 240 in → 168 out (159 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.3:xhigh__json_schema PASS 100% 293 in → 260 out (247 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
x-ai/grok-4.5:high PASS 100% 260 in → 183 out (173 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.5:high__json_schema PASS 100% 328 in → 293 out (278 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
x-ai/grok-4.5:low PASS 100% 258 in → 177 out (163 reasoning)
``` git tag -d v0.1-beta ```
x-ai/grok-4.5:low__json_schema PASS 100% 329 in → 238 out (223 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
x-ai/grok-4.5:max PASS 100% 262 in → 183 out (173 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.5:max__json_schema PASS 100% 329 in → 234 out (219 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
x-ai/grok-4.5:medium PASS 100% 258 in → 196 out (186 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.5:medium__json_schema PASS 100% 326 in → 165 out (150 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
x-ai/grok-4.5:xhigh PASS 100% 261 in → 152 out (142 reasoning)
git tag -d v0.1-beta
x-ai/grok-4.5:xhigh__json_schema PASS 100% 326 in → 319 out (304 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
z-ai/glm-4.7-flash:high PASS 100% 59 in → 434 out (401 reasoning)
git tag -d v0.1-beta
z-ai/glm-4.7-flash:low PASS 100% 59 in → 835 out (832 reasoning)
git tag -d v0.1-beta
z-ai/glm-4.7-flash:low__json_schema PASS 100% 57 in → 290 out (248 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
z-ai/glm-4.7-flash:medium PASS 100% 59 in → 99 out (85 reasoning)
git tag -d v0.1-beta
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 58 in → 487 out (442 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
z-ai/glm-4.7-flash:none PASS 100% 57 in → 9 out (0 reasoning)
git tag -d v0.1-beta
z-ai/glm-4.7-flash:none__json_schema PASS 100% 61 in → 15 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git tag -d v0.1-beta"}
z-ai/glm-4.7-flash:xhigh PASS 100% 59 in → 837 out (826 reasoning)
git tag -d v0.1-beta
z-ai/glm-5.2:high PASS 100% 66 in → 261 out (250 reasoning)
git tag -d v0.1-beta
z-ai/glm-5.2:high__json_schema PASS 100% 66 in → 335 out (317 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) { "command": "git tag -d v0.1-beta" }
z-ai/glm-5.2:low PASS 100% 66 in → 441 out (430 reasoning)
git tag -d v0.1-beta
z-ai/glm-5.2:low__json_schema PASS 100% 241 in → 15 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
z-ai/glm-5.2:medium PASS 100% 64 in → 214 out (186 reasoning)
git tag -d v0.1-beta
z-ai/glm-5.2:medium__json_schema PASS 100% 240 in → 15 out (0 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
z-ai/glm-5.2:none PASS 100% 60 in → 10 out (0 reasoning)
git tag -d v0.1-beta
z-ai/glm-5.2:xhigh PASS 100% 66 in → 299 out (0 reasoning)
git tag -d v0.1-beta
z-ai/glm-5.2:xhigh__json_schema PASS 100% 241 in → 223 out (208 reasoning)
git tag -d v0.1-beta
JSON Schema Structured Output
(raw) {"command":"git tag -d v0.1-beta"}
arcee-ai/trinity-mini:high FAIL 50% 63 in → 440 out (466 reasoning)
gittag -d v0.1-beta
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v0.1-beta'}]
poolside/laguna-xs.2:medium FAIL 50% 110 in → 169 out (156 reasoning)
git push --delete origin v0.1-beta
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v0.1-beta'}]
poolside/laguna-xs.2:none FAIL 50% 111 in → 12 out (0 reasoning)
git push --delete origin v0.1-beta
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v0.1-beta'}]
poolside/laguna-xs.2:none__json_schema FAIL 50% 108 in → 17 out (0 reasoning)
git push origin --delete tag v0.1-beta
JSON Schema Structured Output
(raw) {"command": "git push origin --delete tag v0.1-beta"}
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git tag -l', 'not_contains': 'v0.1-beta'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v0.1-beta
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git tag -d v0.1-beta
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git tag -d v0.1-beta
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-16a7f155-0bf3-9842-9fa6-d760dbcc2040","request_id":"16a7f155-0bf3-9842-9fa6-d760dbcc2040"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3dd9571b-d3f8-9e99-86b1-9b6c1145850e","request_id":"3dd9571b-d3f8-9e99-86b1-9b6c1145850e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5bc34707-6817-9700-8fd8-c79c609cd2cc","request_id":"5bc34707-6817-9700-8fd8-c79c609cd2cc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-600c088d-4efb-94e1-be08-1026d8583b56","request_id":"600c088d-4efb-94e1-be08-1026d8583b56"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-dbd4a90a-14ab-9fe4-97a5-3c29137ebc18","request_id":"dbd4a90a-14ab-9fe4-97a5-3c29137ebc18"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-a134d43b-d679-9259-ab25-18283796364b"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ebe14ba1-75cb-9f0e-bb62-b699cb592df7"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ef6b8cb1-5333-9050-992d-4896afb9c594"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-8c9e409d-7a59-9567-83dd-c690707f13bf"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-294f7802-00cc-9ca5-a58c-1368ab827faf"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-75fd0565-0016-9217-b7ce-9091dd3a1a55"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-004a76bb-e33e-9ec3-9703-8f16cb06fca5","request_id":"004a76bb-e33e-9ec3-9703-8f16cb06fca5"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-61c60614-de01-917d-95a5-30600ae146e9","request_id":"61c60614-de01-917d-95a5-30600ae146e9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f052cbc2-cef3-9fe2-be91-bd5ab7c32ee0","request_id":"f052cbc2-cef3-9fe2-be91-bd5ab7c32ee0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e5d29025-1a6d-9f7f-949e-f423f2ca40ee","request_id":"e5d29025-1a6d-9f7f-949e-f423f2ca40ee"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-af866b9a-5c00-9caa-bac3-68382ece2bdd","request_id":"af866b9a-5c00-9caa-bac3-68382ece2bdd"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b86aee73-c23f-9265-a6ce-61752ad129dc","request_id":"b86aee73-c23f-9265-a6ce-61752ad129dc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-38c42ab8-77d2-918b-ab5c-be03d485e738","request_id":"38c42ab8-77d2-918b-ab5c-be03d485e738"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1d46b785-6b8e-9df7-960b-5cac4a44ac65","request_id":"1d46b785-6b8e-9df7-960b-5cac4a44ac65"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git tag -d v0.1-beta{ "command": "git tag -d v0.1-beta" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v0.1-beta
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v0.1-beta
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -d v0.1-beta
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.