List tags matching a pattern
Tests ability to list tags matching a glob pattern. Evaluates tag-filtering with pattern matching.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'v1' > app.txt
  5. 05 git add app.txt
  6. 06 git commit -m 'v1'
  7. 07 git tag v1.0
  8. 08 echo 'v2' > app.txt
  9. 09 git add app.txt
  10. 10 git commit -m 'v2'
  11. 11 git tag v2.0
  12. 12 echo 'beta' > app.txt
  13. 13 git add app.txt
  14. 14 git commit -m 'beta'
  15. 15 git tag beta-1
  16. 16 echo 'rc' > app.txt
  17. 17 git add app.txt
  18. 18 git commit -m 'rc'
  19. 19 git tag rc-1
Prompt
List only git tags matching the pattern 'v*'. Output ONLY the git command, nothing else.
Expected
git tag -l 'v*'

Scoped model quality, cost, API time, and token usage for tag_management/f005.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 101 in → 15 out (0 reasoning)
``` git tag -l "v*" ```
anthropic/claude-fable-5:high__json_schema PASS 100% 311 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-fable-5:low PASS 100% 104 in → 15 out (0 reasoning)
``` git tag -l "v*" ```
anthropic/claude-fable-5:low__json_schema PASS 100% 308 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-fable-5:max PASS 100% 103 in → 82 out (36 reasoning)
git tag -l 'v*'
anthropic/claude-fable-5:max__json_schema PASS 100% 311 in → 99 out (32 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-fable-5:medium PASS 100% 102 in → 15 out (0 reasoning)
``` git tag -l "v*" ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 309 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-fable-5:xhigh PASS 100% 103 in → 27 out (22 reasoning)
git tag -l 'v*'
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 311 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-haiku-4.5:high PASS 100% 112 in → 156 out (110 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 277 in → 415 out (345 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-haiku-4.5:low PASS 100% 111 in → 137 out (109 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 272 in → 210 out (162 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-haiku-4.5:medium PASS 100% 113 in → 185 out (138 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 276 in → 325 out (268 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-haiku-4.5:none PASS 100% 84 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 244 in → 17 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 112 in → 202 out (157 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 275 in → 185 out (142 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.6:high PASS 100% 85 in → 17 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 244 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.6:low PASS 100% 80 in → 12 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-opus-4.6:low__json_schema PASS 100% 249 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.6:max PASS 100% 84 in → 17 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 247 in → 47 out (15 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.6:medium PASS 100% 79 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 248 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.6:none PASS 100% 84 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 243 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.6:xhigh PASS 100% 80 in → 17 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 245 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.7:high PASS 100% 111 in → 14 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-opus-4.7:high__json_schema PASS 100% 314 in → 21 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.7:low PASS 100% 106 in → 14 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-opus-4.7:low__json_schema PASS 100% 315 in → 22 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.7:max PASS 100% 107 in → 19 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 316 in → 21 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.7:medium PASS 100% 103 in → 14 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 318 in → 21 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.7:none PASS 100% 102 in → 14 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-opus-4.7:none__json_schema PASS 100% 316 in → 22 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.7:xhigh PASS 100% 109 in → 14 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 315 in → 22 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.8:high PASS 100% 100 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 314 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.8:low PASS 100% 101 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 312 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-opus-4.8:max PASS 100% 102 in → 159 out (31 reasoning)
`git tag -l 'v*'`
anthropic/claude-opus-4.8:max__json_schema PASS 100% 310 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-opus-4.8:medium PASS 100% 98 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 309 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.8:none PASS 100% 102 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 309 in → 18 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-opus-4.8:xhigh PASS 100% 101 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 308 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-4.6:high PASS 100% 85 in → 50 out (20 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 246 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-4.6:low PASS 100% 85 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 245 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-4.6:max PASS 100% 84 in → 72 out (39 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 247 in → 49 out (19 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-4.6:medium PASS 100% 84 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 246 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-4.6:none PASS 100% 86 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 246 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 83 in → 50 out (20 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 244 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-5:high PASS 100% 99 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 312 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-5:low PASS 100% 102 in → 12 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-sonnet-5:low__json_schema PASS 100% 316 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-5:medium PASS 100% 98 in → 12 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 312 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
anthropic/claude-sonnet-5:none PASS 100% 103 in → 12 out (0 reasoning)
git tag -l 'v*'
anthropic/claude-sonnet-5:none__json_schema PASS 100% 312 in → 18 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
anthropic/claude-sonnet-5:xhigh PASS 100% 101 in → 16 out (0 reasoning)
``` git tag -l 'v*' ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 313 in → 19 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
arcee-ai/trinity-large-thinking:high PASS 100% 79 in → 217 out (205 reasoning)
`git tag -l 'v*'`
arcee-ai/trinity-large-thinking:low PASS 100% 82 in → 323 out (314 reasoning)
git tag -l 'v*'
arcee-ai/trinity-large-thinking:medium PASS 100% 82 in → 217 out (207 reasoning)
git tag -l 'v*'
arcee-ai/trinity-large-thinking:xhigh PASS 100% 80 in → 464 out (455 reasoning)
git tag -l 'v*'
arcee-ai/trinity-mini:high PASS 100% 80 in → 281 out (270 reasoning)
git tag -l 'v*'
arcee-ai/trinity-mini:high__json_schema PASS 100% 78 in → 346 out (338 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
arcee-ai/trinity-mini:low PASS 100% 80 in → 294 out (288 reasoning)
git tag -l 'v*'
arcee-ai/trinity-mini:low__json_schema PASS 100% 77 in → 347 out (321 reasoning)
git tag-l v*
JSON Schema Structured Output
(raw) { "command": "git tag-l v*" }
arcee-ai/trinity-mini:medium PASS 100% 78 in → 320 out (311 reasoning)
git tag -l 'v*'
arcee-ai/trinity-mini:medium__json_schema PASS 100% 80 in → 296 out (286 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 81 in → 344 out (319 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) { "command": "git tag -l \"v*\"" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 155 in → 70 out (56 reasoning)
git tag --list 'v*'
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 71 in → 210 out (180 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
deepseek/deepseek-v4-flash-0731:low PASS 100% 73 in → 243 out (202 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 71 in → 73 out (60 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 153 in → 43 out (34 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 152 in → 51 out (38 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
deepseek/deepseek-v4-flash-0731:medium PASS 100% 76 in → 40 out (33 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 82 in → 102 out (80 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 72 in → 9 out (0 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 74 in → 17 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 71 in → 38 out (32 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 77 in → 151 out (117 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
deepseek/deepseek-v4-flash:high PASS 100% 74 in → 81 out (61 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 73 in → 111 out (95 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
deepseek/deepseek-v4-flash:low PASS 100% 72 in → 141 out (132 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 73 in → 94 out (72 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
deepseek/deepseek-v4-flash:medium PASS 100% 74 in → 131 out (121 reasoning)
git tag --list 'v*'
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 74 in → 115 out (95 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
deepseek/deepseek-v4-flash:none PASS 100% 75 in → 9 out (0 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 76 in → 16 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 152 in → 241 out (224 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 152 in → 269 out (251 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
deepseek/deepseek-v4-pro:high PASS 100% 72 in → 68 out (59 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 75 in → 243 out (219 reasoning)
git tag -l 'v*' --sort=version:refname
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*' --sort=version:refname" }
deepseek/deepseek-v4-pro:low PASS 100% 74 in → 139 out (129 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 75 in → 215 out (200 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
deepseek/deepseek-v4-pro:medium PASS 100% 76 in → 9 out (0 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 260 in → 47 out (32 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
deepseek/deepseek-v4-pro:none PASS 100% 73 in → 8 out (0 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-pro:xhigh PASS 100% 157 in → 334 out (324 reasoning)
git tag -l 'v*'
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 337 in → 386 out (371 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3-flash-preview:high PASS 100% 77 in → 247 out (240 reasoning)
git tag -l 'v*'
google/gemini-3-flash-preview:high__json_schema PASS 100% 138 in → 166 out (154 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3-flash-preview:low PASS 100% 80 in → 144 out (137 reasoning)
git tag -l 'v*'
google/gemini-3-flash-preview:low__json_schema PASS 100% 132 in → 237 out (226 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
google/gemini-3-flash-preview:medium PASS 100% 80 in → 151 out (144 reasoning)
git tag -l 'v*'
google/gemini-3-flash-preview:medium__json_schema PASS 100% 136 in → 198 out (186 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3-flash-preview:xhigh PASS 100% 81 in → 173 out (166 reasoning)
git tag -l 'v*'
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 139 in → 285 out (273 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 81 in → 370 out (363 reasoning)
git tag -l 'v*'
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 136 in → 625 out (614 reasoning)
git tag -l v*
JSON Schema Structured Output
(raw) {"command": "git tag -l v*"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 79 in → 125 out (118 reasoning)
git tag -l "v*"
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 134 in → 112 out (100 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 79 in → 117 out (110 reasoning)
git tag -l "v*"
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 79 in → 178 out (166 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 82 in → 338 out (331 reasoning)
git tag -l 'v*'
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 82 in → 322 out (310 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.1-pro-preview:high PASS 100% 78 in → 338 out (331 reasoning)
git tag -l 'v*'
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 138 in → 232 out (220 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.1-pro-preview:low PASS 100% 78 in → 247 out (240 reasoning)
git tag -l 'v*'
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 131 in → 436 out (425 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
google/gemini-3.1-pro-preview:medium PASS 100% 81 in → 309 out (302 reasoning)
git tag -l 'v*'
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 138 in → 72 out (60 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 82 in → 377 out (370 reasoning)
git tag -l "v*"
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 137 in → 177 out (165 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.5-flash-lite:high PASS 100% 79 in → 269 out (262 reasoning)
git tag -l "v*"
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 83 in → 231 out (219 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.5-flash-lite:low PASS 100% 81 in → 9 out (0 reasoning)
`git tag -l "v*"`
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 77 in → 13 out (0 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command": "git tag -l \"v*\""}
google/gemini-3.5-flash-lite:max PASS 100% 75 in → 381 out (374 reasoning)
git tag -l "v*"
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 83 in → 322 out (310 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.5-flash-lite:medium PASS 100% 82 in → 161 out (154 reasoning)
git tag -l "v*"
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 138 in → 337 out (325 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.5-flash-lite:xhigh PASS 100% 79 in → 335 out (328 reasoning)
git tag -l "v*"
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 80 in → 379 out (367 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.5-flash:high PASS 100% 80 in → 416 out (409 reasoning)
git tag -l 'v*'
google/gemini-3.5-flash:high__json_schema PASS 100% 136 in → 169 out (157 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.5-flash:low PASS 100% 79 in → 208 out (201 reasoning)
git tag -l 'v*'
google/gemini-3.5-flash:low__json_schema PASS 100% 137 in → 64 out (52 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.5-flash:medium PASS 100% 79 in → 331 out (324 reasoning)
git tag -l 'v*'
google/gemini-3.5-flash:medium__json_schema PASS 100% 137 in → 290 out (279 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
google/gemini-3.5-flash:xhigh PASS 100% 80 in → 294 out (287 reasoning)
git tag -l 'v*'
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 136 in → 351 out (339 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemini-3.6-flash:high PASS 100% 80 in → 238 out (231 reasoning)
git tag -l 'v*'
google/gemini-3.6-flash:high__json_schema PASS 100% 80 in → 278 out (266 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemini-3.6-flash:low PASS 100% 81 in → 228 out (221 reasoning)
git tag -l 'v*'
google/gemini-3.6-flash:low__json_schema PASS 100% 134 in → 11 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
google/gemini-3.6-flash:max PASS 100% 78 in → 256 out (249 reasoning)
git tag -l 'v*'
google/gemini-3.6-flash:max__json_schema PASS 100% 137 in → 330 out (319 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
google/gemini-3.6-flash:medium PASS 100% 81 in → 294 out (287 reasoning)
git tag -l "v*"
google/gemini-3.6-flash:medium__json_schema PASS 100% 134 in → 135 out (118 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemini-3.6-flash:xhigh PASS 100% 82 in → 201 out (194 reasoning)
git tag -l 'v*'
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 136 in → 214 out (202 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command":"git tag -l \"v*\""}
google/gemma-4-26b-a4b-it:high PASS 100% 95 in → 232 out (224 reasoning)
git tag -l 'v*'
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 98 in → 294 out (209 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemma-4-26b-a4b-it:low PASS 100% 95 in → 390 out (289 reasoning)
git tag -l 'v*'
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 94 in → 560 out (482 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemma-4-26b-a4b-it:medium PASS 100% 95 in → 387 out (322 reasoning)
git tag -l 'v*'
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 96 in → 238 out (188 reasoning)
git tag 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag 'v*'"}
google/gemma-4-26b-a4b-it:none PASS 100% 92 in → 8 out (0 reasoning)
git tag -l 'v*'
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 93 in → 13 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemma-4-26b-a4b-it:xhigh PASS 100% 96 in → 207 out (144 reasoning)
git tag -l 'v*'
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 96 in → 714 out (546 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemma-4-31b-it:high PASS 100% 95 in → 164 out (133 reasoning)
git tag -l 'v*'
google/gemma-4-31b-it:high__json_schema PASS 100% 98 in → 180 out (138 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemma-4-31b-it:low PASS 100% 96 in → 208 out (186 reasoning)
git tag -l 'v*'
google/gemma-4-31b-it:low__json_schema PASS 100% 94 in → 181 out (134 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemma-4-31b-it:medium PASS 100% 94 in → 122 out (97 reasoning)
git tag -l 'v*'
google/gemma-4-31b-it:medium__json_schema PASS 100% 93 in → 162 out (125 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
google/gemma-4-31b-it:none PASS 100% 92 in → 8 out (0 reasoning)
git tag -l "v*"
google/gemma-4-31b-it:none__json_schema PASS 100% 92 in → 13 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
google/gemma-4-31b-it:xhigh PASS 100% 98 in → 170 out (140 reasoning)
git tag -l 'v*'
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 95 in → 161 out (132 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
ibm-granite/granite-4.1-8b PASS 100% 87 in → 8 out
git tag --list 'v*'
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
inclusionai/ling-2.6-flash PASS 100% 101 in → 14 out
```sh git tag -l 'v*' ```
inclusionai/ling-2.6-flash__json_schema PASS 100% 103 in → 15 out
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.5:high PASS 100% 110 in → 191 out (185 reasoning)
git tag -l 'v*'
minimax/minimax-m2.5:high__json_schema PASS 100% 107 in → 342 out (327 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.5:low PASS 100% 108 in → 241 out (231 reasoning)
git tag -l 'v*'
minimax/minimax-m2.5:low__json_schema PASS 100% 108 in → 308 out (262 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.5:medium PASS 100% 107 in → 215 out (184 reasoning)
git tag -l 'v*'
minimax/minimax-m2.5:medium__json_schema PASS 100% 104 in → 191 out (161 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.5:xhigh PASS 100% 110 in → 183 out (171 reasoning)
git tag -l 'v*'
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 111 in → 182 out (140 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.7:high PASS 100% 109 in → 180 out (170 reasoning)
git tag -l 'v*'
minimax/minimax-m2.7:high__json_schema PASS 100% 240 in → 408 out (391 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.7:low PASS 100% 110 in → 151 out (142 reasoning)
git tag -l 'v*'
minimax/minimax-m2.7:medium PASS 100% 107 in → 292 out (282 reasoning)
git tag -l "v*"
minimax/minimax-m2.7:medium__json_schema PASS 100% 240 in → 103 out (82 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
minimax/minimax-m2.7:xhigh PASS 100% 107 in → 204 out (182 reasoning)
git tag -l 'v*'
minimax/minimax-m3:high PASS 100% 243 in → 80 out (70 reasoning)
git tag -l 'v*'
minimax/minimax-m3:high__json_schema PASS 100% 233 in → 14 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag --list 'v*'"}
minimax/minimax-m3:low PASS 100% 247 in → 34 out (27 reasoning)
git tag -l 'v*'
minimax/minimax-m3:medium PASS 100% 240 in → 48 out (38 reasoning)
git tag -l 'v*'
minimax/minimax-m3:medium__json_schema PASS 100% 233 in → 14 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
minimax/minimax-m3:xhigh PASS 100% 245 in → 79 out (71 reasoning)
git tag -l "v*"
minimax/minimax-m3:xhigh__json_schema PASS 100% 231 in → 14 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
mistralai/devstral-2512 PASS 100% 82 in → 13 out
```bash git tag -l 'v*' ```
mistralai/devstral-2512__json_schema PASS 100% 77 in → 13 out
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
mistralai/mistral-medium-3-5:high PASS 100% 93 in → 489 out (485 reasoning)
git tag -l 'v*'
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 80 in → 310 out (270 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
mistralai/mistral-medium-3-5:low PASS 100% 95 in → 554 out (530 reasoning)
git tag -l 'v*'
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 81 in → 322 out (290 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
mistralai/mistral-medium-3-5:medium PASS 100% 93 in → 199 out (162 reasoning)
git tag -l 'v*'
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 80 in → 83 out (73 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag --list 'v*'"}
mistralai/mistral-medium-3-5:none PASS 100% 91 in → 8 out (0 reasoning)
git tag -l 'v*'
mistralai/mistral-medium-3-5:xhigh PASS 100% 93 in → 438 out (429 reasoning)
git tag -l 'v*'
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 83 in → 195 out (175 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
moonshotai/kimi-k2.6:high PASS 100% 77 in → 494 out (494 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.6:high__json_schema PASS 100% 73 in → 1,041 out (478 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command" : "git tag -l 'v*'" }
moonshotai/kimi-k2.6:low PASS 100% 76 in → 588 out (582 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.6:medium PASS 100% 77 in → 432 out (423 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 78 in → 469 out (487 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
moonshotai/kimi-k2.6:xhigh PASS 100% 77 in → 509 out (491 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 77 in → 87 out (73 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
moonshotai/kimi-k2.7-code:high PASS 100% 75 in → 89 out (86 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 158 in → 142 out (127 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
moonshotai/kimi-k2.7-code:low PASS 100% 78 in → 169 out (170 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.7-code:medium PASS 100% 79 in → 166 out (156 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.7-code:xhigh PASS 100% 76 in → 104 out (94 reasoning)
git tag -l 'v*'
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 76 in → 172 out (157 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
moonshotai/kimi-k3:high PASS 100% 151 in → 67 out (44 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:high__json_schema PASS 100% 151 in → 75 out (52 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:low PASS 100% 157 in → 74 out (51 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:low__json_schema PASS 100% 154 in → 82 out (59 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:max PASS 100% 156 in → 68 out (45 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:max__json_schema PASS 100% 155 in → 66 out (43 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:medium PASS 100% 154 in → 76 out (53 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:medium__json_schema PASS 100% 152 in → 144 out (121 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:xhigh PASS 100% 153 in → 88 out (65 reasoning)
git tag -l 'v*'
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 253 in → 77 out (50 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 92 in → 264 out (219 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 94 in → 129 out (108 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 92 in → 143 out (124 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 93 in → 191 out (162 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 92 in → 128 out (116 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 93 in → 198 out (176 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 95 in → 8 out (0 reasoning)
git tag --list 'v*'
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 95 in → 27 out (0 reasoning)
git tag -l 'v*' --sort=-creatordate
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*' --sort=-creatordate" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 91 in → 353 out (312 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 94 in → 231 out (176 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 91 in → 107 out (95 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 93 in → 89 out (71 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 93 in → 87 out (76 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 93 in → 144 out (128 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 93 in → 65 out (62 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 95 in → 134 out (117 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 95 in → 8 out (0 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 95 in → 16 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 92 in → 128 out (125 reasoning)
git tag -l 'v*'
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 94 in → 104 out (87 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
openai/gpt-5.4-mini:high PASS 100% 75 in → 113 out (99 reasoning)
git tag --list 'v*'
openai/gpt-5.4-mini:high__json_schema PASS 100% 110 in → 102 out (80 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.4-mini:low PASS 100% 81 in → 41 out (27 reasoning)
git tag -l 'v*'
openai/gpt-5.4-mini:low__json_schema PASS 100% 108 in → 60 out (38 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4-mini:medium PASS 100% 74 in → 48 out (34 reasoning)
git tag -l 'v*'
openai/gpt-5.4-mini:medium__json_schema PASS 100% 110 in → 73 out (51 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.4-mini:none PASS 100% 74 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.4-mini:none__json_schema PASS 100% 110 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.4-mini:xhigh PASS 100% 73 in → 669 out (655 reasoning)
git tag -l 'v*'
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 109 in → 351 out (329 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4-nano:high PASS 100% 76 in → 42 out (28 reasoning)
git tag -l 'v*'
openai/gpt-5.4-nano:high__json_schema PASS 100% 108 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.4-nano:low PASS 100% 73 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.4-nano:low__json_schema PASS 100% 110 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4-nano:medium PASS 100% 75 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.4-nano:medium__json_schema PASS 100% 107 in → 47 out (25 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.4-nano:none PASS 100% 78 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.4-nano:none__json_schema PASS 100% 107 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.4-nano:xhigh PASS 100% 79 in → 136 out (122 reasoning)
git tag -l 'v*'
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 112 in → 210 out (188 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4:high PASS 100% 76 in → 85 out (71 reasoning)
git tag -l 'v*'
openai/gpt-5.4:high__json_schema PASS 100% 113 in → 282 out (260 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4:low PASS 100% 73 in → 43 out (29 reasoning)
git tag -l 'v*'
openai/gpt-5.4:low__json_schema PASS 100% 108 in → 48 out (26 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4:medium PASS 100% 73 in → 50 out (36 reasoning)
git tag -l 'v*'
openai/gpt-5.4:medium__json_schema PASS 100% 110 in → 82 out (60 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4:none PASS 100% 75 in → 12 out (0 reasoning)
git tag --list 'v*'
openai/gpt-5.4:none__json_schema PASS 100% 108 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.4:xhigh PASS 100% 73 in → 160 out (146 reasoning)
git tag -l 'v*'
openai/gpt-5.4:xhigh__json_schema PASS 100% 108 in → 325 out (303 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.5:high PASS 100% 75 in → 70 out (56 reasoning)
git tag -l 'v*'
openai/gpt-5.5:high__json_schema PASS 100% 113 in → 88 out (66 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.5:low PASS 100% 74 in → 46 out (32 reasoning)
git tag --list 'v*'
openai/gpt-5.5:low__json_schema PASS 100% 109 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.5:medium PASS 100% 74 in → 55 out (41 reasoning)
git tag --list 'v*'
openai/gpt-5.5:medium__json_schema PASS 100% 113 in → 48 out (26 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.5:none PASS 100% 75 in → 12 out (0 reasoning)
git tag --list 'v*'
openai/gpt-5.5:none__json_schema PASS 100% 107 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.5:xhigh PASS 100% 76 in → 105 out (91 reasoning)
git tag --list 'v*'
openai/gpt-5.5:xhigh__json_schema PASS 100% 110 in → 124 out (102 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-luna:high PASS 100% 75 in → 48 out (34 reasoning)
git tag --list 'v*'
openai/gpt-5.6-luna:high__json_schema PASS 100% 110 in → 56 out (34 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-luna:low PASS 100% 72 in → 39 out (25 reasoning)
git tag --list 'v*'
openai/gpt-5.6-luna:low__json_schema PASS 100% 110 in → 50 out (28 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-luna:max PASS 100% 76 in → 56 out (42 reasoning)
git tag -l 'v*'
openai/gpt-5.6-luna:max__json_schema PASS 100% 110 in → 264 out (242 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-luna:medium PASS 100% 79 in → 42 out (28 reasoning)
git tag -l 'v*'
openai/gpt-5.6-luna:medium__json_schema PASS 100% 111 in → 60 out (38 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-luna:none PASS 100% 74 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.6-luna:none__json_schema PASS 100% 110 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-luna:xhigh PASS 100% 75 in → 48 out (34 reasoning)
git tag --list 'v*'
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 109 in → 111 out (89 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-sol:high PASS 100% 74 in → 12 out (0 reasoning)
git tag --list 'v*'
openai/gpt-5.6-sol:high__json_schema PASS 100% 111 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-sol:low PASS 100% 77 in → 12 out (0 reasoning)
git tag --list 'v*'
openai/gpt-5.6-sol:low__json_schema PASS 100% 114 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-sol:max PASS 100% 74 in → 52 out (38 reasoning)
git tag --list 'v*'
openai/gpt-5.6-sol:max__json_schema PASS 100% 110 in → 150 out (128 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-sol:medium PASS 100% 72 in → 12 out (0 reasoning)
git tag --list 'v*'
openai/gpt-5.6-sol:medium__json_schema PASS 100% 111 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-sol:none PASS 100% 77 in → 12 out (0 reasoning)
git tag --list 'v*'
openai/gpt-5.6-sol:none__json_schema PASS 100% 109 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-sol:xhigh PASS 100% 76 in → 41 out (27 reasoning)
git tag --list 'v*'
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 113 in → 20 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag --list 'v*'"}
openai/gpt-5.6-terra:high PASS 100% 75 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.6-terra:high__json_schema PASS 100% 109 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-terra:low PASS 100% 75 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.6-terra:low__json_schema PASS 100% 114 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-terra:max PASS 100% 72 in → 52 out (38 reasoning)
git tag -l 'v*'
openai/gpt-5.6-terra:max__json_schema PASS 100% 110 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-terra:medium PASS 100% 77 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.6-terra:medium__json_schema PASS 100% 109 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-terra:none PASS 100% 75 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.6-terra:none__json_schema PASS 100% 112 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-5.6-terra:xhigh PASS 100% 74 in → 12 out (0 reasoning)
git tag -l 'v*'
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 108 in → 20 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-oss-120b:high PASS 100% 135 in → 358 out (339 reasoning)
git tag -l "v*"
openai/gpt-oss-120b:high__json_schema PASS 100% 138 in → 286 out (264 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-oss-120b:low PASS 100% 136 in → 87 out (76 reasoning)
git tag -l 'v*'
openai/gpt-oss-120b:low__json_schema PASS 100% 136 in → 56 out (33 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-oss-120b:medium PASS 100% 138 in → 88 out (75 reasoning)
git tag -l 'v*'
openai/gpt-oss-120b:medium__json_schema PASS 100% 140 in → 15 out (0 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
openai/gpt-oss-120b:xhigh PASS 100% 135 in → 389 out (370 reasoning)
git tag -l 'v*'
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 139 in → 329 out (304 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-oss-20b:high PASS 100% 138 in → 429 out (435 reasoning)
git tag -l 'v*'
openai/gpt-oss-20b:high__json_schema PASS 100% 137 in → 304 out (264 reasoning)
git tag --list "v*"
JSON Schema Structured Output
(raw) { "command": "git tag --list \"v*\"" }
openai/gpt-oss-20b:low PASS 100% 121 in → 49 out (29 reasoning)
git tag -l 'v*'
openai/gpt-oss-20b:low__json_schema PASS 100% 135 in → 47 out (22 reasoning)
git tag --list 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag --list 'v*'" }
openai/gpt-oss-20b:medium PASS 100% 136 in → 88 out (81 reasoning)
git tag -l "v*"
openai/gpt-oss-20b:medium__json_schema PASS 100% 138 in → 130 out (106 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
openai/gpt-oss-20b:xhigh PASS 100% 141 in → 166 out (146 reasoning)
git tag -l 'v*'
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 136 in → 349 out (324 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
poolside/laguna-m.1:high PASS 100% 92 in → 389 out (379 reasoning)
git tag -l "v*"
poolside/laguna-m.1:high__json_schema PASS 100% 93 in → 314 out (296 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
poolside/laguna-m.1:low PASS 100% 95 in → 314 out (305 reasoning)
git tag -l v*
poolside/laguna-m.1:low__json_schema PASS 100% 94 in → 134 out (120 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
poolside/laguna-m.1:medium PASS 100% 93 in → 283 out (275 reasoning)
git tag -l v*
poolside/laguna-m.1:medium__json_schema PASS 100% 93 in → 324 out (311 reasoning)
git tag -l v*
JSON Schema Structured Output
(raw) {"command": "git tag -l v*"}
poolside/laguna-m.1:none PASS 100% 92 in → 9 out (0 reasoning)
git tag -l 'v*'
poolside/laguna-m.1:xhigh PASS 100% 93 in → 243 out (236 reasoning)
git tag v*
poolside/laguna-m.1:xhigh__json_schema PASS 100% 89 in → 262 out (243 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
poolside/laguna-xs-2.1:high PASS 100% 92 in → 352 out (344 reasoning)
git tag -l 'v*'
poolside/laguna-xs-2.1:high__json_schema PASS 100% 95 in → 448 out (433 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command": "git tag -l \"v*\""}
poolside/laguna-xs-2.1:low PASS 100% 91 in → 187 out (179 reasoning)
git tag -l 'v*'
poolside/laguna-xs-2.1:low__json_schema PASS 100% 93 in → 340 out (325 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command": "git tag -l \"v*\""}
poolside/laguna-xs-2.1:medium PASS 100% 92 in → 229 out (221 reasoning)
git tag -l 'v*'
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 89 in → 257 out (238 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
poolside/laguna-xs-2.1:none PASS 100% 93 in → 367 out (0 reasoning)
git tag 'v*' ``` v1.0 v2.0 ``` Okay, the user wants me to list only the git tags that match the pattern 'v*'. Let me look at the given tags: beta-1, rc-1, v1.0, v2.0. The pattern 'v*' means any tag starting with 'v' followed by any characters. So, beta-1 and rc-1 don't start with 'v', so they're out. The valid ones are v1.0 and v2.0. The command to list them would be `git tag 'v*'`. I need to make sure the output only includes that command and nothing else. Let me double-check the example response to confirm. Yep, it shows the command and then the matching tags. But the user said to output ONLY the git command. Wait, the example response includes the tags after the command. Wait, the user's instruction says "Output ONLY the git command, nothing else." But in the example, they have the command followed by the tags. Hmm, maybe there's a misunderstanding. Let me read again. The user provided tags and a log, but the actual task is to generate the git command that lists tags matching 'v*'. The example shows the command followed by the tags, but the user might want just the command. Wait, the initial instruction says "List only git tags matching the pattern 'v*'. Output ONLY the git command, nothing else." So the correct answer is just the command `git tag 'v*'`. The example in the problem statement includes the tags after, but the user wants only the command. So I should output only the command. ``` git tag 'v*' ```</think>git tag 'v*'
poolside/laguna-xs-2.1:none__json_schema PASS 100% 93 in → 18 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
poolside/laguna-xs-2.1:xhigh PASS 100% 92 in → 252 out (244 reasoning)
git tag -l 'v*'
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 94 in → 258 out (244 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
poolside/laguna-xs.2:high PASS 100% 131 in → 247 out (237 reasoning)
git tag -l 'v*'
poolside/laguna-xs.2:high__json_schema PASS 100% 129 in → 165 out (150 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command": "git tag -l \"v*\""}
poolside/laguna-xs.2:low PASS 100% 129 in → 292 out (282 reasoning)
git tag -l "v*"
poolside/laguna-xs.2:low__json_schema PASS 100% 132 in → 295 out (283 reasoning)
git tag -l v*
JSON Schema Structured Output
(raw) {"command": "git tag -l v*"}
poolside/laguna-xs.2:medium PASS 100% 130 in → 185 out (175 reasoning)
git tag -l "v*"
poolside/laguna-xs.2:medium__json_schema PASS 100% 132 in → 208 out (193 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command": "git tag -l \"v*\""}
poolside/laguna-xs.2:none PASS 100% 130 in → 9 out (0 reasoning)
git tag -l 'v*'
poolside/laguna-xs.2:none__json_schema PASS 100% 126 in → 13 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
poolside/laguna-xs.2:xhigh PASS 100% 127 in → 214 out (204 reasoning)
git tag -l "v*"
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 130 in → 235 out (222 reasoning)
git tag -l v*
JSON Schema Structured Output
(raw) {"command": "git tag -l v*"}
qwen/qwen3.6-27b:high PASS 100% 87 in → 1,136 out (967 reasoning)
git tag -l 'v*'
qwen/qwen3.6-27b:high__json_schema PASS 100% 85 in → 889 out (1 reasoning)
git tag 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag 'v*'" }
qwen/qwen3.6-27b:low PASS 100% 88 in → 1,531 out (1,370 reasoning)
git tag --list 'v*'
qwen/qwen3.6-27b:medium PASS 100% 86 in → 1,113 out (980 reasoning)
git tag -l 'v*'
qwen/qwen3.6-27b:medium__json_schema PASS 100% 91 in → 1,511 out (1 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
qwen/qwen3.6-27b:none PASS 100% 91 in → 8 out (0 reasoning)
git tag -l 'v*'
qwen/qwen3.6-27b:xhigh PASS 100% 88 in → 984 out (889 reasoning)
git tag -l 'v*'
qwen/qwen3.6-35b-a3b:high PASS 100% 86 in → 751 out (739 reasoning)
git tag -l 'v*'
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 91 in → 371 out (322 reasoning)
git tag v*
JSON Schema Structured Output
(raw) { "command": "git tag v*" }
qwen/qwen3.6-35b-a3b:low PASS 100% 90 in → 848 out (765 reasoning)
git tag 'v*'
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 89 in → 1,325 out (1,155 reasoning)
git tag 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag 'v*'" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 89 in → 1,399 out (1,387 reasoning)
git tag --list 'v*'
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 86 in → 1,279 out (1,102 reasoning)
git tag 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag 'v*'" }
qwen/qwen3.6-35b-a3b:none PASS 100% 92 in → 7 out (0 reasoning)
git tag --list 'v*'
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 87 in → 967 out (958 reasoning)
git tag v*
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 91 in → 1,044 out (876 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
qwen/qwen3.6-flash:high PASS 100% 91 in → 942 out (932 reasoning)
git tag 'v*'
qwen/qwen3.6-flash:low PASS 100% 88 in → 1,180 out (1,168 reasoning)
git tag -l 'v*'
qwen/qwen3.6-flash:medium PASS 100% 88 in → 922 out (912 reasoning)
git tag 'v*'
qwen/qwen3.6-flash:none PASS 100% 91 in → 7 out (0 reasoning)
git tag -l 'v*'
qwen/qwen3.6-flash:xhigh PASS 100% 92 in → 744 out (734 reasoning)
git tag 'v*'
qwen/qwen3.7-flash:high PASS 100% 89 in → 1,109 out (1,100 reasoning)
git tag v*
qwen/qwen3.7-flash:low PASS 100% 90 in → 1,056 out (1,044 reasoning)
git tag --list 'v*'
qwen/qwen3.7-flash:low__json_schema PASS 100% 89 in → 869 out (860 reasoning)
git tag v*
qwen/qwen3.7-flash:max PASS 100% 87 in → 1,427 out (1,417 reasoning)
git tag 'v*'
qwen/qwen3.7-flash:medium PASS 100% 88 in → 706 out (697 reasoning)
git tag v*
qwen/qwen3.7-flash:none PASS 100% 92 in → 7 out (0 reasoning)
git tag -l 'v*'
qwen/qwen3.7-flash:xhigh PASS 100% 88 in → 1,415 out (1,403 reasoning)
git tag --list 'v*'
qwen/qwen3.7-max:high PASS 100% 83 in → 569 out (557 reasoning)
git tag -l 'v*'
qwen/qwen3.7-max:low PASS 100% 89 in → 643 out (631 reasoning)
git tag -l 'v*'
qwen/qwen3.7-max:medium PASS 100% 92 in → 584 out (572 reasoning)
git tag -l 'v*'
qwen/qwen3.7-max:none PASS 100% 90 in → 7 out (0 reasoning)
git tag -l 'v*'
qwen/qwen3.7-max:xhigh PASS 100% 90 in → 472 out (460 reasoning)
git tag -l 'v*'
qwen/qwen3.7-plus:high PASS 100% 87 in → 813 out (801 reasoning)
git tag -l 'v*'
qwen/qwen3.7-plus:low PASS 100% 89 in → 743 out (731 reasoning)
git tag -l "v*"
qwen/qwen3.7-plus:medium PASS 100% 90 in → 694 out (682 reasoning)
git tag -l "v*"
qwen/qwen3.7-plus:none PASS 100% 92 in → 7 out (0 reasoning)
git tag -l 'v*'
qwen/qwen3.7-plus:xhigh PASS 100% 91 in → 542 out (530 reasoning)
git tag -l 'v*'
tencent/hy3:high PASS 100% 79 in → 277 out (267 reasoning)
git tag -l 'v*'
tencent/hy3:high__json_schema PASS 100% 81 in → 948 out (864 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
tencent/hy3:low PASS 100% 83 in → 349 out (294 reasoning)
git tag -l 'v*'
tencent/hy3:low__json_schema PASS 100% 81 in → 329 out (270 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
tencent/hy3:medium PASS 100% 82 in → 199 out (189 reasoning)
git tag -l 'v*'
tencent/hy3:medium__json_schema PASS 100% 82 in → 238 out (199 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
tencent/hy3:none PASS 100% 82 in → 9 out (0 reasoning)
git tag -l 'v*'
tencent/hy3:none__json_schema PASS 100% 84 in → 15 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
tencent/hy3:xhigh PASS 100% 79 in → 198 out (188 reasoning)
git tag -l 'v*'
tencent/hy3:xhigh__json_schema PASS 100% 80 in → 508 out (442 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
thinkingmachines/inkling-small:high PASS 100% 87 in → 76 out (67 reasoning)
git tag -l 'v*'
thinkingmachines/inkling-small:high__json_schema PASS 100% 83 in → 168 out (149 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
thinkingmachines/inkling-small:low PASS 100% 86 in → 37 out (23 reasoning)
git tag -l 'v*'
thinkingmachines/inkling-small:low__json_schema PASS 100% 82 in → 29 out (12 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
thinkingmachines/inkling-small:max PASS 100% 85 in → 115 out (101 reasoning)
git tag -l 'v*'
thinkingmachines/inkling-small:max__json_schema PASS 100% 84 in → 166 out (146 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
thinkingmachines/inkling-small:medium PASS 100% 90 in → 26 out (12 reasoning)
git tag -l 'v*'
thinkingmachines/inkling-small:medium__json_schema PASS 100% 82 in → 59 out (41 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
thinkingmachines/inkling-small:none PASS 100% 85 in → 11 out (0 reasoning)
git tag -l 'v*'<|end_message|>
thinkingmachines/inkling-small:xhigh PASS 100% 85 in → 123 out (109 reasoning)
git tag -l 'v*'
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 86 in → 57 out (37 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
thinkingmachines/inkling:high PASS 100% 83 in → 138 out (124 reasoning)
git tag -l 'v*'
thinkingmachines/inkling:high__json_schema PASS 100% 85 in → 104 out (85 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
thinkingmachines/inkling:low PASS 100% 83 in → 49 out (34 reasoning)
git tag -l 'v*'
thinkingmachines/inkling:low__json_schema PASS 100% 84 in → 38 out (20 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
thinkingmachines/inkling:max PASS 100% 90 in → 175 out (125 reasoning)
git tag -l 'v*'
thinkingmachines/inkling:medium PASS 100% 88 in → 37 out (22 reasoning)
git tag -l 'v*'
thinkingmachines/inkling:none PASS 100% 80 in → 11 out (0 reasoning)
git tag -l 'v*'
thinkingmachines/inkling:xhigh PASS 100% 83 in → 140 out (102 reasoning)
git tag -l 'v*'
thinkingmachines/inkling:xhigh__json_schema PASS 100% 86 in → 103 out (83 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
x-ai/grok-4.3:high PASS 100% 253 in → 608 out (601 reasoning)
git tag --list 'v*'
x-ai/grok-4.3:high__json_schema PASS 100% 311 in → 413 out (402 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
x-ai/grok-4.3:low PASS 100% 258 in → 295 out (285 reasoning)
``` git tag -l 'v*' ```
x-ai/grok-4.3:low__json_schema PASS 100% 313 in → 299 out (284 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
x-ai/grok-4.3:max PASS 100% 253 in → 217 out (210 reasoning)
git tag -l 'v*'
x-ai/grok-4.3:max__json_schema PASS 100% 310 in → 429 out (414 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
x-ai/grok-4.3:medium PASS 100% 265 in → 135 out (128 reasoning)
git tag -l 'v*'
x-ai/grok-4.3:medium__json_schema PASS 100% 315 in → 642 out (627 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
x-ai/grok-4.3:none PASS 100% 254 in → 7 out (0 reasoning)
git tag -l 'v*'
x-ai/grok-4.3:none__json_schema PASS 100% 309 in → 11 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
x-ai/grok-4.3:xhigh PASS 100% 252 in → 275 out (268 reasoning)
git tag -l 'v*'
x-ai/grok-4.3:xhigh__json_schema PASS 100% 307 in → 659 out (644 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
x-ai/grok-4.5:high PASS 100% 276 in → 396 out (385 reasoning)
``` git tag -l 'v*' ```
x-ai/grok-4.5:high__json_schema PASS 100% 340 in → 387 out (375 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
x-ai/grok-4.5:low PASS 100% 273 in → 224 out (217 reasoning)
git tag -l 'v*'
x-ai/grok-4.5:low__json_schema PASS 100% 342 in → 228 out (216 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
x-ai/grok-4.5:max PASS 100% 274 in → 366 out (359 reasoning)
git tag -l 'v*'
x-ai/grok-4.5:max__json_schema PASS 100% 341 in → 241 out (229 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
x-ai/grok-4.5:medium PASS 100% 275 in → 175 out (168 reasoning)
git tag -l 'v*'
x-ai/grok-4.5:medium__json_schema PASS 100% 340 in → 304 out (292 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command":"git tag -l 'v*'"}
x-ai/grok-4.5:xhigh PASS 100% 277 in → 237 out (230 reasoning)
git tag -l 'v*'
x-ai/grok-4.5:xhigh__json_schema PASS 100% 341 in → 257 out (245 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
z-ai/glm-4.7-flash:high PASS 100% 76 in → 299 out (251 reasoning)
git tag -l "v*"
z-ai/glm-4.7-flash:low PASS 100% 77 in → 337 out (302 reasoning)
git tag -l "v*"
z-ai/glm-4.7-flash:low__json_schema PASS 100% 75 in → 265 out (214 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
z-ai/glm-4.7-flash:medium PASS 100% 78 in → 235 out (226 reasoning)
git tag -l "v*"
z-ai/glm-4.7-flash:none PASS 100% 75 in → 12 out (0 reasoning)
```bash git tag -l 'v*' ```
z-ai/glm-4.7-flash:none__json_schema PASS 100% 77 in → 15 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
z-ai/glm-4.7-flash:xhigh PASS 100% 76 in → 285 out (253 reasoning)
git tag v*
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 77 in → 449 out (398 reasoning)
git tag -l "v*"
JSON Schema Structured Output
(raw) {"command": "git tag -l \"v*\""}
z-ai/glm-5.2:high PASS 100% 85 in → 298 out (253 reasoning)
git tag -l 'v*'
z-ai/glm-5.2:high__json_schema PASS 100% 84 in → 219 out (203 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) { "command": "git tag -l 'v*'" }
z-ai/glm-5.2:low PASS 100% 85 in → 209 out (202 reasoning)
git tag -l 'v*'
z-ai/glm-5.2:low__json_schema PASS 100% 77 in → 13 out (0 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
z-ai/glm-5.2:medium PASS 100% 86 in → 278 out (262 reasoning)
git tag -l 'v*'
z-ai/glm-5.2:none PASS 100% 77 in → 8 out (0 reasoning)
git tag -l 'v*'
z-ai/glm-5.2:xhigh PASS 100% 83 in → 304 out (252 reasoning)
git tag -l 'v*'
z-ai/glm-5.2:xhigh__json_schema PASS 100% 83 in → 270 out (256 reasoning)
git tag -l 'v*'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v*'"}
arcee-ai/trinity-mini:xhigh FAIL 80% 79 in → 318 out (294 reasoning)
gittag -l "v*"
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git tag'}]
liquid/lfm-2-24b-a2b FAIL 80% 82 in → 7 out
v* v2.0
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git tag'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: v1.0 v2.0
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git tag -l 'v*'
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0cfb604b-adef-9a38-a393-56db85c070c2","request_id":"0cfb604b-adef-9a38-a393-56db85c070c2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-23bc0b57-9e4c-93cf-a163-c2fdb5243b13","request_id":"23bc0b57-9e4c-93cf-a163-c2fdb5243b13"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a675e6bd-e69c-900d-af02-8991c1e3e12b","request_id":"a675e6bd-e69c-900d-af02-8991c1e3e12b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-07aaec0f-6658-99b0-8990-1943c310e4fa","request_id":"07aaec0f-6658-99b0-8990-1943c310e4fa"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f058d7e4-df47-9e16-999f-8db46025faa1","request_id":"f058d7e4-df47-9e16-999f-8db46025faa1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ab184004-6560-9fe8-9d93-4301ea86c2ae"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-7bfa2dc7-cdb6-95a9-9d28-70fa4153ba0a"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-a72efe63-6fc3-9fed-9e7d-eb9ee4889268"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ddbad502-2a72-98a3-b3fe-f70fc8ac4192"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-b28e7641-58e3-9c15-9185-3c2e06703fe1"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d56fff2a-49ed-9174-8ab4-34d7d1e16408","request_id":"d56fff2a-49ed-9174-8ab4-34d7d1e16408"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-fa65334f-ac09-96dd-b5b4-098ed0e545ce","request_id":"fa65334f-ac09-96dd-b5b4-098ed0e545ce"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-e5d8e3bb-3f49-97a3-ad7a-6f85f97eff6a","request_id":"e5d8e3bb-3f49-97a3-ad7a-6f85f97eff6a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a51c510e-4219-9005-9e4e-3e0fec9d5ef9","request_id":"a51c510e-4219-9005-9e4e-3e0fec9d5ef9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4b47b1dc-2ee3-9473-bfc3-088e1763effd","request_id":"4b47b1dc-2ee3-9473-bfc3-088e1763effd"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-17b5c840-9b6d-9811-adee-b5bc8173a4f9","request_id":"17b5c840-9b6d-9811-adee-b5bc8173a4f9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3dbfee31-618b-9ca0-8871-42b023c5da41","request_id":"3dbfee31-618b-9ca0-8871-42b023c5da41"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-385e2e3b-316e-9b39-9c96-25e1f2132b2b","request_id":"385e2e3b-316e-9b39-9c96-25e1f2132b2b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: git tag -l 'v*'{"command": "git tag -l 'v*'"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -l 'v*'
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -l 'v*'
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -l 'v*'{"command": "git tag -l 'v*'"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -l 'v*'
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git tag -l 'v*'
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.