Show tag details
Tests ability to show detailed tag information. Evaluates tag-inspection with git show.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'App' > app.txt
  5. 05 git add app.txt
  6. 06 git commit -m 'Release candidate'
  7. 07 git tag -a v1.0.0-rc1 -m 'Release candidate 1'
Prompt
Show the details (message, tagger, date) of the tag 'v1.0.0-rc1'. Output ONLY the git command, nothing else.
Expected
git show v1.0.0-rc1

Scoped model quality, cost, API time, and token usage for tag_management/f007.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 83 in → 254 out (83 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-fable-5:high__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-fable-5:low PASS 100% 84 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-fable-5:low__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-fable-5:max PASS 100% 82 in → 321 out (110 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-fable-5:max__json_schema PASS 100% 292 in → 201 out (92 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-fable-5:medium PASS 100% 83 in → 139 out (95 reasoning)
git show v1.0.0-rc1
anthropic/claude-fable-5:medium__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-fable-5:xhigh PASS 100% 84 in → 178 out (66 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-haiku-4.5:high PASS 100% 96 in → 246 out (208 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 259 in → 458 out (356 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-haiku-4.5:low PASS 100% 95 in → 518 out (425 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 259 in → 282 out (215 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-haiku-4.5:medium PASS 100% 97 in → 613 out (464 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 262 in → 729 out (455 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-haiku-4.5:none PASS 100% 66 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 231 in → 19 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 95 in → 356 out (286 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 257 in → 538 out (426 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-opus-4.6:high PASS 100% 66 in → 19 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 230 in → 73 out (28 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.6:low PASS 100% 65 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-opus-4.6:low__json_schema PASS 100% 231 in → 18 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.6:max PASS 100% 68 in → 19 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 231 in → 58 out (26 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.6:medium PASS 100% 69 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 230 in → 18 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.6:none PASS 100% 66 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 231 in → 18 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.6:xhigh PASS 100% 67 in → 19 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 229 in → 54 out (22 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.7:high PASS 100% 90 in → 21 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.7:high__json_schema PASS 100% 298 in → 23 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.7:low PASS 100% 90 in → 18 out (0 reasoning)
`git show v1.0.0-rc1`
anthropic/claude-opus-4.7:low__json_schema PASS 100% 299 in → 23 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.7:max PASS 100% 88 in → 21 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 298 in → 23 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.7:medium PASS 100% 86 in → 18 out (0 reasoning)
`git show v1.0.0-rc1`
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 298 in → 23 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.7:none PASS 100% 88 in → 21 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.7:none__json_schema PASS 100% 296 in → 23 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.7:xhigh PASS 100% 91 in → 21 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 300 in → 23 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.8:high PASS 100% 83 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-opus-4.8:high__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.8:low PASS 100% 83 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-opus-4.8:low__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.8:max PASS 100% 85 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-opus-4.8:max__json_schema PASS 100% 291 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.8:medium PASS 100% 84 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.8:none PASS 100% 84 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 294 in → 20 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-opus-4.8:xhigh PASS 100% 83 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 294 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-4.6:high PASS 100% 67 in → 65 out (25 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 228 in → 124 out (63 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-4.6:low PASS 100% 66 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 229 in → 18 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-4.6:max PASS 100% 66 in → 250 out (172 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 230 in → 189 out (122 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-4.6:medium PASS 100% 67 in → 47 out (16 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 229 in → 18 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-4.6:none PASS 100% 66 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 229 in → 19 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 66 in → 211 out (556 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 230 in → 18 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-5:high PASS 100% 84 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-sonnet-5:high__json_schema PASS 100% 293 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
anthropic/claude-sonnet-5:low PASS 100% 84 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-sonnet-5:low__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-sonnet-5:medium PASS 100% 81 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 295 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-sonnet-5:none PASS 100% 86 in → 14 out (0 reasoning)
git show v1.0.0-rc1
anthropic/claude-sonnet-5:none__json_schema PASS 100% 292 in → 20 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
anthropic/claude-sonnet-5:xhigh PASS 100% 83 in → 18 out (0 reasoning)
``` git show v1.0.0-rc1 ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 292 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
arcee-ai/trinity-large-thinking:high PASS 100% 64 in → 650 out (647 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-large-thinking:low PASS 100% 64 in → 476 out (473 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-large-thinking:medium PASS 100% 64 in → 409 out (404 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-large-thinking:xhigh PASS 100% 64 in → 563 out (556 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-mini:high PASS 100% 64 in → 302 out (307 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-mini:high__json_schema PASS 100% 63 in → 267 out (251 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
arcee-ai/trinity-mini:low PASS 100% 64 in → 293 out (304 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-mini:low__json_schema PASS 100% 63 in → 276 out (273 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
arcee-ai/trinity-mini:medium PASS 100% 63 in → 234 out (221 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-mini:medium__json_schema PASS 100% 63 in → 335 out (321 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command":"git show v1.0.0-rc1" }
arcee-ai/trinity-mini:xhigh PASS 100% 64 in → 267 out (272 reasoning)
git show v1.0.0-rc1
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 63 in → 269 out (248 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 58 in → 147 out (129 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 138 in → 585 out (468 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 59 in → 65 out (61 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 59 in → 236 out (175 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 137 in → 218 out (167 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 138 in → 1,945 out (1,624 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 59 in → 231 out (218 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 61 in → 92 out (74 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
deepseek/deepseek-v4-flash-0731:none PASS 100% 60 in → 24 out (0 reasoning)
``` git show v1.0.0-rc1 --no-patch --format=fuller ```
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 59 in → 45 out (0 reasoning)
git show v1.0.0-rc1 --no-patch --format='%H%n%an%n%ad%n%s' --date=iso
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1 --no-patch --format='%H%n%an%n%ad%n%s' --date=iso" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 59 in → 86 out (73 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 58 in → 135 out (91 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
deepseek/deepseek-v4-flash:high PASS 100% 60 in → 152 out (139 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash:low PASS 100% 60 in → 310 out (290 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 59 in → 206 out (185 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
deepseek/deepseek-v4-flash:medium PASS 100% 60 in → 155 out (142 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash:none PASS 100% 59 in → 12 out (0 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 60 in → 16 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
deepseek/deepseek-v4-flash:xhigh PASS 100% 139 in → 1,146 out (1,134 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-pro:high PASS 100% 61 in → 32 out (0 reasoning)
git show v1.0.0-rc1 --format="%s%n%an%n%ad" --date=short -s
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 248 in → 133 out (114 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
deepseek/deepseek-v4-pro:low PASS 100% 61 in → 615 out (603 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-pro:medium PASS 100% 58 in → 542 out (528 reasoning)
`git show v1.0.0-rc1`
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 323 in → 980 out (962 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3-flash-preview:high PASS 100% 60 in → 258 out (247 reasoning)
git show v1.0.0-rc1
google/gemini-3-flash-preview:high__json_schema PASS 100% 117 in → 207 out (191 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3-flash-preview:low PASS 100% 59 in → 208 out (197 reasoning)
git show v1.0.0-rc1
google/gemini-3-flash-preview:low__json_schema PASS 100% 117 in → 301 out (286 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3-flash-preview:medium PASS 100% 60 in → 236 out (225 reasoning)
git show v1.0.0-rc1
google/gemini-3-flash-preview:medium__json_schema PASS 100% 116 in → 196 out (181 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3-flash-preview:xhigh PASS 100% 60 in → 174 out (163 reasoning)
git show v1.0.0-rc1
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 116 in → 203 out (187 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 60 in → 312 out (301 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 60 in → 546 out (531 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 60 in → 159 out (148 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 116 in → 136 out (120 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.1-flash-lite-preview:medium PASS 100% 59 in → 123 out (112 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 61 in → 188 out (172 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 60 in → 273 out (262 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 61 in → 267 out (251 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.1-pro-preview:high PASS 100% 57 in → 457 out (446 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 116 in → 167 out (152 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.1-pro-preview:low PASS 100% 60 in → 292 out (281 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 117 in → 259 out (244 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.1-pro-preview:medium PASS 100% 60 in → 338 out (327 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 117 in → 82 out (66 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 59 in → 422 out (411 reasoning)
git show v1.0.0-rc1
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 116 in → 378 out (363 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.5-flash-lite:high PASS 100% 58 in → 316 out (305 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 61 in → 303 out (287 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.5-flash-lite:low PASS 100% 59 in → 11 out (0 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 117 in → 21 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemini-3.5-flash-lite:max PASS 100% 60 in → 153 out (142 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 117 in → 345 out (329 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.5-flash-lite:medium PASS 100% 61 in → 126 out (115 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 115 in → 233 out (217 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 60 in → 256 out (245 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 58 in → 437 out (422 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.5-flash:high PASS 100% 58 in → 442 out (431 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash:high__json_schema PASS 100% 117 in → 437 out (422 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.5-flash:low PASS 100% 59 in → 284 out (273 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash:low__json_schema PASS 100% 116 in → 89 out (73 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemini-3.5-flash:medium PASS 100% 59 in → 591 out (580 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash:medium__json_schema PASS 100% 117 in → 258 out (243 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.5-flash:xhigh PASS 100% 60 in → 324 out (313 reasoning)
git show v1.0.0-rc1
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 115 in → 295 out (280 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.6-flash:high PASS 100% 59 in → 424 out (413 reasoning)
git show v1.0.0-rc1
google/gemini-3.6-flash:high__json_schema PASS 100% 116 in → 379 out (364 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.6-flash:low PASS 100% 61 in → 950 out (939 reasoning)
git show v1.0.0-rc1
google/gemini-3.6-flash:max PASS 100% 58 in → 841 out (830 reasoning)
git show v1.0.0-rc1
google/gemini-3.6-flash:max__json_schema PASS 100% 113 in → 382 out (367 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.6-flash:medium PASS 100% 59 in → 533 out (522 reasoning)
git show v1.0.0-rc1
google/gemini-3.6-flash:medium__json_schema PASS 100% 59 in → 355 out (340 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemini-3.6-flash:xhigh PASS 100% 60 in → 513 out (502 reasoning)
git show v1.0.0-rc1
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 61 in → 475 out (460 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
google/gemma-4-26b-a4b-it:high PASS 100% 76 in → 248 out (236 reasoning)
git show v1.0.0-rc1
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 76 in → 281 out (221 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-26b-a4b-it:low PASS 100% 75 in → 303 out (236 reasoning)
git show v1.0.0-rc1
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 75 in → 236 out (181 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-26b-a4b-it:medium PASS 100% 75 in → 210 out (160 reasoning)
git show v1.0.0-rc1
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 75 in → 409 out (334 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-26b-a4b-it:none PASS 100% 72 in → 12 out (0 reasoning)
git show v1.0.0-rc1
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 73 in → 24 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 75 in → 242 out (217 reasoning)
git show v1.0.0-rc1
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 74 in → 253 out (191 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-31b-it:high PASS 100% 74 in → 220 out (166 reasoning)
git show v1.0.0-rc1
google/gemma-4-31b-it:high__json_schema PASS 100% 76 in → 251 out (173 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-31b-it:low PASS 100% 73 in → 167 out (1 reasoning)
git show v1.0.0-rc1
google/gemma-4-31b-it:low__json_schema PASS 100% 75 in → 167 out (123 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-31b-it:medium PASS 100% 74 in → 236 out (185 reasoning)
git show v1.0.0-rc1
google/gemma-4-31b-it:medium__json_schema PASS 100% 75 in → 405 out (308 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
google/gemma-4-31b-it:none PASS 100% 72 in → 12 out (0 reasoning)
git show v1.0.0-rc1
google/gemma-4-31b-it:none__json_schema PASS 100% 72 in → 17 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
google/gemma-4-31b-it:xhigh PASS 100% 75 in → 226 out (1 reasoning)
git show v1.0.0-rc1
ibm-granite/granite-4.1-8b PASS 100% 68 in → 12 out
git show v1.0.0-rc1
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
inclusionai/ling-2.6-flash PASS 100% 80 in → 33 out
```sh git show v1.0.0-rc1 --no-patch --format='%B%n%an%n%ad' ```
git show --format=fuller v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show --format=fuller v1.0.0-rc1" }
liquid/lfm-2-24b-a2b PASS 100% 66 in → 18 out
```bash git show v1.0.0-rc1 ```
minimax/minimax-m2.5:high PASS 100% 93 in → 559 out (458 reasoning)
git show v1.0.0-rc1
minimax/minimax-m2.5:high__json_schema PASS 100% 94 in → 1,032 out (1,022 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
minimax/minimax-m2.5:low PASS 100% 94 in → 316 out (278 reasoning)
git show v1.0.0-rc1
minimax/minimax-m2.5:low__json_schema PASS 100% 93 in → 440 out (422 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
minimax/minimax-m2.5:medium PASS 100% 101 in → 201 out (188 reasoning)
git show v1.0.0-rc1
minimax/minimax-m2.5:medium__json_schema PASS 100% 99 in → 435 out (415 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
minimax/minimax-m2.5:xhigh PASS 100% 92 in → 404 out (335 reasoning)
git show v1.0.0-rc1
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 93 in → 1,904 out (1,884 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
minimax/minimax-m2.7:high__json_schema PASS 100% 226 in → 437 out (416 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
minimax/minimax-m2.7:xhigh PASS 100% 92 in → 478 out (393 reasoning)
git show v1.0.0-rc1
minimax/minimax-m3:high PASS 100% 232 in → 399 out (337 reasoning)
git show v1.0.0-rc1
minimax/minimax-m3:high__json_schema PASS 100% 219 in → 48 out (0 reasoning)
git show v1.0.0-rc1 --no-patch --format='%nMessage:%n%B%nTagger:%n%an <%ae>%nDate:%n%ad'
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1 --no-patch --format='%nMessage:%n%B%nTagger:%n%an <%ae>%nDate:%n%ad'"}
minimax/minimax-m3:low PASS 100% 230 in → 140 out (124 reasoning)
git show v1.0.0-rc1
minimax/minimax-m3:medium PASS 100% 234 in → 47 out (36 reasoning)
git show v1.0.0-rc1
minimax/minimax-m3:medium__json_schema PASS 100% 217 in → 19 out (0 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
minimax/minimax-m3:xhigh PASS 100% 233 in → 253 out (237 reasoning)
git show v1.0.0-rc1
minimax/minimax-m3:xhigh__json_schema PASS 100% 220 in → 52 out (0 reasoning)
git show v1.0.0-rc1 --no-patch --format='%nMessage: %s%n%nTagger: %an <%ae>%nDate: %ad%n' --date=iso
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1 --no-patch --format='%nMessage: %s%n%nTagger: %an <%ae>%nDate: %ad%n' --date=iso"}
mistralai/devstral-2512 PASS 100% 59 in → 33 out
```bash git show v1.0.0-rc1 --format="%H %an %ad %s" --no-patch ```
mistralai/devstral-2512__json_schema PASS 100% 61 in → 17 out
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
mistralai/mistral-medium-3-5:high PASS 100% 73 in → 432 out (399 reasoning)
git show v1.0.0-rc1
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 59 in → 415 out (365 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
mistralai/mistral-medium-3-5:low PASS 100% 72 in → 434 out (385 reasoning)
git show v1.0.0-rc1
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 60 in → 300 out (275 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
mistralai/mistral-medium-3-5:medium PASS 100% 73 in → 281 out (266 reasoning)
git show v1.0.0-rc1
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 61 in → 306 out (287 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
mistralai/mistral-medium-3-5:none PASS 100% 73 in → 22 out (0 reasoning)
```bash git show v1.0.0-rc1 --format=fuller ```
mistralai/mistral-medium-3-5:xhigh PASS 100% 72 in → 248 out (227 reasoning)
git show v1.0.0-rc1
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 60 in → 349 out (320 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
moonshotai/kimi-k2.6:high PASS 100% 59 in → 575 out (554 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.6:high__json_schema PASS 100% 60 in → 445 out (217 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
moonshotai/kimi-k2.6:low PASS 100% 62 in → 1,167 out (1,148 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.6:medium PASS 100% 61 in → 1,301 out (1,289 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 244 in → 1,022 out (1,006 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
moonshotai/kimi-k2.6:xhigh PASS 100% 61 in → 185 out (173 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 61 in → 765 out (746 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
moonshotai/kimi-k2.7-code:high PASS 100% 60 in → 914 out (902 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 61 in → 640 out (116 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
moonshotai/kimi-k2.7-code:low PASS 100% 64 in → 388 out (376 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.7-code:medium PASS 100% 61 in → 150 out (101 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 61 in → 940 out (848 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 61 in → 193 out (181 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 61 in → 124 out (105 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
moonshotai/kimi-k3:high PASS 100% 139 in → 214 out (189 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:high__json_schema PASS 100% 141 in → 158 out (133 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:low PASS 100% 139 in → 144 out (119 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:low__json_schema PASS 100% 137 in → 73 out (48 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:max PASS 100% 139 in → 117 out (92 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:max__json_schema PASS 100% 138 in → 193 out (163 reasoning)
```bash git show v1.0.0-rc1 ```
moonshotai/kimi-k3:medium PASS 100% 138 in → 232 out (207 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:medium__json_schema PASS 100% 138 in → 202 out (177 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:xhigh PASS 100% 139 in → 196 out (171 reasoning)
git show v1.0.0-rc1
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 137 in → 101 out (76 reasoning)
git show v1.0.0-rc1
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 74 in → 1,287 out (1,130 reasoning)
git show -s v1.0.0-rc1
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 73 in → 397 out (303 reasoning)
git show --pretty=fuller v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show --pretty=fuller v1.0.0-rc1" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 73 in → 784 out (604 reasoning)
git show -s --format="%b%nTagger: %an%nDate: %ad" --date=short v1.0.0-rc1
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 72 in → 179 out (126 reasoning)
git show --pretty=fuller v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show --pretty=fuller v1.0.0-rc1" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 73 in → 474 out (382 reasoning)
git show --pretty=fuller v1.0.0-rc1
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 74 in → 1,072 out (908 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 74 in → 975 out (819 reasoning)
git show -s --format='%b%nTagger: %ci' v1.0.0-rc1
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 74 in → 1,064 out (864 reasoning)
git show --pretty=fuller tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show --pretty=fuller tags/v1.0.0-rc1" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 73 in → 335 out (311 reasoning)
git show v1.0.0-rc1^{tag}
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 73 in → 145 out (125 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 73 in → 271 out (256 reasoning)
git show v1.0.0-rc1
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 74 in → 267 out (243 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 74 in → 146 out (120 reasoning)
git show v1.0.0-rc1
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 74 in → 340 out (320 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 72 in → 28 out (0 reasoning)
git show v1.0.0-rc1 --format="%s%n%an%n%ad" --date=iso
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 73 in → 45 out (0 reasoning)
git show v1.0.0-rc1 --format="Message: %s%nTagger: %GN <%GE>%nDate: %cI"
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1 --format=\"Message: %s%nTagger: %GN <%GE>%nDate: %cI\"" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 74 in → 219 out (178 reasoning)
git show v1.0.0-rc1
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 74 in → 251 out (230 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
openai/gpt-5.4-mini:high PASS 100% 62 in → 1,628 out (1,607 reasoning)
git show --no-patch v1.0.0-rc1
openai/gpt-5.4-mini:low PASS 100% 61 in → 537 out (516 reasoning)
git show --no-patch v1.0.0-rc1
openai/gpt-5.4-mini:none PASS 100% 60 in → 30 out (0 reasoning)
git show --no-patch --format='%s%n%cn%n%cd' v1.0.0-rc1
openai/gpt-5.4-mini:none__json_schema PASS 100% 96 in → 32 out (0 reasoning)
git show --no-patch --format=fuller v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show --no-patch --format=fuller v1.0.0-rc1"}
openai/gpt-5.4-mini:xhigh PASS 100% 62 in → 6,231 out (6,214 reasoning)
git show v1.0.0-rc1
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 96 in → 6,018 out (5,993 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
openai/gpt-5.4-nano:high PASS 100% 62 in → 107 out (86 reasoning)
git show v1.0.0-rc1 --no-patch
openai/gpt-5.4-nano:high__json_schema PASS 100% 96 in → 38 out (0 reasoning)
git show -s --format=%%s%n%%aN%n%%aD v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show -s --format=%%s%n%%aN%n%%aD v1.0.0-rc1"}
openai/gpt-5.4-nano:low PASS 100% 62 in → 42 out (0 reasoning)
git show -s --format=%B%nTagger: %an <%aE>%nDate: %ad --date=iso-strict v1.0.0-rc1
openai/gpt-5.4-nano:low__json_schema PASS 100% 97 in → 41 out (0 reasoning)
git show -s --format=%B%n%an%n%ad --date=iso8601 v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show -s --format=%B%n%an%n%ad --date=iso8601 v1.0.0-rc1"}
openai/gpt-5.4-nano:medium__json_schema PASS 100% 96 in → 43 out (0 reasoning)
git show -s --format=%B%nTagger: %an <%ae>%nDate: %ad v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show -s --format=%B%nTagger: %an <%ae>%nDate: %ad v1.0.0-rc1"}
openai/gpt-5.4-nano:none PASS 100% 62 in → 40 out (0 reasoning)
git show --quiet --format='%B%nTagger: %an <%ae>%nDate: %ad' v1.0.0-rc1 --no-patch
openai/gpt-5.4-nano:none__json_schema PASS 100% 96 in → 37 out (0 reasoning)
git show -s --format=%B%n%aN%n%aD v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show -s --format=%B%n%aN%n%aD v1.0.0-rc1"}
openai/gpt-5.4-nano:xhigh PASS 100% 61 in → 1,576 out (1,555 reasoning)
git show v1.0.0-rc1 --no-patch
openai/gpt-5.4:xhigh__json_schema PASS 100% 96 in → 17,620 out (17,591 reasoning)
git show --no-patch v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show --no-patch v1.0.0-rc1"}
openai/gpt-5.5:high PASS 100% 62 in → 537 out (516 reasoning)
git show --no-patch v1.0.0-rc1
openai/gpt-5.5:xhigh PASS 100% 61 in → 533 out (516 reasoning)
git show v1.0.0-rc1
openai/gpt-5.5:xhigh__json_schema PASS 100% 96 in → 541 out (516 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
openai/gpt-5.6-luna:high PASS 100% 62 in → 319 out (302 reasoning)
git show v1.0.0-rc1
openai/gpt-5.6-luna:high__json_schema PASS 100% 97 in → 541 out (516 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
openai/gpt-5.6-luna:max PASS 100% 61 in → 533 out (516 reasoning)
git show v1.0.0-rc1
openai/gpt-5.6-luna:max__json_schema PASS 100% 96 in → 697 out (672 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
openai/gpt-5.6-luna:none__json_schema PASS 100% 96 in → 32 out (0 reasoning)
git show --format=fuller --no-patch v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show --format=fuller --no-patch v1.0.0-rc1"}
openai/gpt-5.6-luna:xhigh PASS 100% 61 in → 533 out (516 reasoning)
git show v1.0.0-rc1
openai/gpt-5.6-sol:low PASS 100% 60 in → 143 out (126 reasoning)
git show v1.0.0-rc1
openai/gpt-5.6-sol:low__json_schema PASS 100% 96 in → 57 out (0 reasoning)
git show --no-patch --format='%(contents)%nTagger: %(taggername) <%(taggeremail)>%nDate: %(taggerdate)' v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show --no-patch --format='%(contents)%nTagger: %(taggername) <%(taggeremail)>%nDate: %(taggerdate)' v1.0.0-rc1"}
openai/gpt-5.6-sol:max PASS 100% 61 in → 282 out (265 reasoning)
git show v1.0.0-rc1
openai/gpt-5.6-sol:medium PASS 100% 62 in → 184 out (163 reasoning)
git show --no-patch v1.0.0-rc1
openai/gpt-5.6-sol:none PASS 100% 62 in → 24 out (0 reasoning)
git show v1.0.0-rc1 --no-patch --format=fuller
openai/gpt-5.6-sol:xhigh PASS 100% 60 in → 537 out (516 reasoning)
git show --no-patch v1.0.0-rc1
openai/gpt-5.6-terra:max PASS 100% 62 in → 356 out (339 reasoning)
git show v1.0.0-rc1
openai/gpt-5.6-terra:xhigh PASS 100% 62 in → 358 out (341 reasoning)
git show v1.0.0-rc1
openai/gpt-oss-120b:high PASS 100% 122 in → 214 out (178 reasoning)
git show v1.0.0-rc1 --no-patch --pretty=format:"%s%nTagger: %an <%ae>%nDate: %ad"
openai/gpt-oss-120b:high__json_schema PASS 100% 109 in → 644 out (524 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
openai/gpt-oss-120b:low PASS 100% 121 in → 287 out (237 reasoning)
git show v1.0.0-rc1 --no-patch --pretty=format:"Message: %s%nTagger: %an <%ae>%nDate: %ad"
openai/gpt-oss-120b:medium__json_schema PASS 100% 120 in → 136 out (83 reasoning)
git show v1.0.0-rc1 --no-patch --pretty=fuller
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1 --no-patch --pretty=fuller"}
openai/gpt-oss-120b:xhigh PASS 100% 122 in → 2,574 out (2,555 reasoning)
git show v1.0.0-rc1
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 121 in → 1,833 out (1,815 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
openai/gpt-oss-20b:high PASS 100% 122 in → 1,462 out (1,333 reasoning)
git show v1.0.0-rc1
openai/gpt-oss-20b:high__json_schema PASS 100% 111 in → 513 out (390 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
openai/gpt-oss-20b:low PASS 100% 124 in → 52 out (24 reasoning)
git show v1.0.0-rc1
openai/gpt-oss-20b:medium PASS 100% 124 in → 365 out (289 reasoning)
git show v1.0.0-rc1
openai/gpt-oss-20b:medium__json_schema PASS 100% 125 in → 486 out (322 reasoning)
git show -s --format="%aN <%aE>%n%ai%n%s" v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show -s --format=\"%aN <%aE>%n%ai%n%s\" v1.0.0-rc1" }
openai/gpt-oss-20b:xhigh PASS 100% 119 in → 376 out (335 reasoning)
git show v1.0.0-rc1
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 122 in → 1,183 out (1,163 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
poolside/laguna-m.1:high PASS 100% 70 in → 341 out (328 reasoning)
git show v1.0.0-rc1
poolside/laguna-m.1:high__json_schema PASS 100% 71 in → 223 out (206 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
poolside/laguna-m.1:low PASS 100% 71 in → 335 out (322 reasoning)
git show v1.0.0-rc1
poolside/laguna-m.1:low__json_schema PASS 100% 72 in → 289 out (269 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
poolside/laguna-m.1:medium PASS 100% 70 in → 540 out (527 reasoning)
git show v1.0.0-rc1
poolside/laguna-m.1:medium__json_schema PASS 100% 69 in → 388 out (366 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
poolside/laguna-m.1:none PASS 100% 71 in → 17 out (0 reasoning)
```bash git show v1.0.0-rc1 ```
poolside/laguna-m.1:xhigh PASS 100% 69 in → 597 out (585 reasoning)
git show v1.0.0-rc1
poolside/laguna-m.1:xhigh__json_schema PASS 100% 69 in → 713 out (697 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
poolside/laguna-xs-2.1:high PASS 100% 72 in → 325 out (314 reasoning)
git show v1.0.0-rc1
poolside/laguna-xs-2.1:high__json_schema PASS 100% 71 in → 398 out (376 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
poolside/laguna-xs-2.1:low PASS 100% 71 in → 279 out (268 reasoning)
git show v1.0.0-rc1
poolside/laguna-xs-2.1:low__json_schema PASS 100% 68 in → 246 out (229 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
poolside/laguna-xs-2.1:medium PASS 100% 72 in → 667 out (656 reasoning)
git show v1.0.0-rc1
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 72 in → 435 out (413 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
poolside/laguna-xs-2.1:xhigh PASS 100% 72 in → 280 out (270 reasoning)
git show v1.0.0-rc1
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 70 in → 365 out (348 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
poolside/laguna-xs.2:high PASS 100% 109 in → 394 out (378 reasoning)
git show v1.0.0-rc1 --no-patch
poolside/laguna-xs.2:high__json_schema PASS 100% 107 in → 222 out (205 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
poolside/laguna-xs.2:low PASS 100% 107 in → 136 out (121 reasoning)
`git show v1.0.0-rc1`
poolside/laguna-xs.2:low__json_schema PASS 100% 109 in → 221 out (204 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
poolside/laguna-xs.2:medium PASS 100% 109 in → 220 out (207 reasoning)
git show v1.0.0-rc1
poolside/laguna-xs.2:medium__json_schema PASS 100% 107 in → 123 out (102 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
poolside/laguna-xs.2:none PASS 100% 109 in → 21 out (0 reasoning)
git show v1.0.0-rc1 --format="%m %an %ad"
poolside/laguna-xs.2:none__json_schema PASS 100% 109 in → 31 out (0 reasoning)
git show v1.0.0-rc1 --format="%s%n%an%n%ad" --no-patch
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1 --format=\"%s%n%an%n%ad\" --no-patch"}
poolside/laguna-xs.2:xhigh PASS 100% 109 in → 429 out (401 reasoning)
`git show v1.0.0-rc1 --no-patch --format="%h %s %an %ad"`
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 109 in → 218 out (202 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
qwen/qwen3.6-27b:high PASS 100% 68 in → 2,502 out (1,878 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-27b:high__json_schema PASS 100% 69 in → 1,777 out (1 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-27b:low PASS 100% 68 in → 1,577 out (1,239 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-27b:low__json_schema PASS 100% 70 in → 1,955 out (1,486 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-27b:medium PASS 100% 70 in → 2,084 out (1,599 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-27b:medium__json_schema PASS 100% 67 in → 1,809 out (1,399 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-27b:none PASS 100% 71 in → 12 out (0 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-27b:xhigh PASS 100% 70 in → 1,131 out (919 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 69 in → 1,763 out (1,393 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-35b-a3b:high PASS 100% 69 in → 1,669 out (1,323 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 68 in → 1,398 out (1,106 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-35b-a3b:low PASS 100% 69 in → 1,911 out (1,521 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 68 in → 1,671 out (1,326 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 69 in → 1,659 out (1,313 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 70 in → 1,664 out (1,304 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-35b-a3b:none PASS 100% 71 in → 12 out (0 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 69 in → 1,070 out (1,054 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 68 in → 1,655 out (1,311 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
qwen/qwen3.6-flash:high PASS 100% 68 in → 1,335 out (1,319 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-flash:low PASS 100% 68 in → 1,542 out (1,526 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-flash:medium PASS 100% 69 in → 1,143 out (1,127 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-flash:none PASS 100% 72 in → 11 out (0 reasoning)
git show v1.0.0-rc1
qwen/qwen3.6-flash:xhigh PASS 100% 70 in → 2,844 out (2,828 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-flash:high PASS 100% 70 in → 1,161 out (1,145 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-flash:low PASS 100% 70 in → 2,891 out (2,875 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-flash:max PASS 100% 69 in → 1,482 out (1,466 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-flash:medium PASS 100% 69 in → 1,537 out (1,521 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-flash:xhigh PASS 100% 68 in → 862 out (846 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-max:high PASS 100% 70 in → 1,065 out (1,049 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-max:low PASS 100% 69 in → 1,006 out (990 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-max:medium PASS 100% 70 in → 1,182 out (1,166 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-max:xhigh PASS 100% 70 in → 948 out (932 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-plus:high PASS 100% 69 in → 1,246 out (1,230 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-plus:low PASS 100% 68 in → 863 out (847 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-plus:medium PASS 100% 69 in → 1,529 out (1,513 reasoning)
git show v1.0.0-rc1
qwen/qwen3.7-plus:xhigh PASS 100% 70 in → 1,282 out (1,266 reasoning)
git show v1.0.0-rc1
tencent/hy3:high PASS 100% 68 in → 2,195 out (2,182 reasoning)
git show v1.0.0-rc1
tencent/hy3:high__json_schema PASS 100% 67 in → 1,845 out (1,511 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
tencent/hy3:low PASS 100% 69 in → 728 out (559 reasoning)
git show v1.0.0-rc1
tencent/hy3:low__json_schema PASS 100% 66 in → 1,253 out (958 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
tencent/hy3:medium PASS 100% 67 in → 1,314 out (1,050 reasoning)
git show v1.0.0-rc1
tencent/hy3:medium__json_schema PASS 100% 68 in → 956 out (750 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
tencent/hy3:none PASS 100% 71 in → 12 out (0 reasoning)
git show v1.0.0-rc1
tencent/hy3:xhigh PASS 100% 67 in → 1,801 out (1,788 reasoning)
git show v1.0.0-rc1
tencent/hy3:xhigh__json_schema PASS 100% 68 in → 1,041 out (808 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
thinkingmachines/inkling-small:low PASS 100% 71 in → 275 out (258 reasoning)
git show v1.0.0-rc1
thinkingmachines/inkling-small:low__json_schema PASS 100% 72 in → 226 out (201 reasoning)
git show v1.0.0-rc1 --no-patch
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1 --no-patch"}
thinkingmachines/inkling-small:max PASS 100% 72 in → 432 out (411 reasoning)
git show v1.0.0-rc1 --no-patch
thinkingmachines/inkling-small:medium__json_schema PASS 100% 69 in → 257 out (233 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
thinkingmachines/inkling:high__json_schema PASS 100% 69 in → 559 out (536 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
thinkingmachines/inkling:low__json_schema PASS 100% 71 in → 132 out (109 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
thinkingmachines/inkling:max PASS 100% 70 in → 536 out (436 reasoning)
git show v1.0.0-rc1
thinkingmachines/inkling:medium PASS 100% 70 in → 364 out (291 reasoning)
git show v1.0.0-rc1
thinkingmachines/inkling:xhigh PASS 100% 70 in → 391 out (319 reasoning)
git show v1.0.0-rc1
thinkingmachines/inkling:xhigh__json_schema PASS 100% 71 in → 594 out (571 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
x-ai/grok-4.3:high PASS 100% 238 in → 1,048 out (1,038 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.3:high__json_schema PASS 100% 294 in → 973 out (955 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
x-ai/grok-4.3:max PASS 100% 239 in → 402 out (392 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.3:max__json_schema PASS 100% 294 in → 952 out (938 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
x-ai/grok-4.3:medium PASS 100% 243 in → 1,521 out (1,511 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.3:medium__json_schema PASS 100% 300 in → 711 out (696 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
x-ai/grok-4.3:none PASS 100% 237 in → 34 out (0 reasoning)
git show v1.0.0-rc1 --quiet --format="message: %s%ntagger: %an <%ae>%ndate: %ad"
x-ai/grok-4.3:none__json_schema PASS 100% 291 in → 31 out (0 reasoning)
git show v1.0.0-rc1 --quiet --format="%s%n%an <%ae>%n%ad"
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1 --quiet --format=\"%s%n%an <%ae>%n%ad\""}
x-ai/grok-4.3:xhigh PASS 100% 238 in → 613 out (603 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.3:xhigh__json_schema PASS 100% 295 in → 705 out (691 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
x-ai/grok-4.5:high PASS 100% 261 in → 313 out (302 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.5:high__json_schema PASS 100% 327 in → 367 out (351 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
x-ai/grok-4.5:low PASS 100% 262 in → 417 out (406 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.5:low__json_schema PASS 100% 326 in → 301 out (281 reasoning)
git show v1.0.0-rc1 --no-patch
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1 --no-patch"}
x-ai/grok-4.5:max PASS 100% 263 in → 707 out (696 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.5:max__json_schema PASS 100% 328 in → 390 out (374 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
x-ai/grok-4.5:medium PASS 100% 261 in → 395 out (384 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.5:medium__json_schema PASS 100% 328 in → 667 out (651 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
x-ai/grok-4.5:xhigh PASS 100% 261 in → 562 out (551 reasoning)
git show v1.0.0-rc1
x-ai/grok-4.5:xhigh__json_schema PASS 100% 328 in → 364 out (348 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
z-ai/glm-4.7-flash:high PASS 100% 59 in → 432 out (355 reasoning)
git show v1.0.0-rc1
z-ai/glm-4.7-flash:low PASS 100% 60 in → 433 out (401 reasoning)
git show v1.0.0-rc1
z-ai/glm-4.7-flash:low__json_schema PASS 100% 61 in → 620 out (540 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
z-ai/glm-4.7-flash:medium PASS 100% 60 in → 376 out (311 reasoning)
git show v1.0.0-rc1
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 59 in → 368 out (304 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
z-ai/glm-4.7-flash:none PASS 100% 61 in → 32 out (0 reasoning)
git show v1.0.0-rc1 --format="%H%n%an%n%ai%n%s%n%b" --no-patch
z-ai/glm-4.7-flash:none__json_schema PASS 100% 62 in → 69 out (0 reasoning)
git show v1.0.0-rc1 --format="%(refname:short): %(subject)\n%(taggername): %(taggerdate)\n%(taggeremail)\n%(taggerdate:iso)\n%(body)" --no-patch
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1 --format=\"%(refname:short): %(subject)\\n%(taggername): %(taggerdate)\\n%(taggeremail)\\n%(taggerdate:iso)\\n%(body)\" --no-patch" }
z-ai/glm-4.7-flash:xhigh PASS 100% 61 in → 416 out (329 reasoning)
git show v1.0.0-rc1
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 62 in → 448 out (365 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command": "git show v1.0.0-rc1"}
z-ai/glm-5.2:high PASS 100% 67 in → 426 out (325 reasoning)
git show v1.0.0-rc1
z-ai/glm-5.2:high__json_schema PASS 100% 68 in → 236 out (219 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
z-ai/glm-5.2:low PASS 100% 67 in → 137 out (124 reasoning)
git show v1.0.0-rc1
z-ai/glm-5.2:low__json_schema PASS 100% 67 in → 396 out (307 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git show v1.0.0-rc1" }
z-ai/glm-5.2:medium PASS 100% 67 in → 145 out (132 reasoning)
git show v1.0.0-rc1
z-ai/glm-5.2:medium__json_schema PASS 100% 243 in → 164 out (147 reasoning)
git show v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git show v1.0.0-rc1"}
z-ai/glm-5.2:none PASS 100% 61 in → 12 out (0 reasoning)
git show v1.0.0-rc1
z-ai/glm-5.2:xhigh PASS 100% 68 in → 492 out (479 reasoning)
git show v1.0.0-rc1
deepseek/deepseek-v4-flash:high__json_schema FAIL 66.7% 151 in → 39 out (0 reasoning)
git tag -l 'v1.0.0-rc1' --format='%(contents) %(taggername) %(taggerdate)'
JSON Schema Structured Output
(raw) {"command": "git tag -l 'v1.0.0-rc1' --format='%(contents) %(taggername) %(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
deepseek/deepseek-v4-flash:medium__json_schema FAIL 66.7% 61 in → 155 out (127 reasoning)
git tag -n1 -l 'v1.0.0-rc1'
JSON Schema Structured Output
(raw) { "command": "git tag -n1 -l 'v1.0.0-rc1'" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
deepseek/deepseek-v4-pro:low__json_schema FAIL 66.7% 60 in → 590 out (451 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git cat-file -p v1.0.0-rc1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
deepseek/deepseek-v4-pro:medium__json_schema FAIL 66.7% 58 in → 41 out (0 reasoning)
git tag -l --format='%(refname:strip=2) %(taggername) %(taggerdate)' v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git tag -l --format='%(refname:strip=2) %(taggername) %(taggerdate)' v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
deepseek/deepseek-v4-pro:none FAIL 66.7% 60 in → 28 out (0 reasoning)
git log -1 --format="%s%n%cn%n%cd" v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
deepseek/deepseek-v4-pro:xhigh FAIL 66.7% 137 in → 1,075 out (1,059 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
google/gemini-3.6-flash:low__json_schema FAIL 66.7% 117 in → 32 out (0 reasoning)
git tag -l --format='%(contents)%(tagger)%(taggerdate)' v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git tag -l --format='%(contents)%(tagger)%(taggerdate)' v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
minimax/minimax-m2.7:high FAIL 66.7% 97 in → 643 out (628 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
minimax/minimax-m2.7:low FAIL 66.7% 92 in → 867 out (712 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
minimax/minimax-m2.7:medium FAIL 66.7% 92 in → 447 out (358 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
minimax/minimax-m2.7:medium__json_schema FAIL 66.7% 226 in → 710 out (683 reasoning)
git cat-file -p refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git cat-file -p refs/tags/v1.0.0-rc1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
minimax/minimax-m2.7:xhigh__json_schema FAIL 66.7% 225 in → 667 out (648 reasoning)
git cat-file tag v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git cat-file tag v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
nvidia/nemotron-3-nano-30b-a3b:none FAIL 66.7% 72 in → 26 out (0 reasoning)
list-oid sha256{$(git rev-parse --short=v1.0.0-rc1)}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
nvidia/nemotron-3-nano-30b-a3b:none__json_schema FAIL 66.7% 73 in → 48 out (0 reasoning)
git log --tags=v1.0.0-rc1 --pretty=format:'%H%x09%an%x09%ad' --date=short -n 1
JSON Schema Structured Output
(raw) { "command": "git log --tags=v1.0.0-rc1 --pretty=format:'%H%x09%an%x09%ad' --date=short -n 1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4-mini:high__json_schema FAIL 66.7% 96 in → 3,165 out (3,106 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(taggername) <%(taggeremail)> %(taggerdate:iso8601)%0A%(contents)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(taggername) <%(taggeremail)> %(taggerdate:iso8601)%0A%(contents)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4-mini:low__json_schema FAIL 66.7% 98 in → 254 out (175 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(refname:short)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate:iso8601)%0aMessage:%0a%(contents)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(refname:short)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate:iso8601)%0aMessage:%0a%(contents)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4-mini:medium FAIL 66.7% 61 in → 297 out (246 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(newline)%(taggername) %(taggeremail) %(taggerdate:iso8601)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4-mini:medium__json_schema FAIL 66.7% 96 in → 569 out (516 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(taggername) %(taggerdate:iso8601)%0a%(contents)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(taggername) %(taggerdate:iso8601)%0a%(contents)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4-nano:medium FAIL 66.7% 60 in → 796 out (738 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0ATagger: %(taggername) <%(taggeremail)>%0ADate: %(taggerdate:iso8601)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4-nano:xhigh__json_schema FAIL 66.7% 97 in → 9,146 out (9,118 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git cat-file -p v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:high FAIL 66.7% 61 in → 570 out (516 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='message: %(contents)%ntagger: %(taggername) <%(taggeremail)>%ndate: %(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:high__json_schema FAIL 66.7% 97 in → 583 out (516 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents:subject)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents:subject)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:low FAIL 66.7% 62 in → 304 out (235 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(if)%(taggername)%(then)%09%(taggername)%(end)%(if)%(taggerdate)%(then)%09%(taggerdate)%(end)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:low__json_schema FAIL 66.7% 97 in → 162 out (87 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(if)%(contents)%(then)%(end)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(if)%(contents)%(then)%(end)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:medium FAIL 66.7% 62 in → 332 out (275 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:medium__json_schema FAIL 66.7% 97 in → 338 out (278 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='Tagger: %(taggername) %(taggeremail)%nDate: %(taggerdate)%n%n%(contents)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='Tagger: %(taggername) %(taggeremail)%nDate: %(taggerdate)%n%n%(contents)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:none FAIL 66.7% 60 in → 53 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(if)%(taggername)%(then)%09%(taggername)%09%(taggerdate)%(end)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:none__json_schema FAIL 66.7% 95 in → 67 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(taggername) <%(taggeremail)>%(if)%(taggerdate)%(then)%0a%(taggerdate)%(end)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(taggername) <%(taggeremail)>%(if)%(taggerdate)%(then)%0a%(taggerdate)%(end)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.4:xhigh FAIL 66.7% 62 in → 4,357 out (4,300 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:high__json_schema FAIL 66.7% 96 in → 515 out (459 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0a%(taggername) %(taggeremail)%0a%(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0a%(taggername) %(taggeremail)%0a%(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:low FAIL 66.7% 60 in → 358 out (302 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%09%(taggername) <%(taggeremail:trim)>%09%(taggerdate:iso8601)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:low__json_schema FAIL 66.7% 97 in → 54 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(taggername) <%(taggeremail)>%0a%(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(taggername) <%(taggeremail)>%0a%(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:medium FAIL 66.7% 61 in → 568 out (516 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0a%(taggername) %(taggeremail)%0a%(taggerdate:iso8601)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:medium__json_schema FAIL 66.7% 95 in → 360 out (298 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0atagger: %(taggername) <%(taggeremail)>%0adate: %(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0atagger: %(taggername) <%(taggeremail)>%0adate: %(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:none FAIL 66.7% 61 in → 37 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents) %(taggername) %(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.5:none__json_schema FAIL 66.7% 95 in → 54 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(taggername) <%(taggeremail)>%0a%(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%(taggername) <%(taggeremail)>%0a%(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-luna:low FAIL 66.7% 62 in → 318 out (268 reasoning)
git for-each-ref --format='Message: %(contents:subject)%nTagger: %(taggername)%nDate: %(taggerdate)' refs/tags/v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-luna:low__json_schema FAIL 66.7% 99 in → 382 out (328 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents)Tagger: %(taggername)Date: %(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents)Tagger: %(taggername)Date: %(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-luna:medium FAIL 66.7% 60 in → 491 out (431 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='tag: %(refname:short)%nmessage: %(contents)%ntagger: %(taggername)%ndate: %(taggerdate:iso8601)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-luna:medium__json_schema FAIL 66.7% 96 in → 371 out (311 reasoning)
git for-each-ref --format='Message: %(contents)%nTagger: %(taggername)%nDate: %(taggerdate:iso8601)' refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git for-each-ref --format='Message: %(contents)%nTagger: %(taggername)%nDate: %(taggerdate:iso8601)' refs/tags/v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-luna:none FAIL 66.7% 60 in → 41 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%09%(taggername)%09%(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-luna:xhigh__json_schema FAIL 66.7% 98 in → 572 out (516 reasoning)
git for-each-ref --format='Message: %(contents)%nTagger: %(taggername)%nDate: %(taggerdate)' refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git for-each-ref --format='Message: %(contents)%nTagger: %(taggername)%nDate: %(taggerdate)' refs/tags/v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-sol:high FAIL 66.7% 61 in → 258 out (215 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%n%(taggername)%n%(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-sol:high__json_schema FAIL 66.7% 95 in → 65 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate:iso8601)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate:iso8601)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-sol:max__json_schema FAIL 66.7% 95 in → 1,103 out (1,034 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents:subject)%0aTagger: %(taggername) %(taggeremail)%0aDate: %(taggerdate:iso8601)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='Message: %(contents:subject)%0aTagger: %(taggername) %(taggeremail)%0aDate: %(taggerdate:iso8601)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-sol:medium__json_schema FAIL 66.7% 96 in → 285 out (227 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0a%(taggername) <%(taggeremail)>%0a%(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents)%0a%(taggername) <%(taggeremail)>%0a%(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-sol:none__json_schema FAIL 66.7% 96 in → 64 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='message: %(contents)%ntagger: %(taggername) <%(taggeremail)>%ndate: %(taggerdate:iso8601)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='message: %(contents)%ntagger: %(taggername) <%(taggeremail)>%ndate: %(taggerdate:iso8601)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-sol:xhigh__json_schema FAIL 66.7% 96 in → 372 out (309 reasoning)
git for-each-ref --format='Message: %(contents)%0aTagger: %(taggername) %(taggeremail)%0aDate: %(taggerdate)' refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git for-each-ref --format='Message: %(contents)%0aTagger: %(taggername) %(taggeremail)%0aDate: %(taggerdate)' refs/tags/v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:high FAIL 66.7% 61 in → 192 out (147 reasoning)
git for-each-ref --format='%(contents:subject)%0a%(taggername)%0a%(taggerdate)' refs/tags/v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:high__json_schema FAIL 66.7% 96 in → 185 out (128 reasoning)
git for-each-ref --format='%(contents)%(taggername) <%(taggeremail)>%(taggerdate:iso8601)' refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git for-each-ref --format='%(contents)%(taggername) <%(taggeremail)>%(taggerdate:iso8601)' refs/tags/v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:low FAIL 66.7% 62 in → 265 out (220 reasoning)
git for-each-ref --format='%(contents:subject)%0a%(taggername)%0a%(taggerdate)' refs/tags/v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:low__json_schema FAIL 66.7% 96 in → 62 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%0a%(taggername) <%(taggeremail)>%0a%(taggerdate:iso8601)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%0a%(taggername) <%(taggeremail)>%0a%(taggerdate:iso8601)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:max__json_schema FAIL 66.7% 97 in → 275 out (247 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git cat-file -p v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:medium FAIL 66.7% 60 in → 131 out (111 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:medium__json_schema FAIL 66.7% 97 in → 58 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%0a%(taggername) <%(taggeremail)>%0a%(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%0a%(taggername) <%(taggeremail)>%0a%(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:none FAIL 66.7% 60 in → 48 out (0 reasoning)
git for-each-ref refs/tags/v1.0.0-rc1 --format='%(contents:subject)%09%(taggername) <%(*taggeremail)>%09%(taggerdate)'
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:none__json_schema FAIL 66.7% 96 in → 70 out (0 reasoning)
git tag -n99 v1.0.0-rc1 && git for-each-ref refs/tags/v1.0.0-rc1 --format='Tagger: %(taggername) <%(taggeremail)>%nDate: %(taggerdate)'
JSON Schema Structured Output
(raw) {"command":"git tag -n99 v1.0.0-rc1 && git for-each-ref refs/tags/v1.0.0-rc1 --format='Tagger: %(taggername) <%(taggeremail)>%nDate: %(taggerdate)'"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-5.6-terra:xhigh__json_schema FAIL 66.7% 96 in → 518 out (437 reasoning)
git for-each-ref --format='Message: %(contents:subject)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate:iso8601)%0a%0a%(contents:body)' refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git for-each-ref --format='Message: %(contents:subject)%0aTagger: %(taggername) <%(taggeremail)>%0aDate: %(taggerdate:iso8601)%0a%0a%(contents:body)' refs/tags/v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-oss-120b:low__json_schema FAIL 66.7% 121 in → 141 out (91 reasoning)
git log -1 --pretty=format:'%h %s%nTagger: %an <%ae>%nDate: %ad' v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git log -1 --pretty=format:'%h %s%nTagger: %an <%ae>%nDate: %ad' v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-oss-120b:medium FAIL 66.7% 119 in → 524 out (403 reasoning)
git for-each-ref --format='%(refname:short) %(taggername) %(taggerdate) %(subject)' refs/tags/v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
openai/gpt-oss-20b:low__json_schema FAIL 66.7% 122 in → 154 out (115 reasoning)
git log -1 v1.0.0-rc1 --pretty=format:"%s,%an,%ad"
JSON Schema Structured Output
(raw) { "command":"git log -1 v1.0.0-rc1 --pretty=format:\"%s,%an,%ad\"" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
poolside/laguna-xs-2.1:none FAIL 66.7% 70 in → 25 out (0 reasoning)
git log -1 --pretty=format:"%s%n%an%n%ad" v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
poolside/laguna-xs-2.1:none__json_schema FAIL 66.7% 71 in → 45 out (0 reasoning)
git for-each-ref --format='%(message:subject)\n%(taggername)\n%(taggerdate)' refs/tags/v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git for-each-ref --format='%(message:subject)\\n%(taggername)\\n%(taggerdate)' refs/tags/v1.0.0-rc1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
qwen/qwen3.7-flash:none FAIL 66.7% 71 in → 13 out (0 reasoning)
git tag -v v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
qwen/qwen3.7-max:none FAIL 66.7% 72 in → 29 out (0 reasoning)
git tag -l --format='%(contents) %(taggername) %(taggerdate)' v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
qwen/qwen3.7-plus:none FAIL 66.7% 70 in → 13 out (0 reasoning)
git tag -v v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:high FAIL 66.7% 71 in → 386 out (275 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:high__json_schema FAIL 66.7% 71 in → 765 out (739 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git cat-file -p v1.0.0-rc1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:max__json_schema FAIL 66.7% 70 in → 1,922 out (1,895 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git cat-file -p v1.0.0-rc1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:medium FAIL 66.7% 69 in → 315 out (227 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:none FAIL 66.7% 67 in → 48 out (0 reasoning)
git tag -l -n1 v1.0.0-rc1 --format='%(contents:subject) %(taggername) %(taggerdate:short)' v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:xhigh FAIL 66.7% 70 in → 707 out (687 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling-small:xhigh__json_schema FAIL 66.7% 69 in → 584 out (558 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) { "command": "git cat-file -p v1.0.0-rc1" }
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling:high FAIL 66.7% 71 in → 1,396 out (1,171 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling:low FAIL 66.7% 71 in → 180 out (159 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
thinkingmachines/inkling:none FAIL 66.7% 69 in → 60 out (0 reasoning)
git for-each-ref --format='%(refname:short) %(objecttype) %(objectname:short) %(contents:subject) %(contents:body) %(taggeremail) %(taggerdate)' refs/tags/v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
x-ai/grok-4.3:low FAIL 66.7% 245 in → 512 out (499 reasoning)
git cat-file -p v1.0.0-rc1
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
x-ai/grok-4.3:low__json_schema FAIL 66.7% 299 in → 677 out (660 reasoning)
git cat-file -p v1.0.0-rc1
JSON Schema Structured Output
(raw) {"command":"git cat-file -p v1.0.0-rc1"}
Failure: Failed assertions: [{'type': 'model_output', 'contains': 'git show'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git show v1.0.0-rc1
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git show v1{ "command": "git show v1.0.0-rc1" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: ```bash git show v1.0.0-rc1 ```
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-928add30-a4cc-9c0f-9923-3bbf369aba20","request_id":"928add30-a4cc-9c0f-9923-3bbf369aba20"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-95eaad5c-d384-9f08-8aac-ec8c011d6526","request_id":"95eaad5c-d384-9f08-8aac-ec8c011d6526"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6b33c3cd-1520-944b-a9a8-ad7704c11bc1","request_id":"6b33c3cd-1520-944b-a9a8-ad7704c11bc1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-20ce4ccf-f8cf-9fe5-91d8-f6c7b5551948","request_id":"20ce4ccf-f8cf-9fe5-91d8-f6c7b5551948"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f70960d0-adcf-97bd-8820-eaffa1de90c8","request_id":"f70960d0-adcf-97bd-8820-eaffa1de90c8"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-897942ef-fd31-9143-96ff-f19ab967ef69"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-bfea5603-6ed9-9b93-acea-04e71066be12"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-9df544d3-696d-9a5e-8ace-ec83eedcfc52"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-d253c1a5-8cea-99df-a60a-b007ecda2ecf"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-fcaf7dd8-41c0-992c-aa1e-2204111e88fd"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-47a274ea-71b2-925d-8870-1d74e27d38dd"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-11c45849-f501-9bff-8afa-526c88b9a68a","request_id":"11c45849-f501-9bff-8afa-526c88b9a68a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-006efcec-1758-9d3a-be30-a36991efe1be","request_id":"006efcec-1758-9d3a-be30-a36991efe1be"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-270c30e0-6bcf-90b1-b4cb-26bc9d2e6241","request_id":"270c30e0-6bcf-90b1-b4cb-26bc9d2e6241"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-804d6d2c-05a2-9f49-b826-a0517414948e","request_id":"804d6d2c-05a2-9f49-b826-a0517414948e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-af3efc9e-c7b9-9620-bfae-1659339bac1a","request_id":"af3efc9e-c7b9-9620-bfae-1659339bac1a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4bf97bcf-7785-9cec-8746-d5c600656bf0","request_id":"4bf97bcf-7785-9cec-8746-d5c600656bf0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-18ccb03c-7965-9694-8066-0529d15fae39","request_id":"18ccb03c-7965-9694-8066-0529d15fae39"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Connection error.
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-39344438-8361-9857-9262-5776d05d6bad","request_id":"39344438-8361-9857-9262-5776d05d6bad"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Model 'tencent/hy3:none' violated the reasoning_level=none invariant: provider reported 1096 reasoning token(s)
Invalid JSON. Output: git show v1.0.0-rc1 --quiet --format='%(contents:subject)%nTagger: %(taggername) <%(taggeremail)>%nDate: %(taggerdate:format:%c)'{"command": "git show v1.0.0-rc1 --quiet --format='%(contents:subject)%nTagger: %(taggername) <%(taggeremail)>%nDate: %(taggerdate:format:%c)'"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git show v1.0.0-rc1
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git show v1.0.0-rc1
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git show v1.0.0-rc1 --quiet --format="message: %B%ntagger: %an <%ae>%ndate: %aI"{ "command": "git show v1.0.0-rc1 --quiet --format=\"message: %B%ntagger: %an <%ae>%ndate: %aI\"" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Connection error.
Invalid JSON. Output: git show v1.0.0-rc1
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)