Deinit a submodule
Tests ability to deinitialize a submodule without removing it. Evaluates submodule-deinit command understanding.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'App' > app.txt
  5. 05 git add app.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 mkdir -p ../lib-src && cd ../lib-src && git init && git config user.email 'test@test.com' && git config user.name 'Test User' && echo 'lib code' > lib.py && git add lib.py && git commit -m 'Initial lib'
  8. 08 git clone --bare ../lib-src ../lib-bare
  9. 09 git config protocol.file.allow always
  10. 10 git -c protocol.file.allow=always submodule add ../lib-bare lib
  11. 11 git add .gitmodules lib
  12. 12 git commit -m 'Add lib submodule'
Prompt
Deinitialize the submodule 'lib' (remove its working tree but keep the config). Output ONLY the git command, nothing else.
Expected
git submodule deinit -f lib

Scoped model quality, cost, API time, and token usage for submodule_usage/f008.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 123 in → 166 out (63 reasoning)
``` git submodule deinit lib ```
anthropic/claude-fable-5:high__json_schema PASS 100% 336 in → 119 out (61 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-fable-5:low__json_schema PASS 100% 341 in → 29 out (13 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-fable-5:max__json_schema PASS 100% 341 in → 380 out (87 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-fable-5:medium__json_schema PASS 100% 337 in → 70 out (51 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 339 in → 116 out (56 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-haiku-4.5:high PASS 100% 136 in → 867 out (605 reasoning)
``` git submodule deinit lib ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 298 in → 3,124 out (1,342 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
anthropic/claude-haiku-4.5:low PASS 100% 134 in → 489 out (472 reasoning)
``` git submodule deinit lib ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 298 in → 264 out (233 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-haiku-4.5:medium PASS 100% 134 in → 612 out (516 reasoning)
``` git submodule deinit lib ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 297 in → 401 out (382 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-haiku-4.5:none PASS 100% 104 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 266 in → 16 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 128 in → 547 out (526 reasoning)
``` git submodule deinit lib ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 297 in → 415 out (371 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
anthropic/claude-opus-4.6:high PASS 100% 106 in → 72 out (40 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 273 in → 80 out (47 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.6:low PASS 100% 105 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.6:low__json_schema PASS 100% 273 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.6:max PASS 100% 103 in → 78 out (47 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 269 in → 77 out (45 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.6:medium PASS 100% 106 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 273 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.6:none PASS 100% 107 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 269 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.6:xhigh PASS 100% 107 in → 16 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 268 in → 63 out (27 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.7:high PASS 100% 136 in → 15 out (0 reasoning)
`git submodule deinit lib`
anthropic/claude-opus-4.7:high__json_schema PASS 100% 342 in → 20 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.7:low PASS 100% 134 in → 13 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.7:low__json_schema PASS 100% 340 in → 20 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.7:max PASS 100% 133 in → 13 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.7:max__json_schema PASS 100% 341 in → 20 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.7:medium PASS 100% 135 in → 13 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 345 in → 20 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.7:none PASS 100% 136 in → 15 out (0 reasoning)
`git submodule deinit lib`
anthropic/claude-opus-4.7:none__json_schema PASS 100% 343 in → 20 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.7:xhigh PASS 100% 133 in → 13 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 345 in → 20 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.8:high PASS 100% 126 in → 11 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.8:high__json_schema PASS 100% 336 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.8:low PASS 100% 131 in → 11 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.8:low__json_schema PASS 100% 334 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-opus-4.8:max PASS 100% 125 in → 11 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.8:max__json_schema PASS 100% 332 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.8:medium PASS 100% 130 in → 11 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 341 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-opus-4.8:none PASS 100% 131 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 337 in → 17 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-opus-4.8:xhigh PASS 100% 133 in → 11 out (0 reasoning)
git submodule deinit lib
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 338 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-sonnet-4.6:high PASS 100% 107 in → 71 out (39 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 268 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-4.6:low PASS 100% 108 in → 13 out (0 reasoning)
`git submodule deinit lib`
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 273 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-4.6:max PASS 100% 106 in → 128 out (95 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 269 in → 72 out (40 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-4.6:medium PASS 100% 106 in → 34 out (6 reasoning)
git submodule deinit lib
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 271 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-4.6:none PASS 100% 105 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 267 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 107 in → 71 out (39 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 272 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-5:high PASS 100% 125 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 340 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-sonnet-5:low PASS 100% 132 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-5:low__json_schema PASS 100% 335 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
anthropic/claude-sonnet-5:medium PASS 100% 126 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 332 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-sonnet-5:none PASS 100% 129 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 338 in → 17 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
anthropic/claude-sonnet-5:xhigh PASS 100% 126 in → 15 out (0 reasoning)
``` git submodule deinit lib ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 336 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
arcee-ai/trinity-large-thinking:high PASS 100% 85 in → 316 out (308 reasoning)
git submodule deinit lib
arcee-ai/trinity-large-thinking:low PASS 100% 86 in → 287 out (279 reasoning)
git submodule deinit lib
arcee-ai/trinity-large-thinking:medium PASS 100% 86 in → 407 out (400 reasoning)
git submodule deinit -- lib
arcee-ai/trinity-large-thinking:xhigh PASS 100% 86 in → 458 out (452 reasoning)
git submodule deinit lib
arcee-ai/trinity-mini:high PASS 100% 86 in → 297 out (330 reasoning)
git submodule deinit lib
arcee-ai/trinity-mini:high__json_schema PASS 100% 87 in → 359 out (384 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
arcee-ai/trinity-mini:low PASS 100% 87 in → 337 out (376 reasoning)
git submodule deinit lib
arcee-ai/trinity-mini:low__json_schema PASS 100% 87 in → 262 out (279 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
arcee-ai/trinity-mini:medium PASS 100% 85 in → 894 out (1,056 reasoning)
git submodule deinit lib git rm --cached lib
arcee-ai/trinity-mini:medium__json_schema PASS 100% 89 in → 282 out (299 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
arcee-ai/trinity-mini:xhigh PASS 100% 89 in → 297 out (311 reasoning)
git submodule deinit lib
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 93 in → 339 out (350 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 85 in → 1,402 out (1,522 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 164 in → 2,198 out (2,251 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 88 in → 149 out (141 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 86 in → 132 out (117 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash-0731:max PASS 100% 168 in → 2,401 out (2,394 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 178 in → 10,932 out (11,118 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 90 in → 296 out (314 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 84 in → 541 out (550 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 83 in → 7 out (0 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 85 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 162 in → 3,494 out (3,487 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 82 in → 115 out (107 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash:high PASS 100% 81 in → 87 out (79 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 83 in → 56 out (43 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash:low PASS 100% 83 in → 823 out (815 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 80 in → 76 out (69 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash:medium PASS 100% 85 in → 888 out (879 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 86 in → 90 out (83 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash:none PASS 100% 88 in → 7 out (0 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 91 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 165 in → 69 out (61 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 86 in → 119 out (103 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
deepseek/deepseek-v4-pro:high PASS 100% 85 in → 166 out (157 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-pro:low PASS 100% 88 in → 9 out (0 reasoning)
git submodule deinit -f lib
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 87 in → 238 out (223 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
deepseek/deepseek-v4-pro:medium PASS 100% 82 in → 372 out (365 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 277 in → 141 out (126 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
deepseek/deepseek-v4-pro:none PASS 100% 84 in → 7 out (0 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 89 in → 15 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 87 in → 7 out (0 reasoning)
git submodule deinit lib
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 170 in → 411 out (393 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
google/gemini-3-flash-preview:high PASS 100% 100 in → 150 out (145 reasoning)
git submodule deinit lib
google/gemini-3-flash-preview:high__json_schema PASS 100% 157 in → 237 out (228 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3-flash-preview:low PASS 100% 100 in → 223 out (218 reasoning)
git submodule deinit lib
google/gemini-3-flash-preview:low__json_schema PASS 100% 157 in → 180 out (171 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3-flash-preview:medium PASS 100% 103 in → 166 out (161 reasoning)
git submodule deinit lib
google/gemini-3-flash-preview:medium__json_schema PASS 100% 161 in → 217 out (208 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3-flash-preview:xhigh PASS 100% 96 in → 218 out (213 reasoning)
git submodule deinit lib
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 105 in → 88 out (79 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 100 in → 1,858 out (1,853 reasoning)
git submodule deinit lib
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 98 in → 274 out (264 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 94 in → 127 out (122 reasoning)
git submodule deinit lib
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 97 in → 126 out (111 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemini-3.1-flash-lite-preview:medium PASS 100% 99 in → 119 out (114 reasoning)
git submodule deinit lib
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 97 in → 195 out (185 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 98 in → 1,368 out (1,363 reasoning)
git submodule deinit lib
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 155 in → 266 out (256 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.1-pro-preview:high PASS 100% 103 in → 724 out (719 reasoning)
git submodule deinit lib
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 158 in → 279 out (270 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.1-pro-preview:low PASS 100% 100 in → 349 out (344 reasoning)
git submodule deinit lib
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 156 in → 236 out (226 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.1-pro-preview:medium PASS 100% 95 in → 356 out (351 reasoning)
git submodule deinit lib
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 154 in → 188 out (178 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 98 in → 487 out (482 reasoning)
git submodule deinit lib
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 153 in → 334 out (325 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.5-flash-lite:high PASS 100% 97 in → 675 out (670 reasoning)
git submodule deinit lib
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 156 in → 1,040 out (1,031 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.5-flash-lite:low PASS 100% 97 in → 5 out (0 reasoning)
git submodule deinit lib
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 156 in → 10 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.5-flash-lite:max PASS 100% 102 in → 1,070 out (1,065 reasoning)
git submodule deinit lib
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 154 in → 272 out (263 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.5-flash-lite:medium PASS 100% 101 in → 643 out (638 reasoning)
git submodule deinit lib
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 103 in → 339 out (330 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 100 in → 214 out (209 reasoning)
git submodule deinit lib
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 153 in → 692 out (682 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.5-flash:high PASS 100% 94 in → 790 out (785 reasoning)
git submodule deinit lib
google/gemini-3.5-flash:high__json_schema PASS 100% 155 in → 299 out (289 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.5-flash:low PASS 100% 98 in → 236 out (231 reasoning)
git submodule deinit lib
google/gemini-3.5-flash:low__json_schema PASS 100% 158 in → 62 out (52 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.5-flash:medium PASS 100% 97 in → 427 out (422 reasoning)
git submodule deinit lib
google/gemini-3.5-flash:medium__json_schema PASS 100% 153 in → 225 out (215 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.5-flash:xhigh PASS 100% 97 in → 570 out (565 reasoning)
git submodule deinit lib
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 157 in → 579 out (569 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemini-3.6-flash:low PASS 100% 102 in → 95 out (90 reasoning)
git submodule deinit lib
google/gemini-3.6-flash:low__json_schema PASS 100% 157 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemini-3.6-flash:max__json_schema PASS 100% 102 in → 238 out (229 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemini-3.6-flash:medium PASS 100% 98 in → 1,095 out (1,090 reasoning)
git submodule deinit lib
google/gemini-3.6-flash:medium__json_schema PASS 100% 156 in → 296 out (281 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemini-3.6-flash:xhigh PASS 100% 92 in → 595 out (590 reasoning)
git submodule deinit lib
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 105 in → 344 out (335 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
google/gemma-4-26b-a4b-it:high PASS 100% 118 in → 303 out (309 reasoning)
git submodule deinit lib
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 114 in → 343 out (333 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-26b-a4b-it:low PASS 100% 114 in → 238 out (213 reasoning)
git submodule deinit lib
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 113 in → 800 out (796 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-26b-a4b-it:medium PASS 100% 116 in → 252 out (215 reasoning)
git submodule deinit lib
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 117 in → 231 out (180 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-26b-a4b-it:none PASS 100% 112 in → 6 out (0 reasoning)
git submodule deinit lib
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 113 in → 16 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 115 in → 146 out (136 reasoning)
git submodule deinit lib
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 113 in → 159 out (143 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-31b-it:high PASS 100% 117 in → 148 out (146 reasoning)
git submodule deinit lib
google/gemma-4-31b-it:high__json_schema PASS 100% 113 in → 544 out (520 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-31b-it:low PASS 100% 112 in → 126 out (120 reasoning)
git submodule deinit lib
google/gemma-4-31b-it:low__json_schema PASS 100% 115 in → 125 out (117 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemma-4-31b-it:medium PASS 100% 114 in → 144 out (142 reasoning)
git submodule deinit lib
google/gemma-4-31b-it:medium__json_schema PASS 100% 115 in → 192 out (185 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
google/gemma-4-31b-it:none__json_schema PASS 100% 111 in → 16 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
google/gemma-4-31b-it:xhigh PASS 100% 113 in → 167 out (160 reasoning)
git submodule deinit lib
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 115 in → 180 out (171 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
ibm-granite/granite-4.1-8b PASS 100% 95 in → 8 out
git submodule deinit --force lib
inclusionai/ling-2.6-flash PASS 100% 119 in → 12 out
``` git submodule deinit -- lib ```
inclusionai/ling-2.6-flash__json_schema PASS 100% 120 in → 16 out
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
liquid/lfm-2-24b-a2b PASS 100% 94 in → 9 out
git submodule deinit -f lib
minimax/minimax-m2.5:high PASS 100% 119 in → 1,603 out (1,596 reasoning)
git submodule deinit lib
minimax/minimax-m2.5:high__json_schema PASS 100% 121 in → 548 out (613 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m2.5:low PASS 100% 130 in → 458 out (451 reasoning)
git submodule deinit lib
minimax/minimax-m2.5:low__json_schema PASS 100% 114 in → 406 out (392 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m2.5:medium PASS 100% 126 in → 721 out (714 reasoning)
git submodule deinit lib
minimax/minimax-m2.5:medium__json_schema PASS 100% 113 in → 146 out (132 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m2.5:xhigh PASS 100% 119 in → 520 out (591 reasoning)
git submodule deinit lib
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 115 in → 219 out (205 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m2.7:high PASS 100% 102 in → 79 out (90 reasoning)
git submodule deinit lib
minimax/minimax-m2.7:low PASS 100% 113 in → 827 out (820 reasoning)
git submodule deinit lib
minimax/minimax-m2.7:low__json_schema PASS 100% 116 in → 14 out (14 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m2.7:medium PASS 100% 116 in → 2,237 out (2,564 reasoning)
git submodule deinit lib
minimax/minimax-m2.7:medium__json_schema PASS 100% 248 in → 286 out (271 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m2.7:xhigh PASS 100% 118 in → 1,154 out (1,146 reasoning)
git submodule deinit lib
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 250 in → 154 out (139 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m3:high PASS 100% 250 in → 79 out (83 reasoning)
git submodule deinit lib
minimax/minimax-m3:high__json_schema PASS 100% 244 in → 11 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
minimax/minimax-m3:low PASS 100% 256 in → 30 out (28 reasoning)
git submodule deinit lib
minimax/minimax-m3:low__json_schema PASS 100% 243 in → 11 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
minimax/minimax-m3:medium PASS 100% 255 in → 82 out (95 reasoning)
git submodule deinit lib
minimax/minimax-m3:medium__json_schema PASS 100% 241 in → 14 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
minimax/minimax-m3:xhigh PASS 100% 252 in → 43 out (47 reasoning)
git submodule deinit lib
minimax/minimax-m3:xhigh__json_schema PASS 100% 247 in → 14 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
mistralai/devstral-2512 PASS 100% 95 in → 14 out
```sh git submodule deinit -f lib ```
mistralai/devstral-2512__json_schema PASS 100% 96 in → 14 out
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
mistralai/mistral-medium-3-5:high PASS 100% 108 in → 808 out (870 reasoning)
git submodule deinit lib
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 93 in → 274 out (295 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
mistralai/mistral-medium-3-5:low PASS 100% 106 in → 840 out (908 reasoning)
git submodule deinit lib
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 97 in → 409 out (435 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
mistralai/mistral-medium-3-5:medium PASS 100% 109 in → 555 out (587 reasoning)
git submodule deinit lib
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 96 in → 371 out (411 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
mistralai/mistral-medium-3-5:none PASS 100% 111 in → 9 out (0 reasoning)
git Submodule deinit -f lib
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 98 in → 14 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 93 in → 288 out (305 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
moonshotai/kimi-k2.6:high PASS 100% 91 in → 579 out (572 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.6:low PASS 100% 86 in → 556 out (592 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.6:low__json_schema PASS 100% 83 in → 561 out (546 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
moonshotai/kimi-k2.6:medium PASS 100% 83 in → 745 out (739 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 83 in → 411 out (397 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
moonshotai/kimi-k2.6:xhigh PASS 100% 86 in → 1,293 out (1,300 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 88 in → 365 out (411 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
moonshotai/kimi-k2.7-code:high PASS 100% 87 in → 417 out (410 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 167 in → 343 out (332 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
moonshotai/kimi-k2.7-code:low PASS 100% 81 in → 307 out (300 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 84 in → 731 out (719 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
moonshotai/kimi-k2.7-code:medium PASS 100% 83 in → 97 out (114 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 167 in → 438 out (427 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
moonshotai/kimi-k2.7-code:xhigh PASS 100% 84 in → 344 out (337 reasoning)
git submodule deinit lib
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 86 in → 343 out (384 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
moonshotai/kimi-k3:high PASS 100% 160 in → 61 out (41 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:high__json_schema PASS 100% 163 in → 358 out (338 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:low PASS 100% 161 in → 72 out (52 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:low__json_schema PASS 100% 163 in → 53 out (33 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:max PASS 100% 162 in → 283 out (263 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:max__json_schema PASS 100% 261 in → 84 out (60 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
moonshotai/kimi-k3:medium PASS 100% 164 in → 85 out (65 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:medium__json_schema PASS 100% 157 in → 89 out (69 reasoning)
git submodule deinit lib
moonshotai/kimi-k3:xhigh PASS 100% 166 in → 653 out (630 reasoning)
``` git submodule deinit lib ```
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 163 in → 150 out (130 reasoning)
git submodule deinit lib
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 110 in → 1,789 out (1,956 reasoning)
git submodule deinit -f --quiet lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f --quiet lib" }
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 106 in → 1,834 out (1,962 reasoning)
git submodule deinit --force lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit --force lib" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 109 in → 1,632 out (1,793 reasoning)
`git submodule deinit -f lib`
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 109 in → 25 out (0 reasoning)
git submodule deinit -f lib && git rm -f --cached lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib && git rm -f --cached lib" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 110 in → 1,068 out (1,157 reasoning)
git submodule deinit -f lib
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 109 in → 1,969 out (2,112 reasoning)
git submodule deinit -f -- lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f -- lib" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 109 in → 122 out (104 reasoning)
git submodule deinit -f lib
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 110 in → 126 out (103 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 110 in → 221 out (234 reasoning)
git submodule deinit -f lib
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 109 in → 95 out (73 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 107 in → 340 out (383 reasoning)
git submodule deinit -f lib
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 108 in → 380 out (345 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 103 in → 9 out (0 reasoning)
git submodule deinit -f lib
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 109 in → 15 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 109 in → 106 out (98 reasoning)
git submodule deinit -f lib
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 109 in → 504 out (468 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
openai/gpt-5.4-mini:high PASS 100% 85 in → 80 out (66 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-mini:high__json_schema PASS 100% 120 in → 304 out (284 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.4-mini:low PASS 100% 81 in → 48 out (34 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-mini:low__json_schema PASS 100% 122 in → 49 out (27 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-mini:medium PASS 100% 90 in → 51 out (37 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-mini:medium__json_schema PASS 100% 122 in → 136 out (114 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-mini:none PASS 100% 82 in → 12 out (0 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-mini:none__json_schema PASS 100% 119 in → 26 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-mini:xhigh PASS 100% 84 in → 4,745 out (4,733 reasoning)
git submodule deinit lib
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 117 in → 536 out (516 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.4-nano:high__json_schema PASS 100% 118 in → 85 out (63 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-nano:low PASS 100% 87 in → 12 out (0 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-nano:low__json_schema PASS 100% 117 in → 20 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-nano:medium PASS 100% 83 in → 12 out (0 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-nano:medium__json_schema PASS 100% 120 in → 20 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-nano:none__json_schema PASS 100% 118 in → 20 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4-nano:xhigh PASS 100% 83 in → 2,557 out (2,543 reasoning)
git submodule deinit -f lib
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 117 in → 3,665 out (3,643 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-5.4:high PASS 100% 86 in → 415 out (403 reasoning)
git submodule deinit lib
openai/gpt-5.4:high__json_schema PASS 100% 122 in → 245 out (225 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.4:low PASS 100% 92 in → 44 out (32 reasoning)
git submodule deinit lib
openai/gpt-5.4:low__json_schema PASS 100% 117 in → 46 out (26 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.4:medium PASS 100% 85 in → 160 out (148 reasoning)
git submodule deinit lib
openai/gpt-5.4:medium__json_schema PASS 100% 120 in → 73 out (53 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.4:none PASS 100% 85 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.4:none__json_schema PASS 100% 115 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.4:xhigh PASS 100% 84 in → 470 out (458 reasoning)
git submodule deinit lib
openai/gpt-5.4:xhigh__json_schema PASS 100% 120 in → 3,127 out (3,106 reasoning)
git submodule deinit -- lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -- lib"}
openai/gpt-5.5:high PASS 100% 87 in → 366 out (354 reasoning)
git submodule deinit lib
openai/gpt-5.5:high__json_schema PASS 100% 119 in → 460 out (440 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.5:low PASS 100% 83 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.5:low__json_schema PASS 100% 118 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.5:medium PASS 100% 85 in → 392 out (379 reasoning)
git submodule deinit -- lib
openai/gpt-5.5:medium__json_schema PASS 100% 117 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.5:none PASS 100% 83 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.5:none__json_schema PASS 100% 117 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.5:xhigh PASS 100% 83 in → 528 out (516 reasoning)
git submodule deinit lib
openai/gpt-5.5:xhigh__json_schema PASS 100% 119 in → 536 out (516 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-luna:high PASS 100% 84 in → 528 out (516 reasoning)
git submodule deinit lib
openai/gpt-5.6-luna:high__json_schema PASS 100% 118 in → 537 out (516 reasoning)
git submodule deinit -- lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -- lib"}
openai/gpt-5.6-luna:low PASS 100% 92 in → 305 out (290 reasoning)
git submodule deinit -f -- lib
openai/gpt-5.6-luna:low__json_schema PASS 100% 121 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-luna:max PASS 100% 84 in → 528 out (516 reasoning)
git submodule deinit lib
openai/gpt-5.6-luna:max__json_schema PASS 100% 120 in → 1,575 out (1,552 reasoning)
git submodule deinit --force -- lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit --force -- lib"}
openai/gpt-5.6-luna:medium PASS 100% 86 in → 53 out (41 reasoning)
git submodule deinit lib
openai/gpt-5.6-luna:medium__json_schema PASS 100% 113 in → 44 out (24 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-luna:none PASS 100% 84 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-luna:none__json_schema PASS 100% 125 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-luna:xhigh PASS 100% 83 in → 994 out (980 reasoning)
git submodule deinit --force lib
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 120 in → 373 out (352 reasoning)
git submodule deinit -- lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -- lib"}
openai/gpt-5.6-sol:high PASS 100% 85 in → 168 out (156 reasoning)
git submodule deinit lib
openai/gpt-5.6-sol:high__json_schema PASS 100% 121 in → 355 out (335 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-sol:low PASS 100% 86 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-sol:low__json_schema PASS 100% 122 in → 237 out (214 reasoning)
git submodule deinit -f -- lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f -- lib"}
openai/gpt-5.6-sol:max PASS 100% 83 in → 715 out (703 reasoning)
git submodule deinit lib
openai/gpt-5.6-sol:max__json_schema PASS 100% 121 in → 536 out (516 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-sol:medium PASS 100% 86 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-sol:medium__json_schema PASS 100% 117 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-sol:none PASS 100% 85 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-sol:none__json_schema PASS 100% 122 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-sol:xhigh PASS 100% 86 in → 389 out (377 reasoning)
git submodule deinit lib
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 118 in → 224 out (204 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-terra:high PASS 100% 83 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-terra:high__json_schema PASS 100% 117 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-terra:low PASS 100% 87 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-terra:low__json_schema PASS 100% 115 in → 111 out (91 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-terra:max PASS 100% 81 in → 145 out (133 reasoning)
git submodule deinit lib
openai/gpt-5.6-terra:max__json_schema PASS 100% 120 in → 93 out (73 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-terra:medium PASS 100% 88 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-terra:medium__json_schema PASS 100% 116 in → 93 out (73 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-terra:none PASS 100% 83 in → 10 out (0 reasoning)
git submodule deinit lib
openai/gpt-5.6-terra:none__json_schema PASS 100% 124 in → 18 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-5.6-terra:xhigh PASS 100% 85 in → 242 out (230 reasoning)
git submodule deinit lib
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 121 in → 178 out (158 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-oss-120b:high PASS 100% 148 in → 176 out (168 reasoning)
git submodule deinit -f lib
openai/gpt-oss-120b:high__json_schema PASS 100% 144 in → 1,603 out (1,507 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-oss-120b:low PASS 100% 147 in → 120 out (109 reasoning)
git submodule deinit -f lib
openai/gpt-oss-120b:low__json_schema PASS 100% 146 in → 51 out (34 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-oss-120b:medium PASS 100% 136 in → 155 out (146 reasoning)
git submodule deinit -f lib
openai/gpt-oss-120b:medium__json_schema PASS 100% 140 in → 196 out (194 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-oss-120b:xhigh PASS 100% 145 in → 593 out (632 reasoning)
git submodule deinit lib
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 145 in → 346 out (355 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
openai/gpt-oss-20b:high PASS 100% 147 in → 4,534 out (4,931 reasoning)
git submodule deinit -f lib
openai/gpt-oss-20b:low PASS 100% 145 in → 52 out (38 reasoning)
git submodule deinit -f lib
openai/gpt-oss-20b:low__json_schema PASS 100% 151 in → 170 out (142 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-oss-20b:medium PASS 100% 142 in → 560 out (604 reasoning)
git submodule deinit lib
openai/gpt-oss-20b:medium__json_schema PASS 100% 150 in → 152 out (138 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
openai/gpt-oss-20b:xhigh PASS 100% 145 in → 1,832 out (1,974 reasoning)
git submodule deinit -f -- lib
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 132 in → 9,401 out (10,494 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
poolside/laguna-m.1:high PASS 100% 108 in → 252 out (244 reasoning)
git submodule deinit lib
poolside/laguna-m.1:high__json_schema PASS 100% 109 in → 350 out (334 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
poolside/laguna-m.1:low PASS 100% 108 in → 440 out (430 reasoning)
git submodule deinit -f lib
poolside/laguna-m.1:low__json_schema PASS 100% 106 in → 800 out (782 reasoning)
git submodule deinit --force lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit --force lib" }
poolside/laguna-m.1:medium PASS 100% 110 in → 284 out (274 reasoning)
git submodule deinit -f lib
poolside/laguna-m.1:medium__json_schema PASS 100% 109 in → 647 out (635 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
poolside/laguna-m.1:none PASS 100% 107 in → 7 out (0 reasoning)
git submodule deinit lib
poolside/laguna-m.1:none__json_schema PASS 100% 105 in → 11 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
poolside/laguna-m.1:xhigh PASS 100% 105 in → 354 out (346 reasoning)
git submodule deinit lib
poolside/laguna-m.1:xhigh__json_schema PASS 100% 111 in → 906 out (894 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
poolside/laguna-xs-2.1:high PASS 100% 102 in → 452 out (446 reasoning)
git submodule deinit lib
poolside/laguna-xs-2.1:high__json_schema PASS 100% 106 in → 208 out (189 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
poolside/laguna-xs-2.1:low PASS 100% 103 in → 652 out (646 reasoning)
git submodule deinit lib
poolside/laguna-xs-2.1:low__json_schema PASS 100% 107 in → 423 out (406 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
poolside/laguna-xs-2.1:medium PASS 100% 108 in → 296 out (288 reasoning)
git submodule deinit -f lib
poolside/laguna-xs-2.1:none PASS 100% 106 in → 7 out (0 reasoning)
git submodule deinit -f lib
poolside/laguna-xs-2.1:none__json_schema PASS 100% 108 in → 18 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
poolside/laguna-xs-2.1:xhigh PASS 100% 108 in → 444 out (438 reasoning)
git submodule deinit lib
poolside/laguna-xs.2:low PASS 100% 140 in → 477 out (469 reasoning)
git submodule deinit lib
poolside/laguna-xs.2:low__json_schema PASS 100% 143 in → 167 out (150 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-27b:high PASS 100% 106 in → 1,365 out (1,384 reasoning)
git submodule deinit --force lib
qwen/qwen3.6-27b:medium PASS 100% 102 in → 988 out (1,034 reasoning)
git submodule deinit lib
qwen/qwen3.6-27b:medium__json_schema PASS 100% 105 in → 1,814 out (1 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-27b:none PASS 100% 107 in → 8 out (0 reasoning)
git submodule deinit --force lib
qwen/qwen3.6-27b:none__json_schema PASS 100% 106 in → 13 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-27b:xhigh PASS 100% 108 in → 1,253 out (1,240 reasoning)
git submodule deinit -- lib
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 103 in → 974 out (995 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-35b-a3b:high PASS 100% 104 in → 1,275 out (1,308 reasoning)
git submodule deinit lib
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 104 in → 1,276 out (1,245 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-35b-a3b:low PASS 100% 103 in → 1,339 out (1,410 reasoning)
git submodule deinit lib
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 99 in → 2,187 out (2,221 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 105 in → 1,120 out (1,152 reasoning)
git submodule deinit lib
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 102 in → 1,104 out (1,033 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
qwen/qwen3.6-35b-a3b:none PASS 100% 102 in → 6 out (0 reasoning)
git submodule deinit lib
qwen/qwen3.6-35b-a3b:none__json_schema PASS 100% 104 in → 19 out (0 reasoning)
git submodule deinit -f -- lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f -- lib" }
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 102 in → 1,797 out (1,787 reasoning)
git submodule deinit lib
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 105 in → 1,428 out (1,468 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
qwen/qwen3.6-flash:high PASS 100% 104 in → 1,035 out (1,025 reasoning)
git submodule deinit lib
qwen/qwen3.6-flash:low PASS 100% 107 in → 2,458 out (2,446 reasoning)
git submodule deinit -f lib
qwen/qwen3.6-flash:medium PASS 100% 100 in → 689 out (679 reasoning)
git submodule deinit lib
qwen/qwen3.6-flash:none PASS 100% 108 in → 5 out (0 reasoning)
git submodule deinit lib
qwen/qwen3.6-flash:xhigh PASS 100% 101 in → 824 out (814 reasoning)
git submodule deinit lib
qwen/qwen3.7-flash:high PASS 100% 104 in → 1,158 out (1,148 reasoning)
git submodule deinit lib
qwen/qwen3.7-flash:low PASS 100% 101 in → 1,525 out (1,515 reasoning)
git submodule deinit lib
qwen/qwen3.7-flash:max PASS 100% 101 in → 2,716 out (2,706 reasoning)
git submodule deinit lib
qwen/qwen3.7-flash:medium PASS 100% 102 in → 1,013 out (1,003 reasoning)
git submodule deinit lib
qwen/qwen3.7-flash:none PASS 100% 105 in → 5 out (0 reasoning)
git submodule deinit lib
qwen/qwen3.7-flash:xhigh PASS 100% 103 in → 2,528 out (2,518 reasoning)
git submodule deinit lib
qwen/qwen3.7-max:high PASS 100% 104 in → 794 out (784 reasoning)
git submodule deinit lib
qwen/qwen3.7-max:low PASS 100% 106 in → 969 out (959 reasoning)
git submodule deinit lib
qwen/qwen3.7-max:medium PASS 100% 105 in → 476 out (466 reasoning)
git submodule deinit lib
qwen/qwen3.7-max:none PASS 100% 109 in → 5 out (0 reasoning)
git submodule deinit lib
qwen/qwen3.7-max:xhigh PASS 100% 104 in → 342 out (332 reasoning)
git submodule deinit lib
qwen/qwen3.7-plus:high PASS 100% 105 in → 1,006 out (996 reasoning)
git submodule deinit lib
qwen/qwen3.7-plus:low PASS 100% 102 in → 561 out (551 reasoning)
git submodule deinit lib
qwen/qwen3.7-plus:medium PASS 100% 107 in → 1,218 out (1,208 reasoning)
git submodule deinit lib
qwen/qwen3.7-plus:none PASS 100% 104 in → 5 out (0 reasoning)
git submodule deinit lib
qwen/qwen3.7-plus:xhigh PASS 100% 102 in → 653 out (643 reasoning)
git submodule deinit lib
tencent/hy3:high PASS 100% 95 in → 1,170 out (1,229 reasoning)
git submodule deinit lib
tencent/hy3:high__json_schema PASS 100% 92 in → 584 out (627 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
tencent/hy3:low PASS 100% 91 in → 751 out (744 reasoning)
git submodule deinit lib
tencent/hy3:low__json_schema PASS 100% 93 in → 454 out (467 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
tencent/hy3:medium PASS 100% 93 in → 555 out (548 reasoning)
git submodule deinit lib
tencent/hy3:medium__json_schema PASS 100% 94 in → 665 out (696 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
tencent/hy3:none__json_schema PASS 100% 97 in → 14 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
tencent/hy3:xhigh PASS 100% 90 in → 466 out (459 reasoning)
git submodule deinit lib
tencent/hy3:xhigh__json_schema PASS 100% 92 in → 1,043 out (1,118 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
thinkingmachines/inkling-small:high PASS 100% 100 in → 237 out (244 reasoning)
git submodule deinit lib
thinkingmachines/inkling-small:high__json_schema PASS 100% 96 in → 266 out (248 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
thinkingmachines/inkling-small:low PASS 100% 89 in → 83 out (77 reasoning)
git submodule deinit lib
thinkingmachines/inkling-small:low__json_schema PASS 100% 90 in → 77 out (59 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
thinkingmachines/inkling-small:max PASS 100% 98 in → 5,219 out (5,207 reasoning)
git submodule deinit lib
thinkingmachines/inkling-small:max__json_schema PASS 100% 96 in → 5,208 out (5,190 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
thinkingmachines/inkling-small:medium PASS 100% 93 in → 495 out (522 reasoning)
git submodule deinit lib
thinkingmachines/inkling-small:medium__json_schema PASS 100% 95 in → 887 out (869 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
thinkingmachines/inkling-small:none PASS 100% 92 in → 9 out (0 reasoning)
git submodule deinit lib
thinkingmachines/inkling-small:xhigh PASS 100% 95 in → 1,521 out (1,509 reasoning)
git submodule deinit lib
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 90 in → 1,421 out (1,403 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
thinkingmachines/inkling:high PASS 100% 91 in → 426 out (448 reasoning)
git submodule deinit lib
thinkingmachines/inkling:low PASS 100% 97 in → 67 out (54 reasoning)
git submodule deinit lib
thinkingmachines/inkling:max PASS 100% 90 in → 127 out (113 reasoning)
git submodule deinit lib
thinkingmachines/inkling:max__json_schema PASS 100% 95 in → 111 out (95 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit lib"}
thinkingmachines/inkling:medium PASS 100% 89 in → 188 out (175 reasoning)
git submodule deinit lib
thinkingmachines/inkling:xhigh PASS 100% 92 in → 207 out (194 reasoning)
git submodule deinit lib
thinkingmachines/inkling:xhigh__json_schema PASS 100% 93 in → 319 out (301 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
x-ai/grok-4.3:high PASS 100% 265 in → 380 out (375 reasoning)
git submodule deinit lib
x-ai/grok-4.3:high__json_schema PASS 100% 320 in → 328 out (315 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
x-ai/grok-4.3:low PASS 100% 270 in → 301 out (296 reasoning)
git submodule deinit lib
x-ai/grok-4.3:low__json_schema PASS 100% 323 in → 286 out (273 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
x-ai/grok-4.3:max PASS 100% 264 in → 260 out (255 reasoning)
git submodule deinit lib
x-ai/grok-4.3:max__json_schema PASS 100% 318 in → 291 out (279 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
x-ai/grok-4.3:medium PASS 100% 266 in → 119 out (114 reasoning)
git submodule deinit lib
x-ai/grok-4.3:medium__json_schema PASS 100% 322 in → 374 out (361 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
x-ai/grok-4.3:none PASS 100% 261 in → 5 out (0 reasoning)
git submodule deinit lib
x-ai/grok-4.3:none__json_schema PASS 100% 320 in → 13 out (0 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
x-ai/grok-4.3:xhigh PASS 100% 261 in → 127 out (120 reasoning)
`git submodule deinit lib`
x-ai/grok-4.3:xhigh__json_schema PASS 100% 318 in → 865 out (856 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
x-ai/grok-4.5:high PASS 100% 286 in → 1,592 out (1,587 reasoning)
git submodule deinit lib
x-ai/grok-4.5:high__json_schema PASS 100% 353 in → 457 out (447 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
x-ai/grok-4.5:low PASS 100% 284 in → 370 out (361 reasoning)
``` git submodule deinit lib ```
x-ai/grok-4.5:low__json_schema PASS 100% 358 in → 381 out (371 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
x-ai/grok-4.5:max PASS 100% 289 in → 523 out (514 reasoning)
``` git submodule deinit lib ```
x-ai/grok-4.5:max__json_schema PASS 100% 350 in → 1,164 out (1,154 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
x-ai/grok-4.5:medium PASS 100% 285 in → 501 out (496 reasoning)
git submodule deinit lib
x-ai/grok-4.5:medium__json_schema PASS 100% 356 in → 1,243 out (1,233 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
x-ai/grok-4.5:xhigh PASS 100% 288 in → 822 out (813 reasoning)
``` git submodule deinit lib ```
x-ai/grok-4.5:xhigh__json_schema PASS 100% 353 in → 539 out (529 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit lib"}
z-ai/glm-4.7-flash:high PASS 100% 85 in → 1,188 out (1,179 reasoning)
git submodule deinit -f lib
z-ai/glm-4.7-flash:high__json_schema PASS 100% 84 in → 1,748 out (1,808 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
z-ai/glm-4.7-flash:low PASS 100% 87 in → 516 out (551 reasoning)
git submodule deinit lib
z-ai/glm-4.7-flash:medium PASS 100% 87 in → 524 out (538 reasoning)
git submodule deinit -f lib
z-ai/glm-4.7-flash:none PASS 100% 90 in → 8 out (0 reasoning)
git submodule deinit -f lib
z-ai/glm-4.7-flash:none__json_schema PASS 100% 85 in → 13 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit -f lib"}
z-ai/glm-4.7-flash:xhigh PASS 100% 86 in → 455 out (472 reasoning)
git submodule deinit -f lib
z-ai/glm-5.2:high PASS 100% 93 in → 296 out (296 reasoning)
git submodule deinit lib
z-ai/glm-5.2:high__json_schema PASS 100% 95 in → 255 out (249 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
z-ai/glm-5.2:low PASS 100% 90 in → 303 out (296 reasoning)
git submodule deinit lib
z-ai/glm-5.2:low__json_schema PASS 100% 266 in → 265 out (251 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit -f lib"}
z-ai/glm-5.2:medium PASS 100% 93 in → 469 out (0 reasoning)
git submodule deinit lib
z-ai/glm-5.2:none PASS 100% 90 in → 6 out (0 reasoning)
git submodule deinit lib
z-ai/glm-5.2:none__json_schema PASS 100% 90 in → 16 out (0 reasoning)
git submodule deinit -f lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f lib" }
z-ai/glm-5.2:xhigh PASS 100% 95 in → 794 out (872 reasoning)
git submodule deinit lib
z-ai/glm-5.2:xhigh__json_schema PASS 100% 92 in → 897 out (883 reasoning)
git submodule deinit lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib" }
openai/gpt-5.4-nano:high FAIL 66.7% 84 in → 14 out (0 reasoning)
git rm --cached lib && rm -rf lib
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
openai/gpt-5.4-nano:none FAIL 66.7% 83 in → 14 out (0 reasoning)
git rm --cached lib && rm -rf lib
Failure: Failed assertions: [{'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
anthropic/claude-fable-5:low FAIL 33.3% 125 in → 43 out (6 reasoning)
(empty output)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
anthropic/claude-fable-5:max FAIL 33.3% 128 in → 6 out (0 reasoning)
(empty output)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
anthropic/claude-fable-5:medium FAIL 33.3% 127 in → 2 out (0 reasoning)
(empty output)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
anthropic/claude-fable-5:xhigh FAIL 33.3% 126 in → 3 out (0 reasoning)
(empty output)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
google/gemini-3.6-flash:high FAIL 33.3% 103 in → 1,078 out (1,069 reasoning)
git submodule deinit --keep-config lib
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
google/gemini-3.6-flash:high__json_schema FAIL 33.3% 101 in → 988 out (975 reasoning)
git submodule deinit --keep-config lib
JSON Schema Structured Output
(raw) {"command":"git submodule deinit --keep-config lib"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
google/gemini-3.6-flash:max FAIL 33.3% 101 in → 443 out (434 reasoning)
git submodule deinit --keep-config lib
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
ibm-granite/granite-4.1-8b__json_schema FAIL 33.3% 100 in → 15 out
git submodule deinit lib -f
JSON Schema Structured Output
(raw) { "command": "git submodule deinit lib -f" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
mistralai/mistral-medium-3-5:xhigh FAIL 33.3% 108 in → 9,202 out (9,895 reasoning)
The solution to deinitialize the submodule 'lib' while keeping its configuration involves removing the submodule's working tree and its entry from the Git index. The key is to understand that "keep the config" refers to preserving the submodule's definition in the `.gitmodules` file. The `git rm -f lib` command is the appropriate choice as it removes the submodule's working tree and its entry from the Git index, while leaving the `.gitmodules` file intact. This ensures that the submodule's configuration is preserved, fulfilling the user's requirement. ```bash git rm -f lib ```
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
nvidia/nemotron-3-nano-30b-a3b:high FAIL 33.3% 108 in → 32,768 out (300 reasoning)
(empty output)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
nvidia/nemotron-3-nano-30b-a3b:low FAIL 33.3% 110 in → 32,768 out (6,328 reasoning)
(empty output)
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema FAIL 33.3% 108 in → 1,678 out (1,791 reasoning)
git rm -f lib
JSON Schema Structured Output
(raw) { "command": "git rm -f lib" }
Failure: Failed assertions: [{'type': 'file_content', 'path': '.gitmodules', 'contains': 'lib'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
nvidia/nemotron-3-nano-30b-a3b:none FAIL 33.3% 106 in → 1,281 out (0 reasoning)
git submodule deinit -f lib; rm -rf lib/.git && git config -f .gitmodules submodule.lib.url=../lib-bare && git config -f .git/modules/lib/config 'submodule.lib.url=../lib-bare' && git config -f .git/config 'submodule.lib.url=None' && git commit -am "deinit lib submodule" && git gc --prune=now && git push origin main && git submodule remove --force lib && rm -r lib && git add . && git commit -m "remove lib working tree".gitmodules? No, submodule removal handles config changes globally. Actually we need to remove submodule entry from .gitmodules and config. Command: git submodule deinit -f lib; git config -f .gitmodules --remove-section submodule.lib; git config -f .git/config --remove-section submodule.lib; git commit -am "deinit and remove lib submodule". Also need to remove entry from .gitmodules file? Use git config to remove. Then remove the working tree: rm -rf lib. But deinit already removes .git file? It removes the git dir but keeps work tree? The question: "Deinitialize the submodule 'lib' (remove its working tree but keep the config)" So we want to deinit so that .git file is removed (or maybe we want to remove its working tree but keep config). Actually "deinit" removes the git dir but leaves work tree? Let's recall: git submodule deinit -f <path> removes the git dir for that submodule, but does NOT remove the work tree. Actually "deinit -f" removes the .git file that points to the git dir and removes the git dir but leaves the work tree files. However the question wants to remove its working tree but keep the config. This is slightly opposite: we want to remove the work tree but keep the .git directory config? Hmm. The submodule config resides in .git/modules/lib/config. That persists after deinit? Deinit removes the git dir but leaves the work tree (source files). The config file remains? Actually .git/modules/<path>/config may be removed? Not sure. Better answer: Use git submodule deinit -f lib; rm -rf lib/*; git config -f .gitmodules ...? Actually the config is in .git/modules/lib/config; removing working tree (the checkout) doesn't delete that config. The config is in the git dir; deinit may keep it? Let's think: deinit removes .git file and the git directory under that path? It removes it entirely? Actually "git submodule deinit" removes the git directory; it does not remove the submodule directory itself, just the .git file that points to the git dir. So the config remains? Might be removed along with .git dir. But we want to keep config - maybe it's stored in .gitmodules and .git/config. The config is mostly in .gitmodules and top-level .git/config; the actual per-submodule config (.git/modules/lib/config) is separate but can be removed. The question: "remove its working tree but keep the config". That suggests we delete the checked-out files (work tree) but leave the .git subdirectory (and its config). To do that, we can deinit? Actually deinit removes the git dir but not the work tree. To keep the git thing but remove work tree, we could delete the checkout directory manually while preserving .git modules config? Actually the config for that submodule is stored in .git/modules/lib/config; that persists when you remove the working tree (i.e., delete the contents of lib directory). The config file resides outside lib; it's inside .git/modules/lib/. So after deinit, the .git/.gitmodules config remains but the submodule's git dir may be removed; but we want to keep config. Simpler: Delete the working tree directory lib (i.e., rm -rf lib/*) but keep the git dir .git/modules/lib (i.e., config). However deinit also removes .git file pointing to it, which is in .git/modules? Actually the submodule record is stored as a gitlink in the index, and a .git file that points to .git/modules/lib. Deinit removes that .git file and the git dir. So config would be removed. But maybe they want to deinit then remove the work tree, leaving the config (i.e., .gitmodule and .git/config entries) intact. Thus answer likely: git submodule deinit -f lib; git rm --cached lib; rm -rf lib; git commit -am "deinit lib". But they ask "Output ONLY the git command". Maybe they want: git submodule deinit -f lib && rm -rf lib && git rm --cached lib? Hmm. Probably the command: git submodule deinit -f lib; rm -rf lib && git config -f .gitmodules --remove-section submodule.lib && git commit -am "deinit lib submodule". But we need just one command? Possibly "git submodule deinit -f lib && rm -rf lib && git config -f .gitmodules --remove-section submodule.lib && git commit -am 'deinit lib'". Output only the command. Thus final answer: git submodule deinit -f lib && rm -rf lib && git config -f .gitmodules --remove-section submodule.lib && git commit -am "deinit lib submodule". But that's multiple commands separated by &&. They said "Output ONLY the git command". Could be a series separated by &&. We'll output a single line containing multiple commands. I'll output: git submodule deinit -f lib && rm -rf lib && git config -f .gitmodules --remove-section submodule.lib && git commit -am "deinit and remove lib working tree". git submodule deinit -f lib && rm -rf lib && git config -f .gitmodules --remove-section submodule.lib && git commit -am "deinit and remove lib working tree"
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
openai/gpt-oss-20b:high__json_schema FAIL 33.3% 145 in → 63 out (0 reasoning)
git submodule deinit -f --all --recursive --work-tree lib????????????????--????????.???????????? الحلقة..??…. ??……..Content.....…….. let's possibly......
JSON Schema Structured Output
(raw) { "command":"git submodule deinit -f --all --recursive --work-tree lib????????????????--????????.???????????? الحلقة..??…. ??……..Content.....…….. let's possibly......" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs-2.1:medium__json_schema FAIL 33.3% 110 in → 617 out (603 reasoning)
git submodule deinit --keep lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinit --keep lib"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs-2.1:xhigh__json_schema FAIL 33.3% 107 in → 305 out (284 reasoning)
git submodule deinit -f --keep lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinit -f --keep lib" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:high FAIL 33.3% 143 in → 129 out (122 reasoning)
git deinit lib
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:high__json_schema FAIL 33.3% 147 in → 75 out (55 reasoning)
git submodule deinitialize --keep-active lib
JSON Schema Structured Output
(raw) { "command": "git submodule deinitialize --keep-active lib" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:medium FAIL 33.3% 145 in → 114 out (103 reasoning)
git submodule deinit --keep-config lib
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:medium__json_schema FAIL 33.3% 142 in → 90 out (74 reasoning)
git deinit lib
JSON Schema Structured Output
(raw) { "command": "git deinit lib" }
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:none FAIL 33.3% 145 in → 6 out (0 reasoning)
git deinit lib
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:none__json_schema FAIL 33.3% 144 in → 10 out (0 reasoning)
git deinit lib
JSON Schema Structured Output
(raw) {"command": "git deinit lib"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:xhigh FAIL 33.3% 145 in → 355 out (344 reasoning)
git submodule deinit --keep-conf lib
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
poolside/laguna-xs.2:xhigh__json_schema FAIL 33.3% 146 in → 89 out (74 reasoning)
git submodule deinitialize --keep-active lib
JSON Schema Structured Output
(raw) {"command": "git submodule deinitialize --keep-active lib"}
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
qwen/qwen3.6-27b:low FAIL 33.3% 102 in → 1,389 out (1,414 reasoning)
git submodule deinit --
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
thinkingmachines/inkling:none FAIL 33.3% 98 in → 13 out (0 reasoning)
git rm --cached lib && rm -rf lib<|end_message|>
Failure: Failed assertions: [{'type': 'file_not_exists', 'path': 'lib/lib.py'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
thinkingmachines/inkling:none__json_schema FAIL 33.3% 91 in → 16 out (0 reasoning)
git rm lib
JSON Schema Structured Output
(raw) { "command" : "git rm lib" }
Failure: Failed assertions: [{'type': 'file_content', 'path': '.gitmodules', 'contains': 'lib'}, {'type': 'git_output', 'command': 'git config --get submodule.lib.url', 'not_contains': 'lib-bare'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
Failure: Model 'google/gemma-4-31b-it:none' violated the reasoning_level=none invariant: provider reported 294 reasoning token(s)
Invalid JSON. Output: git submodule deinit -f lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-13c8c189-74b2-9c1b-a9c5-8d8152ba017e","request_id":"13c8c189-74b2-9c1b-a9c5-8d8152ba017e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-58c0f444-3df1-9701-91d3-b36737acf49c","request_id":"58c0f444-3df1-9701-91d3-b36737acf49c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-2fe151b5-e81a-9ff1-83a3-d204b9343ec7","request_id":"2fe151b5-e81a-9ff1-83a3-d204b9343ec7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-192d4f4f-0ccf-9171-ac9d-4875ae12536d","request_id":"192d4f4f-0ccf-9171-ac9d-4875ae12536d"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-30ec19f6-8a8e-9655-82f7-b53a5ec2e8d0","request_id":"30ec19f6-8a8e-9655-82f7-b53a5ec2e8d0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ea384ad0-1737-9b6b-aa9f-132aee584d43"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-3604e811-8370-9423-998d-10c44c9f56d7"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-7d5c77cd-2fc9-9a2f-a8bb-2c7c6cdf50db"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-0eedb999-612e-9a45-9925-0b57da8a55f3"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-b1a515a3-f18f-9a3f-99e4-9deff1fc9745"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-d3de5779-e5ae-9b87-8f9a-d2ebc2957bd2"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-693197e2-cac8-9f63-adca-701cd58696c4","request_id":"693197e2-cac8-9f63-adca-701cd58696c4"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-071128bb-3f55-91cb-ae85-a7651c0de41b","request_id":"071128bb-3f55-91cb-ae85-a7651c0de41b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-2baeea1b-9a76-9d0f-9d8a-f8c6af98000f","request_id":"2baeea1b-9a76-9d0f-9d8a-f8c6af98000f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-43bf2d76-f48a-94cb-9279-92d3c2ff70e2","request_id":"43bf2d76-f48a-94cb-9279-92d3c2ff70e2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b9c7692f-4549-9386-a42b-e33f2bfbbe39","request_id":"b9c7692f-4549-9386-a42b-e33f2bfbbe39"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d9e052a0-8811-96e3-a67d-0a02809d4039","request_id":"d9e052a0-8811-96e3-a67d-0a02809d4039"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-446ef031-a425-952d-845f-b290f4212d53","request_id":"446ef031-a425-952d-845f-b290f4212d53"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-85c4c761-9658-995b-93e5-8b36ab89d3f9","request_id":"85c4c761-9658-995b-93e5-8b36ab89d3f9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-5dfd9391-4330-902d-8ae2-96bde20b675a","request_id":"5dfd9391-4330-902d-8ae2-96bde20b675a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-928a6ab5-8b3c-97f7-91a8-20ff34c719ca","request_id":"928a6ab5-8b3c-97f7-91a8-20ff34c719ca"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
Failure: Model 'tencent/hy3:none' violated the reasoning_level=none invariant: provider reported 441 reasoning token(s)
Invalid JSON. Output: git submodule deinit lib{"command": "git submodule deinit lib"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule deinit lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)