Add submodule with specific branch
Tests ability to add a submodule tracking a specific branch. Evaluates branch-specific submodule configuration.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 echo 'App' > app.txt
  5. 05 git add app.txt
  6. 06 git commit -m 'Initial commit'
  7. 07 mkdir -p ../lib-src && cd ../lib-src && git init && git config user.email 'test@test.com' && git config user.name 'Test User' && echo 'lib main' > lib.py && git add lib.py && git commit -m 'Initial lib on main' && git checkout -b develop && echo 'lib develop' > dev.py && git add dev.py && git commit -m 'Develop branch'
  8. 08 git clone --bare ../lib-src ../lib-bare
  9. 09 git config protocol.file.allow always
Prompt
Add the repository at '../lib-bare' as a submodule at path 'lib', tracking the 'develop' branch. Output ONLY the git submodule command, nothing else.
Expected
git submodule add -b develop ../lib-bare lib

Scoped model quality, cost, API time, and token usage for submodule_usage/f012.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 72 in → 47 out (31 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-fable-5:high__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-fable-5:low PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-fable-5:low__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-fable-5:max PASS 100% 72 in → 174 out (51 reasoning)
git submodule add -b develop ../lib-bare lib
anthropic/claude-fable-5:max__json_schema PASS 100% 282 in → 127 out (52 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-fable-5:medium PASS 100% 72 in → 47 out (31 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-fable-5:medium__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-fable-5:xhigh PASS 100% 72 in → 47 out (31 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-haiku-4.5:high PASS 100% 85 in → 176 out (137 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 248 in → 168 out (126 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-haiku-4.5:low PASS 100% 85 in → 203 out (164 reasoning)
``` git submodule add --branch develop ../lib-bare lib ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 248 in → 196 out (157 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-haiku-4.5:medium PASS 100% 85 in → 124 out (91 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 248 in → 177 out (143 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-haiku-4.5:none PASS 100% 56 in → 20 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 218 in → 21 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 85 in → 176 out (135 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 248 in → 277 out (259 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.6:high PASS 100% 56 in → 21 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.6:high__json_schema PASS 100% 219 in → 65 out (26 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.6:low PASS 100% 56 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
anthropic/claude-opus-4.6:low__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.6:max PASS 100% 56 in → 103 out (65 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.6:max__json_schema PASS 100% 219 in → 89 out (48 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.6:medium PASS 100% 56 in → 21 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.6:none PASS 100% 56 in → 20 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.6:xhigh PASS 100% 56 in → 21 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 219 in → 66 out (26 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.7:high PASS 100% 77 in → 27 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.7:high__json_schema PASS 100% 287 in → 29 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.7:low PASS 100% 77 in → 24 out (0 reasoning)
`git submodule add -b develop ../lib-bare lib`
anthropic/claude-opus-4.7:low__json_schema PASS 100% 287 in → 29 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.7:max PASS 100% 77 in → 140 out (33 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.7:max__json_schema PASS 100% 287 in → 29 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.7:medium PASS 100% 77 in → 24 out (0 reasoning)
`git submodule add -b develop ../lib-bare lib`
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 287 in → 29 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.7:none PASS 100% 77 in → 27 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.7:none__json_schema PASS 100% 287 in → 29 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.7:xhigh PASS 100% 77 in → 27 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 287 in → 29 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.8:high PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.8:high__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.8:low PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.8:max PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 282 in → 93 out (37 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.8:medium PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.8:none PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 282 in → 26 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-opus-4.8:xhigh PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-4.6:high PASS 100% 56 in → 93 out (51 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-4.6:low PASS 100% 56 in → 20 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-4.6:low__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-4.6:max PASS 100% 56 in → 124 out (79 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 219 in → 119 out (76 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-4.6:medium PASS 100% 56 in → 49 out (16 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-4.6:none PASS 100% 56 in → 20 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-4.6:none__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-4.6:xhigh PASS 100% 56 in → 84 out (44 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 219 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-5:high PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-5:high__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-5:low PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-5:low__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-5:medium PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-5:none PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-5:none__json_schema PASS 100% 282 in → 26 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
anthropic/claude-sonnet-5:xhigh PASS 100% 72 in → 24 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 282 in → 27 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
arcee-ai/trinity-large-thinking:high PASS 100% 50 in → 205 out (191 reasoning)
git submodule add -b develop ../lib-bare lib
arcee-ai/trinity-large-thinking:low PASS 100% 50 in → 249 out (235 reasoning)
git submodule add -b develop ../lib-bare lib
arcee-ai/trinity-large-thinking:medium PASS 100% 50 in → 245 out (231 reasoning)
git submodule add -b develop ../lib-bare lib
arcee-ai/trinity-large-thinking:xhigh PASS 100% 50 in → 330 out (316 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:high PASS 100% 125 in → 69 out (56 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 125 in → 69 out (55 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
deepseek/deepseek-v4-flash-0731:low PASS 100% 125 in → 63 out (50 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 46 in → 142 out (122 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
deepseek/deepseek-v4-flash-0731:max PASS 100% 125 in → 65 out (56 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 138 in → 99 out (90 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash-0731:medium PASS 100% 46 in → 103 out (90 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 46 in → 85 out (65 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 46 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 46 in → 21 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 46 in → 144 out (131 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash-0731:xhigh__json_schema PASS 100% 125 in → 72 out (55 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash:high PASS 100% 46 in → 71 out (59 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash:high__json_schema PASS 100% 48 in → 95 out (73 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash:low PASS 100% 46 in → 77 out (63 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash:medium PASS 100% 46 in → 87 out (76 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 46 in → 120 out (105 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash:none PASS 100% 46 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 48 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 125 in → 235 out (221 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 127 in → 209 out (187 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-pro:low PASS 100% 46 in → 282 out (268 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 46 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-pro:medium PASS 100% 46 in → 197 out (183 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 48 in → 122 out (103 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
deepseek/deepseek-v4-pro:none PASS 100% 46 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 46 in → 23 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 125 in → 302 out (288 reasoning)
git submodule add -b develop ../lib-bare lib
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 127 in → 145 out (123 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemini-3-flash-preview:high PASS 100% 41 in → 156 out (145 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3-flash-preview:high__json_schema PASS 100% 98 in → 360 out (345 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemini-3-flash-preview:low PASS 100% 41 in → 130 out (119 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3-flash-preview:low__json_schema PASS 100% 98 in → 252 out (237 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemini-3-flash-preview:medium PASS 100% 41 in → 138 out (127 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3-flash-preview:medium__json_schema PASS 100% 98 in → 233 out (217 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3-flash-preview:xhigh PASS 100% 41 in → 156 out (145 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 42 in → 238 out (223 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 42 in → 285 out (274 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 42 in → 276 out (261 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 42 in → 111 out (100 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-flash-lite-preview:low__json_schema PASS 100% 98 in → 131 out (111 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemini-3.1-flash-lite-preview:medium PASS 100% 42 in → 135 out (124 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 42 in → 173 out (157 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 41 in → 173 out (162 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 42 in → 204 out (188 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-pro-preview:high PASS 100% 41 in → 224 out (213 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 98 in → 372 out (356 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-pro-preview:low PASS 100% 41 in → 163 out (152 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 98 in → 94 out (79 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-pro-preview:medium PASS 100% 41 in → 252 out (241 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 98 in → 101 out (85 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 41 in → 250 out (239 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 98 in → 323 out (308 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash-lite:high PASS 100% 41 in → 322 out (311 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 98 in → 258 out (242 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash-lite:low PASS 100% 41 in → 13 out (0 reasoning)
`git submodule add -b develop ../lib-bare lib`
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 98 in → 21 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemini-3.5-flash-lite:max PASS 100% 42 in → 296 out (285 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 98 in → 297 out (281 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash-lite:medium PASS 100% 42 in → 191 out (180 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 98 in → 326 out (310 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash-lite:xhigh PASS 100% 42 in → 321 out (310 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 98 in → 306 out (290 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash:high PASS 100% 41 in → 337 out (326 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash:high__json_schema PASS 100% 98 in → 275 out (259 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash:low PASS 100% 41 in → 219 out (208 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash:medium PASS 100% 41 in → 254 out (243 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash:medium__json_schema PASS 100% 98 in → 293 out (277 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.5-flash:xhigh PASS 100% 41 in → 285 out (274 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 98 in → 328 out (312 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.6-flash:high PASS 100% 41 in → 322 out (311 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.6-flash:high__json_schema PASS 100% 98 in → 250 out (235 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemini-3.6-flash:low PASS 100% 42 in → 202 out (191 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.6-flash:low__json_schema PASS 100% 98 in → 21 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemini-3.6-flash:max PASS 100% 41 in → 302 out (291 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.6-flash:max__json_schema PASS 100% 98 in → 239 out (223 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemini-3.6-flash:medium PASS 100% 42 in → 203 out (192 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.6-flash:medium__json_schema PASS 100% 42 in → 153 out (138 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemini-3.6-flash:xhigh PASS 100% 42 in → 353 out (342 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 42 in → 329 out (314 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
google/gemma-4-26b-a4b-it:high PASS 100% 56 in → 179 out (140 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 56 in → 171 out (131 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-26b-a4b-it:low PASS 100% 56 in → 163 out (136 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 57 in → 154 out (116 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-26b-a4b-it:medium PASS 100% 57 in → 184 out (147 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 57 in → 165 out (130 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-26b-a4b-it:none PASS 100% 54 in → 12 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 54 in → 22 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 56 in → 181 out (147 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 57 in → 237 out (186 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemma-4-31b-it:high PASS 100% 57 in → 134 out (103 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-31b-it:high__json_schema PASS 100% 56 in → 163 out (119 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-31b-it:low PASS 100% 57 in → 130 out (105 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-31b-it:low__json_schema PASS 100% 57 in → 144 out (107 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-31b-it:medium PASS 100% 57 in → 138 out (114 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-31b-it:medium__json_schema PASS 100% 56 in → 157 out (124 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
google/gemma-4-31b-it:none PASS 100% 54 in → 12 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-31b-it:none__json_schema PASS 100% 54 in → 17 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
google/gemma-4-31b-it:xhigh PASS 100% 57 in → 167 out (136 reasoning)
git submodule add -b develop ../lib-bare lib
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 57 in → 147 out (116 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
inclusionai/ling-2.6-flash PASS 100% 62 in → 13 out
git submodule add -b develop ../lib-bare lib
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
minimax/minimax-m2.5:high PASS 100% 81 in → 76 out (63 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.5:high__json_schema PASS 100% 81 in → 108 out (88 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m2.5:low PASS 100% 78 in → 78 out (65 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.5:low__json_schema PASS 100% 78 in → 183 out (163 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m2.5:medium PASS 100% 78 in → 79 out (75 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.5:medium__json_schema PASS 100% 78 in → 114 out (112 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m2.5:xhigh PASS 100% 78 in → 79 out (74 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 78 in → 226 out (206 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m2.7:high PASS 100% 78 in → 233 out (219 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.7:high__json_schema PASS 100% 210 in → 472 out (455 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
minimax/minimax-m2.7:low PASS 100% 81 in → 157 out (145 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.7:low__json_schema PASS 100% 78 in → 20 out (20 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m2.7:medium PASS 100% 78 in → 176 out (187 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.7:medium__json_schema PASS 100% 210 in → 241 out (220 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m2.7:xhigh PASS 100% 78 in → 184 out (171 reasoning)
git submodule add -b develop ../lib-bare lib
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 210 in → 259 out (242 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
minimax/minimax-m3:high__json_schema PASS 100% 203 in → 17 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
minimax/minimax-m3:low__json_schema PASS 100% 164 in → 20 out (19 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
minimax/minimax-m3:medium PASS 100% 216 in → 125 out (125 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
minimax/minimax-m3:medium__json_schema PASS 100% 203 in → 17 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
minimax/minimax-m3:xhigh__json_schema PASS 100% 203 in → 19 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
mistralai/devstral-2512 PASS 100% 46 in → 18 out
```bash git submodule add -b develop ../lib-bare lib ```
mistralai/devstral-2512__json_schema PASS 100% 46 in → 18 out
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
mistralai/mistral-medium-3-5:high PASS 100% 58 in → 191 out (176 reasoning)
git submodule add -b develop ../lib-bare lib
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 46 in → 244 out (236 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
mistralai/mistral-medium-3-5:low PASS 100% 58 in → 1,168 out (1,169 reasoning)
git submodule add -b develop ../lib-bare lib
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 46 in → 268 out (258 reasoning)
git submodule add -b develop '../lib-bare' lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop '../lib-bare' lib"}
mistralai/mistral-medium-3-5:medium PASS 100% 58 in → 607 out (639 reasoning)
git submodule add -b develop ../lib-bare lib
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 46 in → 236 out (216 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
mistralai/mistral-medium-3-5:none PASS 100% 58 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 46 in → 18 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 58 in → 300 out (298 reasoning)
git submodule add -b develop ../lib-bare lib
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 46 in → 714 out (724 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
moonshotai/kimi-k2.6:high PASS 100% 48 in → 472 out (459 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.6:high__json_schema PASS 100% 48 in → 192 out (193 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
moonshotai/kimi-k2.6:low PASS 100% 48 in → 96 out (84 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.6:low__json_schema PASS 100% 47 in → 530 out (193 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
moonshotai/kimi-k2.6:medium PASS 100% 48 in → 280 out (267 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 47 in → 298 out (213 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
moonshotai/kimi-k2.6:xhigh PASS 100% 48 in → 328 out (317 reasoning)
git submodule add --branch develop ../lib-bare lib
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 48 in → 118 out (97 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
moonshotai/kimi-k2.7-code:high PASS 100% 48 in → 69 out (56 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 48 in → 252 out (232 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
moonshotai/kimi-k2.7-code:low PASS 100% 47 in → 216 out (203 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 47 in → 496 out (184 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
moonshotai/kimi-k2.7-code:medium PASS 100% 48 in → 68 out (61 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 48 in → 206 out (186 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 48 in → 106 out (93 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 131 in → 121 out (104 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
moonshotai/kimi-k3:high PASS 100% 125 in → 150 out (124 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:high__json_schema PASS 100% 125 in → 105 out (79 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:low PASS 100% 125 in → 123 out (97 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:low__json_schema PASS 100% 125 in → 122 out (96 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:max PASS 100% 125 in → 145 out (119 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:max__json_schema PASS 100% 125 in → 114 out (88 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:medium PASS 100% 125 in → 113 out (87 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:medium__json_schema PASS 100% 125 in → 420 out (387 reasoning)
git -c protocol.file.allow=always submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:xhigh PASS 100% 125 in → 124 out (98 reasoning)
git submodule add -b develop ../lib-bare lib
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 125 in → 134 out (108 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 59 in → 111 out (103 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 59 in → 84 out (73 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 59 in → 162 out (156 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 59 in → 165 out (174 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 59 in → 106 out (90 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 59 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-nano-30b-a3b:none__json_schema PASS 100% 59 in → 21 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 59 in → 54 out (40 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 59 in → 69 out (50 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 59 in → 75 out (53 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 59 in → 102 out (97 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 59 in → 84 out (61 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 59 in → 80 out (61 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 59 in → 82 out (59 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 59 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 59 in → 21 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 59 in → 71 out (52 reasoning)
git submodule add -b develop ../lib-bare lib
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 59 in → 107 out (80 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
openai/gpt-5.4-mini:high PASS 100% 48 in → 95 out (77 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-mini:high__json_schema PASS 100% 83 in → 118 out (92 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-mini:low PASS 100% 48 in → 65 out (47 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-mini:low__json_schema PASS 100% 83 in → 92 out (66 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-mini:medium PASS 100% 48 in → 76 out (58 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-mini:medium__json_schema PASS 100% 83 in → 111 out (85 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-mini:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-mini:none__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-mini:xhigh PASS 100% 48 in → 148 out (130 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 83 in → 542 out (516 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-nano:high PASS 100% 48 in → 49 out (31 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-nano:high__json_schema PASS 100% 83 in → 136 out (110 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-nano:low PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4-nano:low__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4-nano:medium__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add --branch develop ../lib-bare lib"}
openai/gpt-5.4-nano:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add --branch develop ../lib-bare lib
openai/gpt-5.4-nano:none__json_schema PASS 100% 83 in → 38 out (0 reasoning)
git submodule add -b develop --name lib ../lib-bare lib && git submodule set-branch --branch develop lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop --name lib ../lib-bare lib && git submodule set-branch --branch develop lib"}
openai/gpt-5.4-nano:xhigh PASS 100% 48 in → 170 out (152 reasoning)
git submodule add --branch develop ../lib-bare lib
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 83 in → 1,141 out (1,115 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add --branch develop ../lib-bare lib"}
openai/gpt-5.4:high PASS 100% 48 in → 102 out (84 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4:high__json_schema PASS 100% 83 in → 86 out (60 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4:low PASS 100% 48 in → 38 out (20 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4:low__json_schema PASS 100% 83 in → 51 out (25 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4:medium PASS 100% 48 in → 77 out (59 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4:medium__json_schema PASS 100% 83 in → 83 out (57 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4:none__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.4:xhigh PASS 100% 48 in → 182 out (164 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.4:xhigh__json_schema PASS 100% 83 in → 542 out (516 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.5:high PASS 100% 48 in → 93 out (75 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.5:high__json_schema PASS 100% 83 in → 60 out (34 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.5:low PASS 100% 48 in → 49 out (31 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.5:low__json_schema PASS 100% 83 in → 50 out (24 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.5:medium PASS 100% 48 in → 46 out (28 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.5:medium__json_schema PASS 100% 83 in → 64 out (38 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.5:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.5:none__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.5:xhigh PASS 100% 48 in → 104 out (86 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.5:xhigh__json_schema PASS 100% 83 in → 224 out (198 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-luna:high PASS 100% 48 in → 45 out (27 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-luna:high__json_schema PASS 100% 83 in → 57 out (31 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-luna:low PASS 100% 48 in → 40 out (22 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-luna:low__json_schema PASS 100% 83 in → 48 out (22 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-luna:max PASS 100% 48 in → 58 out (40 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-luna:max__json_schema PASS 100% 83 in → 88 out (62 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-luna:medium PASS 100% 48 in → 42 out (24 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-luna:medium__json_schema PASS 100% 83 in → 50 out (24 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-luna:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-luna:none__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-luna:xhigh PASS 100% 48 in → 65 out (47 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 83 in → 153 out (127 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-sol:high PASS 100% 48 in → 43 out (25 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-sol:high__json_schema PASS 100% 83 in → 136 out (110 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-sol:low PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-sol:low__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-sol:max PASS 100% 48 in → 74 out (56 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-sol:max__json_schema PASS 100% 83 in → 198 out (172 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-sol:medium PASS 100% 48 in → 43 out (25 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-sol:medium__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-sol:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-sol:none__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-sol:xhigh PASS 100% 48 in → 51 out (33 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 83 in → 134 out (108 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-terra:high PASS 100% 48 in → 41 out (23 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-terra:high__json_schema PASS 100% 83 in → 51 out (25 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-terra:low PASS 100% 48 in → 41 out (23 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-terra:low__json_schema PASS 100% 83 in → 80 out (54 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-terra:max PASS 100% 48 in → 117 out (99 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-terra:max__json_schema PASS 100% 83 in → 156 out (130 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-terra:medium PASS 100% 48 in → 42 out (24 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-terra:medium__json_schema PASS 100% 83 in → 51 out (25 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-terra:none PASS 100% 48 in → 16 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-terra:none__json_schema PASS 100% 83 in → 24 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-5.6-terra:xhigh PASS 100% 48 in → 70 out (52 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 83 in → 60 out (34 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-oss-120b:high PASS 100% 109 in → 214 out (184 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-120b:high__json_schema PASS 100% 96 in → 518 out (562 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-oss-120b:low PASS 100% 109 in → 66 out (54 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-120b:low__json_schema PASS 100% 109 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
openai/gpt-oss-120b:medium PASS 100% 109 in → 68 out (56 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-120b:medium__json_schema PASS 100% 111 in → 57 out (30 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
openai/gpt-oss-120b:xhigh PASS 100% 96 in → 215 out (209 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 111 in → 222 out (214 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-oss-20b:high PASS 100% 107 in → 276 out (281 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-20b:high__json_schema PASS 100% 107 in → 224 out (225 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-oss-20b:low PASS 100% 111 in → 71 out (49 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-20b:low__json_schema PASS 100% 111 in → 50 out (24 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
openai/gpt-oss-20b:medium PASS 100% 111 in → 162 out (146 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-20b:medium__json_schema PASS 100% 96 in → 230 out (215 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command" : "git submodule add -b develop ../lib-bare lib" }
openai/gpt-oss-20b:xhigh PASS 100% 107 in → 204 out (209 reasoning)
git submodule add -b develop ../lib-bare lib
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 159 in → 934 out (1,162 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
poolside/laguna-m.1:high PASS 100% 54 in → 215 out (201 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-m.1:high__json_schema PASS 100% 54 in → 296 out (276 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
poolside/laguna-m.1:low PASS 100% 54 in → 394 out (381 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-m.1:low__json_schema PASS 100% 54 in → 385 out (364 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
poolside/laguna-m.1:medium PASS 100% 54 in → 539 out (528 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-m.1:medium__json_schema PASS 100% 54 in → 282 out (265 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
poolside/laguna-m.1:none PASS 100% 54 in → 17 out (0 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
poolside/laguna-m.1:none__json_schema PASS 100% 54 in → 17 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
poolside/laguna-m.1:xhigh PASS 100% 54 in → 338 out (327 reasoning)
git submodule add --branch develop ../lib-bare lib
poolside/laguna-m.1:xhigh__json_schema PASS 100% 54 in → 294 out (272 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add --branch develop ../lib-bare lib" }
poolside/laguna-xs-2.1:high PASS 100% 54 in → 314 out (303 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs-2.1:high__json_schema PASS 100% 54 in → 333 out (313 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
poolside/laguna-xs-2.1:low PASS 100% 54 in → 290 out (278 reasoning)
git submodule add --branch develop ../lib-bare lib
poolside/laguna-xs-2.1:low__json_schema PASS 100% 54 in → 325 out (310 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
poolside/laguna-xs-2.1:medium PASS 100% 54 in → 311 out (300 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 54 in → 390 out (372 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
poolside/laguna-xs-2.1:none PASS 100% 54 in → 11 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs-2.1:none__json_schema PASS 100% 54 in → 22 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
poolside/laguna-xs-2.1:xhigh PASS 100% 54 in → 253 out (242 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 54 in → 229 out (207 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
poolside/laguna-xs.2:high PASS 100% 91 in → 135 out (123 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs.2:high__json_schema PASS 100% 91 in → 132 out (116 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add --branch develop ../lib-bare lib"}
poolside/laguna-xs.2:low PASS 100% 91 in → 90 out (77 reasoning)
git submodule add --branch develop ../lib-bare lib
poolside/laguna-xs.2:low__json_schema PASS 100% 91 in → 136 out (119 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add --branch develop ../lib-bare lib"}
poolside/laguna-xs.2:medium PASS 100% 91 in → 131 out (119 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs.2:medium__json_schema PASS 100% 91 in → 137 out (119 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add --branch develop ../lib-bare lib"}
poolside/laguna-xs.2:none PASS 100% 91 in → 13 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
poolside/laguna-xs.2:none__json_schema PASS 100% 91 in → 17 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
poolside/laguna-xs.2:xhigh PASS 100% 91 in → 212 out (199 reasoning)
git submodule add --branch develop ../lib-bare lib
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 91 in → 189 out (171 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
qwen/qwen3.6-27b:high PASS 100% 51 in → 992 out (1 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-27b:high__json_schema PASS 100% 51 in → 1,509 out (1,452 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
qwen/qwen3.6-27b:low PASS 100% 51 in → 349 out (333 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-27b:low__json_schema PASS 100% 53 in → 22 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
qwen/qwen3.6-27b:medium PASS 100% 51 in → 1,102 out (1,093 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.6-27b:medium__json_schema PASS 100% 51 in → 1,085 out (1,069 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
qwen/qwen3.6-27b:none PASS 100% 53 in → 15 out (0 reasoning)
git submodule add -b develop --name lib ../lib-bare lib
qwen/qwen3.6-27b:none__json_schema PASS 100% 53 in → 22 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
qwen/qwen3.6-27b:xhigh PASS 100% 51 in → 1,103 out (1,086 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 51 in → 1,412 out (1,400 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add --branch develop ../lib-bare lib" }
qwen/qwen3.6-35b-a3b:high PASS 100% 51 in → 785 out (781 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 51 in → 957 out (946 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
qwen/qwen3.6-35b-a3b:low PASS 100% 51 in → 526 out (504 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 51 in → 677 out (661 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
qwen/qwen3.6-35b-a3b:medium PASS 100% 51 in → 1,137 out (1,112 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 51 in → 643 out (639 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add --branch develop ../lib-bare lib" }
qwen/qwen3.6-35b-a3b:none PASS 100% 53 in → 12 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 51 in → 904 out (888 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 51 in → 805 out (796 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add --branch develop ../lib-bare lib" }
qwen/qwen3.6-flash:high PASS 100% 51 in → 726 out (710 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-flash:low PASS 100% 51 in → 743 out (727 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-flash:medium PASS 100% 51 in → 1,007 out (991 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.6-flash:none PASS 100% 53 in → 11 out (0 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.6-flash:xhigh PASS 100% 51 in → 319 out (303 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-flash:high PASS 100% 51 in → 644 out (628 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-flash:low PASS 100% 51 in → 1,044 out (1,028 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-flash:max PASS 100% 51 in → 287 out (271 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.7-flash:medium PASS 100% 51 in → 884 out (868 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-flash:none PASS 100% 53 in → 11 out (0 reasoning)
git submodule add --branch develop ../lib-bare lib
qwen/qwen3.7-flash:xhigh PASS 100% 51 in → 612 out (596 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-max:high PASS 100% 51 in → 606 out (590 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-max:low PASS 100% 51 in → 626 out (610 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-max:medium PASS 100% 51 in → 612 out (596 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-max:none PASS 100% 53 in → 11 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-max:xhigh PASS 100% 51 in → 577 out (561 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-plus:high PASS 100% 51 in → 501 out (485 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-plus:low PASS 100% 51 in → 434 out (418 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-plus:medium PASS 100% 51 in → 609 out (593 reasoning)
git submodule add -b develop ../lib-bare lib
qwen/qwen3.7-plus:none PASS 100% 53 in → 12 out (0 reasoning)
git submodule add -b develop -- ../lib-bare lib
qwen/qwen3.7-plus:xhigh PASS 100% 51 in → 565 out (549 reasoning)
git submodule add -b develop ../lib-bare lib
tencent/hy3:high PASS 100% 52 in → 426 out (417 reasoning)
git submodule add -b develop ../lib-bare lib
tencent/hy3:high__json_schema PASS 100% 52 in → 397 out (398 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
tencent/hy3:low PASS 100% 52 in → 208 out (195 reasoning)
git submodule add -b develop ../lib-bare lib
tencent/hy3:low__json_schema PASS 100% 52 in → 164 out (144 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
tencent/hy3:medium PASS 100% 52 in → 169 out (156 reasoning)
git submodule add -b develop ../lib-bare lib
tencent/hy3:medium__json_schema PASS 100% 52 in → 147 out (126 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
tencent/hy3:none PASS 100% 55 in → 12 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
tencent/hy3:xhigh PASS 100% 52 in → 326 out (313 reasoning)
git submodule add -b develop ../lib-bare lib
tencent/hy3:xhigh__json_schema PASS 100% 52 in → 300 out (282 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
thinkingmachines/inkling-small:high PASS 100% 57 in → 188 out (170 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling-small:high__json_schema PASS 100% 57 in → 126 out (101 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
thinkingmachines/inkling-small:low PASS 100% 57 in → 65 out (47 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling-small:low__json_schema PASS 100% 57 in → 74 out (50 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
thinkingmachines/inkling-small:max PASS 100% 57 in → 140 out (128 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling-small:max__json_schema PASS 100% 57 in → 188 out (164 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
thinkingmachines/inkling-small:medium PASS 100% 57 in → 103 out (91 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling-small:medium__json_schema PASS 100% 57 in → 152 out (130 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
thinkingmachines/inkling-small:none PASS 100% 55 in → 15 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling-small:xhigh PASS 100% 57 in → 109 out (97 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 57 in → 143 out (121 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
thinkingmachines/inkling:high PASS 100% 57 in → 78 out (63 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling:low PASS 100% 57 in → 33 out (14 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling:max PASS 100% 57 in → 128 out (110 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling:max__json_schema PASS 100% 57 in → 183 out (156 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
thinkingmachines/inkling:medium PASS 100% 57 in → 74 out (57 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling:medium__json_schema PASS 100% 57 in → 105 out (78 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
thinkingmachines/inkling:xhigh PASS 100% 57 in → 120 out (102 reasoning)
git submodule add -b develop ../lib-bare lib
thinkingmachines/inkling:xhigh__json_schema PASS 100% 57 in → 100 out (79 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.3:high PASS 100% 225 in → 140 out (129 reasoning)
git submodule add -b develop ../lib-bare lib
x-ai/grok-4.3:high__json_schema PASS 100% 281 in → 444 out (429 reasoning)
git submodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add --branch develop ../lib-bare lib"}
x-ai/grok-4.3:low PASS 100% 231 in → 186 out (175 reasoning)
git submodule add -b develop ../lib-bare lib
x-ai/grok-4.3:low__json_schema PASS 100% 287 in → 298 out (283 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.3:max PASS 100% 225 in → 263 out (252 reasoning)
git submodule add -b develop ../lib-bare lib
x-ai/grok-4.3:max__json_schema PASS 100% 281 in → 376 out (361 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.3:medium PASS 100% 231 in → 193 out (182 reasoning)
git submodule add --branch develop ../lib-bare lib
x-ai/grok-4.3:medium__json_schema PASS 100% 287 in → 386 out (367 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
x-ai/grok-4.3:none PASS 100% 223 in → 11 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
x-ai/grok-4.3:none__json_schema PASS 100% 279 in → 15 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.3:xhigh PASS 100% 225 in → 184 out (173 reasoning)
git submodule add -b develop ../lib-bare lib
x-ai/grok-4.3:xhigh__json_schema PASS 100% 281 in → 315 out (300 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.5:high PASS 100% 246 in → 181 out (170 reasoning)
git submodule add -b develop ../lib-bare lib
x-ai/grok-4.5:high__json_schema PASS 100% 312 in → 315 out (299 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.5:low PASS 100% 246 in → 102 out (87 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
x-ai/grok-4.5:low__json_schema PASS 100% 312 in → 202 out (186 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.5:max PASS 100% 246 in → 145 out (130 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
x-ai/grok-4.5:max__json_schema PASS 100% 312 in → 281 out (265 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.5:medium PASS 100% 246 in → 128 out (113 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
x-ai/grok-4.5:medium__json_schema PASS 100% 312 in → 274 out (258 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
x-ai/grok-4.5:xhigh PASS 100% 246 in → 98 out (83 reasoning)
``` git submodule add -b develop ../lib-bare lib ```
x-ai/grok-4.5:xhigh__json_schema PASS 100% 312 in → 245 out (229 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
z-ai/glm-4.7-flash:high PASS 100% 45 in → 525 out (527 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-4.7-flash:low PASS 100% 45 in → 352 out (329 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-4.7-flash:low__json_schema PASS 100% 45 in → 349 out (318 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
z-ai/glm-4.7-flash:medium PASS 100% 45 in → 617 out (640 reasoning)
git submodule add --branch develop ../lib-bare lib
z-ai/glm-4.7-flash:none PASS 100% 45 in → 11 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-4.7-flash:none__json_schema PASS 100% 45 in → 20 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
z-ai/glm-4.7-flash:xhigh PASS 100% 45 in → 382 out (360 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-4.7-flash:xhigh__json_schema PASS 100% 45 in → 369 out (346 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
z-ai/glm-5.2:high PASS 100% 52 in → 226 out (213 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-5.2:high__json_schema PASS 100% 54 in → 213 out (195 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
z-ai/glm-5.2:low PASS 100% 52 in → 129 out (117 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-5.2:low__json_schema PASS 100% 54 in → 112 out (92 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
z-ai/glm-5.2:medium PASS 100% 52 in → 233 out (220 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-5.2:medium__json_schema PASS 100% 52 in → 279 out (256 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add -b develop ../lib-bare lib" }
z-ai/glm-5.2:none PASS 100% 46 in → 12 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-5.2:none__json_schema PASS 100% 46 in → 17 out (0 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b develop ../lib-bare lib"}
z-ai/glm-5.2:xhigh PASS 100% 52 in → 339 out (335 reasoning)
git submodule add -b develop ../lib-bare lib
z-ai/glm-5.2:xhigh__json_schema PASS 100% 227 in → 74 out (57 reasoning)
git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
(raw) {"command":"git submodule add -b develop ../lib-bare lib"}
arcee-ai/trinity-mini:low__json_schema FAIL 66.7% 50 in → 1,072 out (1,288 reasoning)
git submodule add ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add ../lib-bare lib" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
arcee-ai/trinity-mini:medium__json_schema FAIL 66.7% 50 in → 408 out (456 reasoning)
git submodule add ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add ../lib-bare lib" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
arcee-ai/trinity-mini:xhigh FAIL 66.7% 50 in → 1,436 out (1,648 reasoning)
git submodule add ../lib-bare lib
Failure: Failed assertions: [{'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
arcee-ai/trinity-mini:xhigh__json_schema FAIL 66.7% 50 in → 833 out (968 reasoning)
git submodule add ../lib-bare lib
JSON Schema Structured Output
(raw) { "command":"git submodule add ../lib-bare lib" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
liquid/lfm-2-24b-a2b FAIL 66.7% 53 in → 14 out
git submodule add ../lib-bare lib/develop
Failure: Failed assertions: [{'type': 'file_exists', 'path': 'lib/dev.py'}]
qwen/qwen3.6-35b-a3b:none__json_schema FAIL 66.7% 53 in → 19 out (0 reasoning)
git submodule add ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "git submodule add ../lib-bare lib" }
Failure: Failed assertions: [{'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
arcee-ai/trinity-mini:high FAIL 0% 50 in → 290 out (307 reasoning)
git submodule add-b develop ../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
arcee-ai/trinity-mini:high__json_schema FAIL 0% 50 in → 824 out (868 reasoning)
gitsubmodule add --branch develop ../lib-bare lib
JSON Schema Structured Output
(raw) { "command": "gitsubmodule add --branch develop ../lib-bare lib" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
arcee-ai/trinity-mini:low FAIL 0% 50 in → 307 out (334 reasoning)
git submodule add -b develop../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
arcee-ai/trinity-mini:medium FAIL 0% 50 in → 460 out (491 reasoning)
git submodule add--branch develop ../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
Invalid JSON. Output: gitsubmodule add -b develop ../lib-bare lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
deepseek/deepseek-v4-pro:high FAIL 0% 46 in → 118 out (105 reasoning)
git submodule add-b develop ../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
deepseek/deepseek-v4-pro:high__json_schema FAIL 0% 46 in → 21 out (20 reasoning)
git submodule add ../lib-bare lib -b develop
JSON Schema Structured Output
(raw) { "command": "git submodule add ../lib-bare lib -b develop" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
google/gemini-3.5-flash:low__json_schema FAIL 0% 98 in → 122 out (106 reasoning)
git submodule add -b ../lib-bare lib
JSON Schema Structured Output
(raw) {"command": "git submodule add -b ../lib-bare lib"}
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
ibm-granite/granite-4.1-8b FAIL 0% 50 in → 12 out
git submodule add ../lib-bare lib --branch develop
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
git submodule add ../lib-bare lib --branch develop
JSON Schema Structured Output
(raw) { "command": "git submodule add ../lib-bare lib --branch develop" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
Invalid JSON. Output: git submodule add ../lib-bare lib/develop
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
minimax/minimax-m3:high FAIL 0% 216 in → 98 out (93 reasoning)
git submodule add -b develop../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
minimax/minimax-m3:low FAIL 0% 216 in → 101 out (103 reasoning)
gitsubmodule add -b develop ../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
minimax/minimax-m3:xhigh FAIL 0% 216 in → 66 out (56 reasoning)
git submodule add -b develop../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
nvidia/nemotron-3-nano-30b-a3b:high__json_schema FAIL 0% 59 in → 3,145 out (3,574 reasoning)
git submodule add ../lib-bare lib --branch develop
JSON Schema Structured Output
(raw) { "command": "git submodule add ../lib-bare lib --branch develop" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema FAIL 0% 59 in → 789 out (732 reasoning)
git submodule add ../lib-bare lib -b develop
JSON Schema Structured Output
(raw) {"command": "git submodule add ../lib-bare lib -b develop" }
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
openai/gpt-5.4-nano:medium FAIL 0% 48 in → 32 out (0 reasoning)
git submodule add -b develop --track git@github.com:YOUR_ORG_OR_USER/lib-bare.git ../lib-bare lib
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4d3cad70-3480-9a79-9b7c-6b59400dcd50","request_id":"4d3cad70-3480-9a79-9b7c-6b59400dcd50"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6849e96f-8f95-987f-bc48-d5b1587899fd","request_id":"6849e96f-8f95-987f-bc48-d5b1587899fd"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1e1eb893-140b-9448-9022-7fd8688dd08f","request_id":"1e1eb893-140b-9448-9022-7fd8688dd08f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d690b16c-4ade-9191-9281-e28908c7e546","request_id":"d690b16c-4ade-9191-9281-e28908c7e546"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-04e0ffc8-9c9e-9e82-a24d-134c04e964bc","request_id":"04e0ffc8-9c9e-9e82-a24d-134c04e964bc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-a477beac-b250-9e61-b189-3c21bc6dbcfd"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-5661c8fc-8d83-9778-8e68-a63f400c104d"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-fcb02b70-fc43-9620-b035-3c4c7305e271"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-fa404f55-b9f0-97e7-9c34-e8f4a07d3120"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-54c78d3c-2f6c-947e-af76-28819aac5613"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ddf084c6-b692-988a-8eb8-9f8558c1bd30"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-69813b22-e224-93ec-93f9-8c9db35efdcd","request_id":"69813b22-e224-93ec-93f9-8c9db35efdcd"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-780f01af-2b09-96e6-b64a-b87ccbc432ad","request_id":"780f01af-2b09-96e6-b64a-b87ccbc432ad"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-34fdbe6c-8e64-9f7a-a804-8637d7c8a337","request_id":"34fdbe6c-8e64-9f7a-a804-8637d7c8a337"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3cf32fbb-5596-93b7-b1b9-c398374108b1","request_id":"3cf32fbb-5596-93b7-b1b9-c398374108b1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-849bdbc3-4eb6-94c5-810f-dbc1fa68fc98","request_id":"849bdbc3-4eb6-94c5-810f-dbc1fa68fc98"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f7daf4f7-af80-96e2-9c8a-248e2284f052","request_id":"f7daf4f7-af80-96e2-9c8a-248e2284f052"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9f48a1c3-7d80-9a0c-99bd-68498b5e7d28","request_id":"9f48a1c3-7d80-9a0c-99bd-68498b5e7d28"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-50c390ad-84a2-957f-810c-d5df36c56edc","request_id":"50c390ad-84a2-957f-810c-d5df36c56edc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-76740587-7c9f-9d96-a6d1-8bbeeb3ed615","request_id":"76740587-7c9f-9d96-a6d1-8bbeeb3ed615"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-db5721a5-cf59-97f6-b71d-80c21c29e3d6","request_id":"db5721a5-cf59-97f6-b71d-80c21c29e3d6"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Model 'tencent/hy3:none' violated the reasoning_level=none invariant: provider reported 187 reasoning token(s)
Invalid JSON. Output: git submodule add -b develop ../lib-bare lib{"command": "git submodule add -b develop ../lib-bare lib"}
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: git submodule add -b develop ../lib-bare lib
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
thinkingmachines/inkling:none FAIL 0% 57 in → 15 out (0 reasoning)
git submodule add -b develop ../lib-bare lib<|end_message|>
Failure: Failed assertions: [{'type': 'dir_exists', 'path': 'lib'}, {'type': 'file_exists', 'path': 'lib/dev.py'}, {'type': 'git_output', 'command': 'cat .gitmodules', 'contains': 'develop'}]
Invalid JSON. Output: git submodule add -b develop ../lib-bare lib{ "command": "git submodule add -b develop ../lib-bare lib" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)