Three-way: conflicting port and protocol settings in same line range
Tests ability to resolve a three-way conflict with port and protocol settings. Evaluates preference-based conflict resolution with multiple configuration dimensions.

These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.

  1. 01 git init
  2. 02 git config user.email 'test@test.com'
  3. 03 git config user.name 'Test User'
  4. 04 printf 'PORT=8080 PROTOCOL=http' > config.txt
  5. 05 git add config.txt
  6. 06 git commit -m 'Initial config'
  7. 07 git checkout -b branch-a
  8. 08 printf 'PORT=8443 PROTOCOL=https' > config.txt
  9. 09 git add config.txt
  10. 10 git commit -m 'Use HTTPS port'
  11. 11 git checkout main
  12. 12 printf 'PORT=9090 PROTOCOL=http' > config.txt
  13. 13 git add config.txt
  14. 14 git commit -m 'Change to non-standard port'
  15. 15 git cherry-pick $(git rev-parse branch-a)
Prompt
Resolve the cherry-pick conflict in config.txt. The base PORT was 8080/PROTOCOL=http. The incoming commit changed both to PORT=8443/PROTOCOL=https. The current branch (main) has PORT=9090/PROTOCOL=http. Prefer the HTTPS configuration when there is a conflict. Provide ONLY the resolved config.txt content.
Expected
PORT=8443 PROTOCOL=https

Scoped model quality, cost, API time, and token usage for cherry_pick/f007.

Loading...
Loading raw attempt evidence…
anthropic/claude-fable-5:high PASS 100% 226 in → 52 out (23 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-fable-5:high__json_schema PASS 100% 447 in → 62 out (23 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-fable-5:low PASS 100% 226 in → 22 out (0 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-fable-5:low__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-fable-5:max PASS 100% 225 in → 498 out (89 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-fable-5:max__json_schema PASS 100% 444 in → 393 out (80 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-fable-5:medium PASS 100% 224 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-fable-5:medium__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-fable-5:xhigh PASS 100% 226 in → 122 out (46 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-fable-5:xhigh__json_schema PASS 100% 443 in → 64 out (24 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-haiku-4.5:high PASS 100% 173 in → 242 out (213 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-haiku-4.5:high__json_schema PASS 100% 339 in → 302 out (262 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-haiku-4.5:low PASS 100% 171 in → 198 out (172 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-haiku-4.5:low__json_schema PASS 100% 341 in → 246 out (212 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-haiku-4.5:medium PASS 100% 174 in → 199 out (183 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-haiku-4.5:medium__json_schema PASS 100% 338 in → 247 out (206 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-haiku-4.5:none PASS 100% 142 in → 16 out (0 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-haiku-4.5:none__json_schema PASS 100% 311 in → 27 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-haiku-4.5:xhigh PASS 100% 172 in → 295 out (270 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-haiku-4.5:xhigh__json_schema PASS 100% 340 in → 901 out (656 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
anthropic/claude-opus-4.6:high__json_schema PASS 100% 310 in → 129 out (81 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.6:low PASS 100% 142 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.6:low__json_schema PASS 100% 310 in → 26 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.6:max__json_schema PASS 100% 310 in → 204 out (160 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.6:medium PASS 100% 142 in → 17 out (0 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-opus-4.6:medium__json_schema PASS 100% 310 in → 27 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.6:none PASS 100% 143 in → 16 out (0 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-opus-4.6:none__json_schema PASS 100% 312 in → 26 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.6:xhigh__json_schema PASS 100% 312 in → 118 out (76 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.7:high PASS 100% 228 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.7:high__json_schema PASS 100% 450 in → 34 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.7:low PASS 100% 231 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.7:low__json_schema PASS 100% 451 in → 34 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.7:max PASS 100% 231 in → 231 out (85 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.7:max__json_schema PASS 100% 452 in → 162 out (52 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.7:medium PASS 100% 229 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.7:medium__json_schema PASS 100% 449 in → 35 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.7:none PASS 100% 230 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.7:none__json_schema PASS 100% 452 in → 34 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.7:xhigh PASS 100% 231 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.7:xhigh__json_schema PASS 100% 452 in → 34 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.8:high PASS 100% 225 in → 107 out (55 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.8:high__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.8:low PASS 100% 225 in → 53 out (19 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-opus-4.8:low__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.8:max PASS 100% 225 in → 519 out (136 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-opus-4.8:max__json_schema PASS 100% 445 in → 562 out (91 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.8:medium PASS 100% 225 in → 51 out (24 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.8:medium__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.8:none PASS 100% 226 in → 22 out (0 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-opus-4.8:none__json_schema PASS 100% 445 in → 31 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.8:xhigh PASS 100% 226 in → 148 out (35 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-opus-4.8:xhigh__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-4.6:high PASS 100% 146 in → 96 out (61 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-sonnet-4.6:high__json_schema PASS 100% 310 in → 203 out (163 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-4.6:max PASS 100% 143 in → 195 out (159 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-4.6:max__json_schema PASS 100% 311 in → 210 out (176 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-4.6:medium PASS 100% 143 in → 132 out (103 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-4.6:medium__json_schema PASS 100% 310 in → 135 out (90 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-4.6:none PASS 100% 143 in → 16 out (0 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-sonnet-4.6:xhigh PASS 100% 143 in → 186 out (143 reasoning)
``` PORT=8443 PROTOCOL=https ```
anthropic/claude-sonnet-4.6:xhigh__json_schema PASS 100% 312 in → 186 out (137 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-5:high PASS 100% 224 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-5:high__json_schema PASS 100% 445 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-5:low PASS 100% 225 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-5:low__json_schema PASS 100% 446 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-5:medium PASS 100% 226 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-5:medium__json_schema PASS 100% 444 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-5:none PASS 100% 224 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-5:none__json_schema PASS 100% 444 in → 31 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-sonnet-5:xhigh PASS 100% 224 in → 114 out (38 reasoning)
PORT=8443 PROTOCOL=https
anthropic/claude-sonnet-5:xhigh__json_schema PASS 100% 446 in → 32 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
arcee-ai/trinity-large-thinking:high PASS 100% 122 in → 464 out (464 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-large-thinking:low PASS 100% 121 in → 523 out (523 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-large-thinking:medium PASS 100% 124 in → 755 out (762 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-large-thinking:xhigh PASS 100% 122 in → 600 out (603 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-mini:high PASS 100% 123 in → 293 out (323 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-mini:high__json_schema PASS 100% 123 in → 477 out (518 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
arcee-ai/trinity-mini:low PASS 100% 121 in → 512 out (567 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-mini:low__json_schema PASS 100% 125 in → 720 out (796 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
arcee-ai/trinity-mini:medium PASS 100% 122 in → 374 out (414 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-mini:medium__json_schema PASS 100% 124 in → 385 out (433 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
arcee-ai/trinity-mini:xhigh PASS 100% 122 in → 368 out (399 reasoning)
PORT=8443 PROTOCOL=https
arcee-ai/trinity-mini:xhigh__json_schema PASS 100% 122 in → 410 out (422 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-flash-0731:high PASS 100% 129 in → 169 out (176 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash-0731:high__json_schema PASS 100% 128 in → 193 out (194 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
deepseek/deepseek-v4-flash-0731:low PASS 100% 128 in → 311 out (316 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash-0731:low__json_schema PASS 100% 128 in → 143 out (123 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
deepseek/deepseek-v4-flash-0731:max PASS 100% 219 in → 845 out (835 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash-0731:max__json_schema PASS 100% 207 in → 157 out (146 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
deepseek/deepseek-v4-flash-0731:medium PASS 100% 128 in → 157 out (148 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash-0731:medium__json_schema PASS 100% 129 in → 200 out (187 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-flash-0731:none PASS 100% 135 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash-0731:none__json_schema PASS 100% 128 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
deepseek/deepseek-v4-flash-0731:xhigh PASS 100% 209 in → 106 out (96 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash:high PASS 100% 147 in → 155 out (144 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash:low PASS 100% 128 in → 144 out (133 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash:low__json_schema PASS 100% 131 in → 143 out (124 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-flash:medium PASS 100% 128 in → 144 out (133 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash:medium__json_schema PASS 100% 130 in → 136 out (115 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-flash:none PASS 100% 131 in → 9 out (0 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash:none__json_schema PASS 100% 131 in → 19 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
deepseek/deepseek-v4-flash:xhigh PASS 100% 208 in → 267 out (256 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-flash:xhigh__json_schema PASS 100% 209 in → 189 out (161 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-pro:high PASS 100% 129 in → 221 out (210 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-pro:high__json_schema PASS 100% 131 in → 199 out (178 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-pro:low PASS 100% 129 in → 278 out (267 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-pro:low__json_schema PASS 100% 322 in → 225 out (208 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
deepseek/deepseek-v4-pro:medium PASS 100% 129 in → 321 out (310 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-pro:medium__json_schema PASS 100% 322 in → 198 out (180 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
deepseek/deepseek-v4-pro:none PASS 100% 131 in → 14 out (0 reasoning)
```txt PORT=8443 PROTOCOL=https ```
deepseek/deepseek-v4-pro:none__json_schema PASS 100% 321 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
deepseek/deepseek-v4-pro:xhigh PASS 100% 208 in → 381 out (370 reasoning)
PORT=8443 PROTOCOL=https
deepseek/deepseek-v4-pro:xhigh__json_schema PASS 100% 398 in → 429 out (408 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemini-3-flash-preview:high PASS 100% 133 in → 281 out (271 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3-flash-preview:high__json_schema PASS 100% 194 in → 1,265 out (1,248 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3-flash-preview:low PASS 100% 134 in → 269 out (259 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3-flash-preview:low__json_schema PASS 100% 193 in → 798 out (781 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3-flash-preview:medium PASS 100% 134 in → 269 out (259 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3-flash-preview:medium__json_schema PASS 100% 134 in → 2,008 out (1,991 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3-flash-preview:xhigh PASS 100% 132 in → 231 out (221 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3-flash-preview:xhigh__json_schema PASS 100% 194 in → 1,907 out (1,888 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
google/gemini-3.1-flash-lite-preview:high PASS 100% 133 in → 344 out (334 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-flash-lite-preview:high__json_schema PASS 100% 134 in → 1,328 out (1,311 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.1-flash-lite-preview:low PASS 100% 133 in → 148 out (138 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-flash-lite-preview:medium PASS 100% 134 in → 202 out (192 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-flash-lite-preview:medium__json_schema PASS 100% 133 in → 512 out (494 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
google/gemini-3.1-flash-lite-preview:xhigh PASS 100% 134 in → 379 out (369 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-flash-lite-preview:xhigh__json_schema PASS 100% 134 in → 1,867 out (1,850 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.1-pro-preview:high PASS 100% 133 in → 503 out (493 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-pro-preview:high__json_schema PASS 100% 193 in → 852 out (835 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.1-pro-preview:low PASS 100% 130 in → 186 out (176 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-pro-preview:low__json_schema PASS 100% 193 in → 546 out (528 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
google/gemini-3.1-pro-preview:medium PASS 100% 132 in → 213 out (203 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-pro-preview:medium__json_schema PASS 100% 194 in → 582 out (565 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.1-pro-preview:xhigh PASS 100% 133 in → 466 out (456 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.1-pro-preview:xhigh__json_schema PASS 100% 192 in → 1,024 out (1,007 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.5-flash-lite:high PASS 100% 133 in → 453 out (443 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash-lite:high__json_schema PASS 100% 133 in → 2,463 out (2,446 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.5-flash-lite:low PASS 100% 134 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash-lite:low__json_schema PASS 100% 192 in → 23 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemini-3.5-flash-lite:max PASS 100% 135 in → 303 out (293 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash-lite:max__json_schema PASS 100% 133 in → 1,035 out (1,018 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
google/gemini-3.5-flash-lite:medium PASS 100% 134 in → 308 out (298 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash-lite:medium__json_schema PASS 100% 135 in → 500 out (477 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemini-3.5-flash-lite:xhigh PASS 100% 133 in → 427 out (417 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash-lite:xhigh__json_schema PASS 100% 135 in → 842 out (819 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemini-3.5-flash:high PASS 100% 134 in → 637 out (627 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash:high__json_schema PASS 100% 191 in → 735 out (717 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
google/gemini-3.5-flash:low PASS 100% 132 in → 196 out (186 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash:low__json_schema PASS 100% 192 in → 209 out (189 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
google/gemini-3.5-flash:medium PASS 100% 133 in → 569 out (559 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash:medium__json_schema PASS 100% 192 in → 686 out (668 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
google/gemini-3.5-flash:xhigh PASS 100% 130 in → 398 out (388 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.5-flash:xhigh__json_schema PASS 100% 191 in → 1,113 out (1,088 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
google/gemini-3.6-flash:high PASS 100% 133 in → 468 out (458 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.6-flash:high__json_schema PASS 100% 134 in → 1,038 out (1,019 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
google/gemini-3.6-flash:low PASS 100% 134 in → 160 out (150 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.6-flash:low__json_schema PASS 100% 194 in → 23 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemini-3.6-flash:max PASS 100% 135 in → 436 out (426 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.6-flash:max__json_schema PASS 100% 194 in → 979 out (959 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
google/gemini-3.6-flash:medium PASS 100% 134 in → 377 out (367 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.6-flash:medium__json_schema PASS 100% 134 in → 743 out (724 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
google/gemini-3.6-flash:xhigh PASS 100% 135 in → 537 out (527 reasoning)
PORT=8443 PROTOCOL=https
google/gemini-3.6-flash:xhigh__json_schema PASS 100% 134 in → 698 out (679 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
google/gemma-4-26b-a4b-it:high PASS 100% 148 in → 291 out (280 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-26b-a4b-it:high__json_schema PASS 100% 150 in → 279 out (206 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-26b-a4b-it:low PASS 100% 150 in → 258 out (247 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-26b-a4b-it:low__json_schema PASS 100% 149 in → 316 out (250 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-26b-a4b-it:medium PASS 100% 150 in → 295 out (284 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-26b-a4b-it:medium__json_schema PASS 100% 149 in → 268 out (196 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-26b-a4b-it:none PASS 100% 147 in → 11 out (0 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-26b-a4b-it:none__json_schema PASS 100% 145 in → 23 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-26b-a4b-it:xhigh PASS 100% 146 in → 259 out (207 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-26b-a4b-it:xhigh__json_schema PASS 100% 149 in → 323 out (259 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-31b-it:high PASS 100% 150 in → 233 out (187 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-31b-it:high__json_schema PASS 100% 147 in → 220 out (161 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-31b-it:low PASS 100% 149 in → 264 out (217 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-31b-it:medium PASS 100% 149 in → 228 out (1 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-31b-it:medium__json_schema PASS 100% 150 in → 260 out (182 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-31b-it:none PASS 100% 146 in → 11 out (0 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-31b-it:none__json_schema PASS 100% 146 in → 24 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
google/gemma-4-31b-it:xhigh PASS 100% 149 in → 232 out (180 reasoning)
PORT=8443 PROTOCOL=https
google/gemma-4-31b-it:xhigh__json_schema PASS 100% 152 in → 295 out (276 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
ibm-granite/granite-4.1-8b PASS 100% 133 in → 9 out
PORT=8443 PROTOCOL=https
ibm-granite/granite-4.1-8b__json_schema PASS 100% 133 in → 19 out
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
inclusionai/ling-2.6-flash PASS 100% 154 in → 11 out
PORT=8443 PROTOCOL=https
inclusionai/ling-2.6-flash__json_schema PASS 100% 153 in → 22 out
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
minimax/minimax-m2.5:high PASS 100% 154 in → 213 out (203 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.5:high__json_schema PASS 100% 169 in → 932 out (914 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
minimax/minimax-m2.5:low PASS 100% 169 in → 167 out (157 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.5:low__json_schema PASS 100% 153 in → 160 out (142 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
minimax/minimax-m2.5:medium PASS 100% 152 in → 126 out (117 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.5:medium__json_schema PASS 100% 156 in → 163 out (146 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
minimax/minimax-m2.5:xhigh PASS 100% 154 in → 108 out (99 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.5:xhigh__json_schema PASS 100% 154 in → 274 out (287 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
minimax/minimax-m2.7:high PASS 100% 155 in → 235 out (224 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.7:high__json_schema PASS 100% 286 in → 342 out (327 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
minimax/minimax-m2.7:low PASS 100% 153 in → 296 out (321 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.7:low__json_schema PASS 100% 288 in → 160 out (145 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
minimax/minimax-m2.7:medium PASS 100% 155 in → 366 out (357 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.7:medium__json_schema PASS 100% 286 in → 316 out (301 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
minimax/minimax-m2.7:xhigh PASS 100% 138 in → 190 out (183 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m2.7:xhigh__json_schema PASS 100% 287 in → 301 out (281 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
minimax/minimax-m3:high PASS 100% 292 in → 147 out (143 reasoning)
``` PORT=8443 PROTOCOL=https ```
minimax/minimax-m3:high__json_schema PASS 100% 240 in → 21 out (18 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
minimax/minimax-m3:low PASS 100% 294 in → 145 out (150 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m3:low__json_schema PASS 100% 278 in → 19 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
minimax/minimax-m3:medium PASS 100% 292 in → 171 out (172 reasoning)
PORT=8443 PROTOCOL=https
minimax/minimax-m3:medium__json_schema PASS 100% 279 in → 19 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
minimax/minimax-m3:xhigh PASS 100% 292 in → 179 out (173 reasoning)
``` PORT=8443 PROTOCOL=https ```
minimax/minimax-m3:xhigh__json_schema PASS 100% 240 in → 20 out (17 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
mistralai/devstral-2512__json_schema PASS 100% 142 in → 19 out
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
mistralai/mistral-medium-3-5:high PASS 100% 152 in → 508 out (480 reasoning)
PORT=8443 PROTOCOL=https
mistralai/mistral-medium-3-5:high__json_schema PASS 100% 142 in → 333 out (313 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
mistralai/mistral-medium-3-5:low PASS 100% 153 in → 361 out (332 reasoning)
PORT=8443 PROTOCOL=https
mistralai/mistral-medium-3-5:low__json_schema PASS 100% 141 in → 324 out (312 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
mistralai/mistral-medium-3-5:medium PASS 100% 154 in → 512 out (459 reasoning)
PORT=8443 PROTOCOL=https
mistralai/mistral-medium-3-5:medium__json_schema PASS 100% 142 in → 230 out (200 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
mistralai/mistral-medium-3-5:none PASS 100% 152 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
mistralai/mistral-medium-3-5:none__json_schema PASS 100% 141 in → 19 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
mistralai/mistral-medium-3-5:xhigh PASS 100% 154 in → 374 out (334 reasoning)
PORT=8443 PROTOCOL=https
mistralai/mistral-medium-3-5:xhigh__json_schema PASS 100% 142 in → 352 out (328 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
moonshotai/kimi-k2.6:high__json_schema PASS 100% 122 in → 986 out (956 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
moonshotai/kimi-k2.6:low PASS 100% 123 in → 504 out (494 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k2.6:low__json_schema PASS 100% 306 in → 561 out (546 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
moonshotai/kimi-k2.6:medium__json_schema PASS 100% 123 in → 310 out (292 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
moonshotai/kimi-k2.6:xhigh__json_schema PASS 100% 123 in → 707 out (693 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
moonshotai/kimi-k2.7-code:high__json_schema PASS 100% 124 in → 138 out (133 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
moonshotai/kimi-k2.7-code:low PASS 100% 123 in → 164 out (154 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k2.7-code:low__json_schema PASS 100% 123 in → 255 out (148 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
moonshotai/kimi-k2.7-code:medium__json_schema PASS 100% 122 in → 150 out (130 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
moonshotai/kimi-k2.7-code:xhigh PASS 100% 121 in → 138 out (128 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k2.7-code:xhigh__json_schema PASS 100% 206 in → 193 out (178 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
moonshotai/kimi-k3:high PASS 100% 199 in → 90 out (67 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:high__json_schema PASS 100% 201 in → 162 out (136 reasoning)
``` PORT=8443 PROTOCOL=https ```
moonshotai/kimi-k3:low PASS 100% 200 in → 95 out (72 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:low__json_schema PASS 100% 200 in → 257 out (234 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:max PASS 100% 199 in → 101 out (78 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:max__json_schema PASS 100% 298 in → 270 out (242 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
moonshotai/kimi-k3:medium PASS 100% 202 in → 189 out (166 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:medium__json_schema PASS 100% 201 in → 110 out (87 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:xhigh PASS 100% 198 in → 216 out (193 reasoning)
PORT=8443 PROTOCOL=https
moonshotai/kimi-k3:xhigh__json_schema PASS 100% 200 in → 163 out (137 reasoning)
``` PORT=8443 PROTOCOL=https ```
nvidia/nemotron-3-nano-30b-a3b:high PASS 100% 154 in → 138 out (125 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-nano-30b-a3b:high__json_schema PASS 100% 154 in → 127 out (107 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-nano-30b-a3b:low PASS 100% 154 in → 66 out (65 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-nano-30b-a3b:low__json_schema PASS 100% 154 in → 174 out (143 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-nano-30b-a3b:medium PASS 100% 154 in → 180 out (167 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-nano-30b-a3b:medium__json_schema PASS 100% 155 in → 160 out (140 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-nano-30b-a3b:none PASS 100% 153 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-nano-30b-a3b:xhigh PASS 100% 154 in → 240 out (230 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-nano-30b-a3b:xhigh__json_schema PASS 100% 155 in → 1,698 out (1,790 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-super-120b-a12b:high PASS 100% 154 in → 182 out (150 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-super-120b-a12b:high__json_schema PASS 100% 155 in → 115 out (79 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-super-120b-a12b:low PASS 100% 155 in → 119 out (103 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-super-120b-a12b:low__json_schema PASS 100% 155 in → 121 out (84 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-super-120b-a12b:medium PASS 100% 153 in → 124 out (93 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-super-120b-a12b:medium__json_schema PASS 100% 154 in → 133 out (94 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-super-120b-a12b:none PASS 100% 153 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-super-120b-a12b:none__json_schema PASS 100% 152 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
nvidia/nemotron-3-super-120b-a12b:xhigh PASS 100% 154 in → 67 out (46 reasoning)
PORT=8443 PROTOCOL=https
nvidia/nemotron-3-super-120b-a12b:xhigh__json_schema PASS 100% 155 in → 200 out (155 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
openai/gpt-5.4-mini:high PASS 100% 119 in → 108 out (94 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-mini:high__json_schema PASS 100% 153 in → 107 out (84 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4-mini:low PASS 100% 121 in → 36 out (22 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-mini:low__json_schema PASS 100% 154 in → 78 out (55 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4-mini:medium PASS 100% 120 in → 49 out (35 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-mini:medium__json_schema PASS 100% 155 in → 78 out (55 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4-mini:none PASS 100% 119 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-mini:none__json_schema PASS 100% 156 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4-mini:xhigh PASS 100% 119 in → 121 out (107 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-mini:xhigh__json_schema PASS 100% 155 in → 233 out (210 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4-nano:high PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-nano:high__json_schema PASS 100% 156 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4-nano:low PASS 100% 119 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-nano:low__json_schema PASS 100% 155 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
openai/gpt-5.4-nano:medium PASS 100% 117 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-nano:medium__json_schema PASS 100% 154 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
openai/gpt-5.4-nano:none PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-nano:none__json_schema PASS 100% 154 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
openai/gpt-5.4-nano:xhigh PASS 100% 118 in → 78 out (64 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4-nano:xhigh__json_schema PASS 100% 156 in → 102 out (78 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https\n"}
openai/gpt-5.4:high PASS 100% 119 in → 95 out (81 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4:high__json_schema PASS 100% 156 in → 104 out (81 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4:low PASS 100% 118 in → 71 out (57 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4:low__json_schema PASS 100% 156 in → 32 out (9 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4:medium PASS 100% 118 in → 82 out (68 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4:medium__json_schema PASS 100% 155 in → 101 out (78 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4:none PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4:none__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.4:xhigh PASS 100% 119 in → 120 out (106 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.4:xhigh__json_schema PASS 100% 155 in → 218 out (195 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.5:high PASS 100% 118 in → 78 out (64 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.5:high__json_schema PASS 100% 154 in → 322 out (299 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.5:low PASS 100% 118 in → 53 out (39 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.5:low__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.5:medium PASS 100% 120 in → 46 out (32 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.5:medium__json_schema PASS 100% 155 in → 209 out (186 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.5:none PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.5:none__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.5:xhigh PASS 100% 117 in → 100 out (86 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.5:xhigh__json_schema PASS 100% 153 in → 539 out (516 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-luna:high PASS 100% 119 in → 41 out (27 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-luna:high__json_schema PASS 100% 155 in → 52 out (29 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-luna:low PASS 100% 119 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-luna:low__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-luna:max PASS 100% 117 in → 49 out (35 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-luna:max__json_schema PASS 100% 154 in → 337 out (314 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-luna:medium PASS 100% 117 in → 34 out (20 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-luna:medium__json_schema PASS 100% 153 in → 47 out (24 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-luna:none PASS 100% 119 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-luna:none__json_schema PASS 100% 155 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-luna:xhigh PASS 100% 118 in → 41 out (27 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-luna:xhigh__json_schema PASS 100% 154 in → 54 out (31 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-sol:high PASS 100% 120 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-sol:high__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-sol:low PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-sol:low__json_schema PASS 100% 155 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-sol:max PASS 100% 120 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-sol:max__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-sol:medium PASS 100% 119 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-sol:medium__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-sol:none PASS 100% 120 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-sol:none__json_schema PASS 100% 157 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-sol:xhigh PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-sol:xhigh__json_schema PASS 100% 156 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-terra:high PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-terra:high__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-terra:low PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-terra:low__json_schema PASS 100% 156 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-terra:max PASS 100% 119 in → 49 out (35 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-terra:max__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-terra:medium PASS 100% 119 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-terra:medium__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-terra:none PASS 100% 117 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-terra:none__json_schema PASS 100% 155 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-5.6-terra:xhigh PASS 100% 118 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-5.6-terra:xhigh__json_schema PASS 100% 154 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-oss-120b:high PASS 100% 178 in → 326 out (368 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-120b:high__json_schema PASS 100% 168 in → 415 out (448 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
openai/gpt-oss-120b:low PASS 100% 178 in → 42 out (27 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-120b:low__json_schema PASS 100% 165 in → 52 out (28 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
openai/gpt-oss-120b:medium PASS 100% 179 in → 88 out (80 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-120b:medium__json_schema PASS 100% 180 in → 21 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
openai/gpt-oss-120b:xhigh PASS 100% 183 in → 449 out (512 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-120b:xhigh__json_schema PASS 100% 167 in → 393 out (428 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
openai/gpt-oss-20b:high PASS 100% 181 in → 114 out (106 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-20b:high__json_schema PASS 100% 179 in → 436 out (465 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
openai/gpt-oss-20b:low PASS 100% 166 in → 38 out (22 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-20b:low__json_schema PASS 100% 177 in → 45 out (25 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-oss-20b:medium PASS 100% 180 in → 104 out (87 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-20b:medium__json_schema PASS 100% 167 in → 86 out (77 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
openai/gpt-oss-20b:xhigh PASS 100% 166 in → 780 out (868 reasoning)
PORT=8443 PROTOCOL=https
openai/gpt-oss-20b:xhigh__json_schema PASS 100% 178 in → 340 out (378 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
poolside/laguna-m.1:high PASS 100% 142 in → 390 out (377 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-m.1:high__json_schema PASS 100% 141 in → 372 out (355 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
poolside/laguna-m.1:low PASS 100% 143 in → 525 out (512 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-m.1:low__json_schema PASS 100% 143 in → 424 out (400 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
poolside/laguna-m.1:medium PASS 100% 143 in → 479 out (466 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-m.1:medium__json_schema PASS 100% 141 in → 1,107 out (1,084 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
poolside/laguna-m.1:none PASS 100% 141 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-m.1:none__json_schema PASS 100% 141 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
poolside/laguna-m.1:xhigh PASS 100% 143 in → 552 out (539 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-m.1:xhigh__json_schema PASS 100% 141 in → 744 out (721 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
poolside/laguna-xs-2.1:high PASS 100% 143 in → 398 out (387 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs-2.1:high__json_schema PASS 100% 143 in → 297 out (274 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
poolside/laguna-xs-2.1:low PASS 100% 141 in → 218 out (207 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs-2.1:low__json_schema PASS 100% 141 in → 384 out (362 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
poolside/laguna-xs-2.1:medium PASS 100% 141 in → 378 out (367 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs-2.1:medium__json_schema PASS 100% 142 in → 438 out (417 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
poolside/laguna-xs-2.1:none PASS 100% 141 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs-2.1:none__json_schema PASS 100% 142 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
poolside/laguna-xs-2.1:xhigh PASS 100% 141 in → 395 out (384 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs-2.1:xhigh__json_schema PASS 100% 142 in → 369 out (351 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
poolside/laguna-xs.2:high PASS 100% 180 in → 187 out (174 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs.2:high__json_schema PASS 100% 180 in → 228 out (210 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
poolside/laguna-xs.2:low PASS 100% 180 in → 217 out (204 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs.2:low__json_schema PASS 100% 178 in → 235 out (216 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
poolside/laguna-xs.2:medium PASS 100% 178 in → 159 out (146 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs.2:medium__json_schema PASS 100% 179 in → 183 out (165 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
poolside/laguna-xs.2:none PASS 100% 179 in → 12 out (0 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs.2:none__json_schema PASS 100% 179 in → 17 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
poolside/laguna-xs.2:xhigh PASS 100% 176 in → 201 out (188 reasoning)
PORT=8443 PROTOCOL=https
poolside/laguna-xs.2:xhigh__json_schema PASS 100% 178 in → 218 out (200 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
qwen/qwen3.6-27b:high PASS 100% 135 in → 1,165 out (1,127 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-27b:high__json_schema PASS 100% 137 in → 1,130 out (1,089 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-27b:low PASS 100% 139 in → 874 out (848 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-27b:low__json_schema PASS 100% 139 in → 1,073 out (1 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-27b:medium PASS 100% 139 in → 1,390 out (1,358 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-27b:medium__json_schema PASS 100% 138 in → 978 out (1 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-27b:xhigh PASS 100% 137 in → 1,062 out (1,033 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-27b:xhigh__json_schema PASS 100% 140 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-35b-a3b:high PASS 100% 139 in → 829 out (789 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-35b-a3b:high__json_schema PASS 100% 136 in → 799 out (754 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-35b-a3b:low PASS 100% 138 in → 1,029 out (912 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-35b-a3b:low__json_schema PASS 100% 136 in → 968 out (957 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=8443\nPROTOCOL=https"}
qwen/qwen3.6-35b-a3b:medium PASS 100% 136 in → 1,006 out (973 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-35b-a3b:medium__json_schema PASS 100% 139 in → 489 out (456 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-35b-a3b:none PASS 100% 141 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-35b-a3b:none__json_schema PASS 100% 141 in → 22 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-35b-a3b:xhigh PASS 100% 137 in → 377 out (333 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-35b-a3b:xhigh__json_schema PASS 100% 138 in → 825 out (788 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
qwen/qwen3.6-flash:high PASS 100% 139 in → 1,165 out (1,150 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-flash:low PASS 100% 139 in → 895 out (880 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-flash:medium PASS 100% 138 in → 1,109 out (1,094 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-flash:none PASS 100% 141 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.6-flash:xhigh PASS 100% 138 in → 807 out (792 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:high PASS 100% 139 in → 1,072 out (1,057 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:low PASS 100% 138 in → 928 out (913 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:max PASS 100% 137 in → 993 out (978 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:medium PASS 100% 136 in → 873 out (858 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:none PASS 100% 140 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:none__json_schema PASS 100% 139 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-flash:xhigh PASS 100% 138 in → 584 out (569 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-max:high PASS 100% 138 in → 277 out (262 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-max:low PASS 100% 136 in → 290 out (275 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-max:medium PASS 100% 136 in → 390 out (375 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-max:none PASS 100% 139 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-max:xhigh PASS 100% 137 in → 376 out (361 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-plus:high PASS 100% 138 in → 653 out (638 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-plus:low PASS 100% 139 in → 639 out (624 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-plus:medium PASS 100% 137 in → 405 out (386 reasoning)
``` PORT=8443 PROTOCOL=https ```
qwen/qwen3.7-plus:none PASS 100% 139 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
qwen/qwen3.7-plus:xhigh PASS 100% 139 in → 195 out (180 reasoning)
PORT=8443 PROTOCOL=https
tencent/hy3:high PASS 100% 133 in → 457 out (449 reasoning)
PORT=8443 PROTOCOL=https
tencent/hy3:high__json_schema PASS 100% 135 in → 428 out (401 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
tencent/hy3:low PASS 100% 138 in → 402 out (391 reasoning)
PORT=8443 PROTOCOL=https
tencent/hy3:low__json_schema PASS 100% 135 in → 326 out (292 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
tencent/hy3:medium PASS 100% 136 in → 336 out (299 reasoning)
PORT=8443 PROTOCOL=https
tencent/hy3:medium__json_schema PASS 100% 134 in → 468 out (423 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
tencent/hy3:none PASS 100% 137 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
tencent/hy3:none__json_schema PASS 100% 139 in → 20 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
tencent/hy3:xhigh PASS 100% 135 in → 725 out (714 reasoning)
PORT=8443 PROTOCOL=https
tencent/hy3:xhigh__json_schema PASS 100% 135 in → 368 out (352 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling-small:high PASS 100% 128 in → 306 out (292 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling-small:high__json_schema PASS 100% 127 in → 271 out (249 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling-small:low PASS 100% 129 in → 40 out (30 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling-small:low__json_schema PASS 100% 128 in → 97 out (75 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
thinkingmachines/inkling-small:max PASS 100% 128 in → 242 out (244 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling-small:max__json_schema PASS 100% 128 in → 335 out (313 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling-small:medium PASS 100% 126 in → 119 out (113 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling-small:medium__json_schema PASS 100% 127 in → 223 out (202 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling-small:none PASS 100% 127 in → 11 out (0 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling-small:xhigh PASS 100% 127 in → 636 out (618 reasoning)
--- config.txt PORT=8443 PROTOCOL=https
thinkingmachines/inkling-small:xhigh__json_schema PASS 100% 128 in → 208 out (187 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling:high PASS 100% 128 in → 443 out (428 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling:high__json_schema PASS 100% 128 in → 177 out (155 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling:low PASS 100% 128 in → 30 out (15 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling:max PASS 100% 128 in → 347 out (332 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling:max__json_schema PASS 100% 128 in → 243 out (221 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
thinkingmachines/inkling:medium PASS 100% 128 in → 170 out (156 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling:none PASS 100% 127 in → 11 out (0 reasoning)
PORT=8443 PROTOCOL=https
thinkingmachines/inkling:none__json_schema PASS 100% 126 in → 17 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https"}
thinkingmachines/inkling:xhigh PASS 100% 129 in → 328 out (314 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:high PASS 100% 298 in → 217 out (209 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:high__json_schema PASS 100% 353 in → 474 out (457 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
x-ai/grok-4.3:low PASS 100% 303 in → 275 out (267 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:low__json_schema PASS 100% 361 in → 366 out (349 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
x-ai/grok-4.3:max PASS 100% 298 in → 244 out (236 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:max__json_schema PASS 100% 355 in → 502 out (485 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
x-ai/grok-4.3:medium PASS 100% 303 in → 310 out (302 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:medium__json_schema PASS 100% 359 in → 445 out (428 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
x-ai/grok-4.3:none PASS 100% 294 in → 8 out (0 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:xhigh PASS 100% 297 in → 258 out (250 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.3:xhigh__json_schema PASS 100% 355 in → 418 out (401 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
x-ai/grok-4.5:high PASS 100% 320 in → 224 out (216 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.5:high__json_schema PASS 100% 389 in → 279 out (258 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
x-ai/grok-4.5:low PASS 100% 319 in → 171 out (163 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.5:low__json_schema PASS 100% 388 in → 216 out (195 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
x-ai/grok-4.5:max PASS 100% 321 in → 216 out (208 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.5:max__json_schema PASS 100% 389 in → 281 out (260 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
x-ai/grok-4.5:medium PASS 100% 321 in → 190 out (182 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.5:medium__json_schema PASS 100% 389 in → 311 out (292 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
x-ai/grok-4.5:xhigh PASS 100% 320 in → 180 out (172 reasoning)
PORT=8443 PROTOCOL=https
x-ai/grok-4.5:xhigh__json_schema PASS 100% 388 in → 537 out (521 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
z-ai/glm-4.7-flash:high PASS 100% 121 in → 372 out (350 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-4.7-flash:high__json_schema PASS 100% 124 in → 404 out (352 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
z-ai/glm-4.7-flash:low PASS 100% 123 in → 408 out (381 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-4.7-flash:medium PASS 100% 124 in → 403 out (382 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-4.7-flash:medium__json_schema PASS 100% 122 in → 397 out (354 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
z-ai/glm-4.7-flash:none PASS 100% 124 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-4.7-flash:none__json_schema PASS 100% 124 in → 18 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
z-ai/glm-4.7-flash:xhigh PASS 100% 124 in → 398 out (387 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-5.2:high PASS 100% 129 in → 360 out (349 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-5.2:high__json_schema PASS 100% 131 in → 16 out (15 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https"}
z-ai/glm-5.2:low PASS 100% 131 in → 451 out (440 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-5.2:low__json_schema PASS 100% 131 in → 217 out (193 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
z-ai/glm-5.2:medium PASS 100% 130 in → 413 out (402 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-5.2:medium__json_schema PASS 100% 123 in → 413 out (356 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
z-ai/glm-5.2:none PASS 100% 125 in → 10 out (0 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-5.2:none__json_schema PASS 100% 125 in → 19 out (0 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\nPROTOCOL=https" }
z-ai/glm-5.2:xhigh PASS 100% 129 in → 425 out (414 reasoning)
PORT=8443 PROTOCOL=https
z-ai/glm-5.2:xhigh__json_schema PASS 100% 130 in → 616 out (593 reasoning)
PORT=8443 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content": "PORT=8443\nPROTOCOL=https\n"}
anthropic/claude-opus-4.6:high FAIL 0% 143 in → 329 out (174 reasoning)
``` PORT=8443 PROTOCOL=https ``` **Reasoning:** | Setting | Base | HEAD (main) | Incoming (cherry-pick) | Resolution | |------------|--------|-------------|------------------------|------------| | `PORT` | 8080 | 9090 | 8443 | **8443** — both sides changed it (true conflict); prefer HTTPS config | | `PROTOCOL` | http | http | https | **https** — only incoming changed it, and it aligns with the HTTPS preference |
Failure: Content mismatch: ['config.txt']
anthropic/claude-opus-4.6:max FAIL 0% 143 in → 368 out (215 reasoning)
``` PORT=8443 PROTOCOL=https ``` **Reasoning:** | Field | Base | HEAD (main) | Incoming | Resolution | |---|---|---|---|---| | `PORT` | 8080 | 9090 | 8443 | **8443** — Both branches changed PORT (true conflict). Prefer HTTPS config → pick `8443`. | | `PROTOCOL` | http | http | https | **https** — Only incoming changed it, and it aligns with the HTTPS preference → pick `https`. |
Failure: Content mismatch: ['config.txt']
anthropic/claude-opus-4.6:xhigh FAIL 0% 142 in → 389 out (163 reasoning)
``` PORT=8443 PROTOCOL=https ``` **Reasoning:** | Setting | Base | HEAD (main) | Incoming (cherry-pick) | Resolved | |------------|--------|-------------|------------------------|----------| | `PORT` | 8080 | 9090 | 8443 | **8443** | | `PROTOCOL` | http | http | https | **https**| Both lines conflict (both sides changed from base). Per the instruction to **prefer the HTTPS configuration**, the incoming cherry-picked commit's values (`PORT=8443`, `PROTOCOL=https`) are chosen, as they represent the cohesive HTTPS setup. Mixing values (e.g., `PORT=9090` with `PROTOCOL=https`) would be semantically inconsistent since port 8443 is the standard HTTPS port paired with that protocol.
Failure: Content mismatch: ['config.txt']
anthropic/claude-sonnet-4.6:low FAIL 0% 144 in → 16 out (0 reasoning)
``` PORT=9090 PROTOCOL=https ```
Failure: Content mismatch: ['config.txt']
anthropic/claude-sonnet-4.6:low__json_schema FAIL 0% 312 in → 26 out (0 reasoning)
PORT=9090 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=9090\nPROTOCOL=https\n"}
Failure: Content mismatch: ['config.txt']
anthropic/claude-sonnet-4.6:none__json_schema FAIL 0% 311 in → 26 out (0 reasoning)
PORT=9090 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=9090\nPROTOCOL=https\n"}
Failure: Content mismatch: ['config.txt']
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: { { "resolved_content": "PORT=8443\nPROTOCOL=https\n" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
Failure: Failed to parse structured JSON response: Expecting property name enclosed in double quotes: line 2 column 1 (char 2)
Invalid JSON. Output: PORT=8443 PROTOCOL=https
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
google/gemini-3.1-flash-lite-preview:low__json_schema FAIL 0% 193 in → 179 out (158 reasoning)
PORT=8443PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443PROTOCOL=https" }
Failure: Content mismatch: ['config.txt']
Invalid JSON. Output: PORT=8{ "resolved_content": "PORT=8443\nPROTOCOL=https" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
liquid/lfm-2-24b-a2b FAIL 0% 150 in → 29 out
Here’s the resolved `config.txt` with HTTPS preferred: ``` PORT=8443 PROTOCOL=https ```
Failure: Content mismatch: ['config.txt']
Invalid JSON. Output: PORT=8443 PROTOCOL=https
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
mistralai/devstral-2512 FAIL 0% 141 in → 30 out
Here's the resolved `config.txt` content preferring the HTTPS configuration: ``` PORT=8443 PROTOCOL=https ```
Failure: Content mismatch: ['config.txt']
moonshotai/kimi-k2.6:high FAIL 0% 124 in → 241 out (269 reasoning)
PORT=8443 PROTOCOL=https
Failure: Content mismatch: ['config.txt']
moonshotai/kimi-k2.6:medium FAIL 0% 122 in → 481 out (471 reasoning)
PORT=8443 PROTOCOL=https
Failure: Content mismatch: ['config.txt']
moonshotai/kimi-k2.6:xhigh FAIL 0% 122 in → 283 out (308 reasoning)
PORT=8443 PROTOCOL=https
Failure: Content mismatch: ['config.txt']
moonshotai/kimi-k2.7-code:high FAIL 0% 124 in → 133 out (123 reasoning)
PORT=8443 PROTOCOL=https
Failure: Content mismatch: ['config.txt']
moonshotai/kimi-k2.7-code:medium FAIL 0% 123 in → 150 out (140 reasoning)
PORT=8443 PROTOCOL=https
Failure: Content mismatch: ['config.txt']
nvidia/nemotron-3-nano-30b-a3b:none__json_schema FAIL 0% 153 in → 27 out (0 reasoning)
PORT=8443\nPROTOCOL=https\n
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=8443\\nPROTOCOL=https\\n" }
Failure: Content mismatch: ['config.txt']
qwen/qwen3.6-27b:none FAIL 0% 141 in → 11 out (0 reasoning)
PORT=9090 PROTOCOL=https
Failure: Content mismatch: ['config.txt']
qwen/qwen3.6-27b:none__json_schema FAIL 0% 141 in → 25 out (0 reasoning)
PORT=9090 PROTOCOL=https
JSON Schema Structured Output
(raw) { "resolved_content": "PORT=9090\nPROTOCOL=https" }
Failure: Content mismatch: ['config.txt']
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4dc500b0-7f6e-9bdd-804e-c026d97a776f","request_id":"4dc500b0-7f6e-9bdd-804e-c026d97a776f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-28f8ccad-cb3c-9a05-8b1b-cb658b3f8717","request_id":"28f8ccad-cb3c-9a05-8b1b-cb658b3f8717"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-92671406-8445-9eb6-8390-1fcd0ef6aa03","request_id":"92671406-8445-9eb6-8390-1fcd0ef6aa03"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7d1c9a52-f9fb-964a-8037-13fead1f183e","request_id":"7d1c9a52-f9fb-964a-8037-13fead1f183e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-83e8c816-4e97-9cdd-b257-874cf822a6e1","request_id":"83e8c816-4e97-9cdd-b257-874cf822a6e1"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-50a3d16c-2304-934f-923a-d4be9c840ff0"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-ae4e8efd-3e09-9467-959f-718b9fd04aa7"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-44def87b-6a52-97dc-9bdd-7e5f8bd6c9da"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-83ac853f-b2d8-9f84-834d-120d2aaeea5e"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-580e7a06-cff8-9627-98f8-0f0941052ac1"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-34ea3b23-94e1-986d-9ef9-b3c1920b0e62","request_id":"34ea3b23-94e1-986d-9ef9-b3c1920b0e62"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-9c509ce2-9432-942e-b923-72f7e034814a","request_id":"9c509ce2-9432-942e-b923-72f7e034814a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-ae75d40b-d424-90d3-bd3f-9c02d7ce39ca","request_id":"ae75d40b-d424-90d3-bd3f-9c02d7ce39ca"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-daefb73d-3dc2-9499-8497-5971ffe7c9df","request_id":"daefb73d-3dc2-9499-8497-5971ffe7c9df"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7146aea5-3652-9fc2-ba6e-adf63615a2b7","request_id":"7146aea5-3652-9fc2-ba6e-adf63615a2b7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-d2a460bf-b80b-93af-af83-cd779be36727","request_id":"d2a460bf-b80b-93af-af83-cd779be36727"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6dd7b3f7-de8d-9ba0-902d-63609acf4f4d","request_id":"6dd7b3f7-de8d-9ba0-902d-63609acf4f4d"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-41dd0460-6c6e-9fd1-b81d-209b76000555","request_id":"41dd0460-6c6e-9fd1-b81d-209b76000555"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0f4ee425-6208-96bf-8da9-a44b3bdd8b84","request_id":"0f4ee425-6208-96bf-8da9-a44b3bdd8b84"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b35f0436-0333-9d2f-8e37-0b200d03fb94","request_id":"b35f0436-0333-9d2f-8e37-0b200d03fb94"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
Invalid JSON. Output: PORT=8443 PROTOCOL=https{ "resolved_content": "PORT=8443\nPROTOCOL=https" }
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: PORT=8443 PROTOCOL=https
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: PORT=8443 PROTOCOL=https
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: PORT=8443 PROTOCOL=https
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
x-ai/grok-4.3:none__json_schema FAIL 0% 353 in → 13 out (0 reasoning)
PORT=9090 PROTOCOL=https
JSON Schema Structured Output
(raw) {"resolved_content":"PORT=9090\nPROTOCOL=https"}
Failure: Content mismatch: ['config.txt']
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)