Search across branches using git grep on a specific ref
Tests ability to search across branches using git grep on a specific ref. Evaluates revision-scoped search.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
echo 'version = 1.0' > version.txt - 05
git add version.txt - 06
git commit -m 'Initial version 1.0' - 07
git checkout -b feature/v2 - 08
echo 'version = 2.0' > version.txt - 09
git add version.txt - 10
git commit -m 'Bump to version 2.0' - 11
git checkout main - 12
echo 'git grep version feature/v2' > .grep_command - 13
git add .grep_command - 14
git commit -m 'Add grep sentinel'
Prompt
Here is the output of a git grep command run against the feature/v2 branch. What version number is found? Output ONLY the version number, nothing else.
Expected
2.0
Fixture Metrics
Scoped model quality, cost, API time, and token usage for git_grep/f009.
Loading...
Raw Attempt Evidence
Loading raw attempt evidence…
Model Outputs (546)
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
2.0
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw)
{"version_number": "2.0"}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number" : "2.0" }
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number":"2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number"
: "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number":"2.0"}
2.0
2.0
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) {"version_number": "2.0"}
2.0
2.0
JSON Schema
Structured Output
(raw) {
"version_number": "2.0"
}
2.0
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0" }
2.0
2.0
JSON Schema
Structured Output
(raw) { "version_number": "2.0"}
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
(empty output)
Failure: Model 'google/gemma-4-31b-it:none' violated the reasoning_level=none invariant: provider reported 112 reasoning token(s)
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
final2.0
JSON Schema
Structured Output
(raw) {"version_number":"final2.0"}
Failure: Expected '2.0', got 'final2.0'
"
JSON Schema
Structured Output
(raw) {
"version_number": "\""
}
Failure: Expected '2.0', got '"'
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-22e02355-8ca2-9163-b41b-b64c0d772901","request_id":"22e02355-8ca2-9163-b41b-b64c0d772901"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-4bd6912d-cdc1-98f1-b283-78d854b2ab83","request_id":"4bd6912d-cdc1-98f1-b283-78d854b2ab83"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c8cdfa7d-8367-98dc-89ab-04805c801dc4","request_id":"c8cdfa7d-8367-98dc-89ab-04805c801dc4"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-b78706e8-7622-9d48-861d-b420e9b1820e","request_id":"b78706e8-7622-9d48-861d-b420e9b1820e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a3c84243-44c6-9641-8041-c885d3605c2d","request_id":"a3c84243-44c6-9641-8041-c885d3605c2d"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-acc9d096-757f-939b-a133-b0e8bca34443"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-6df18c4d-5e9e-93ee-9796-13106dd37caf"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-28c35f91-7bc5-9d16-ad0f-e0b40ce4d3d7"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-c47c45bb-4e85-9c84-83d2-35165f0e09ae"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-3ddbc57f-8af3-9de6-b621-a4e4901b461d"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-685589af-9f96-9a24-a1ff-bfc5a54cb270"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-7d85e4e4-f365-9ee7-8f87-a878a4a9869f","request_id":"7d85e4e4-f365-9ee7-8f87-a878a4a9869f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-386a4428-eb0a-999f-a83d-5e5fd122dec0","request_id":"386a4428-eb0a-999f-a83d-5e5fd122dec0"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f9c3bded-787a-9fe4-8bf6-d98d935decc7","request_id":"f9c3bded-787a-9fe4-8bf6-d98d935decc7"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0f05b5f0-958c-908b-83e0-0f962ebd5533","request_id":"0f05b5f0-958c-908b-83e0-0f962ebd5533"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-77e2599b-92c7-9135-8c1a-e1e9165de7b2","request_id":"77e2599b-92c7-9135-8c1a-e1e9165de7b2"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c3a4aa50-f12a-9fb9-ad45-795461229b35","request_id":"c3a4aa50-f12a-9fb9-ad45-795461229b35"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-49e8c85b-2836-968e-a36d-5f68fb373eb9","request_id":"49e8c85b-2836-968e-a36d-5f68fb373eb9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-602ce384-6c21-9943-b184-81925e5c291e","request_id":"602ce384-6c21-9943-b184-81925e5c291e"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-0ba9b9a0-c772-96ce-90f8-4f4ae520a284","request_id":"0ba9b9a0-c772-96ce-90f8-4f4ae520a284"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-36fd88ff-49d8-9aba-bcdc-ccdd13a66b91","request_id":"36fd88ff-49d8-9aba-bcdc-ccdd13a66b91"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
2.0<|end_message|>
Failure: Expected '2.0', got '2.0<|end_message|>'
Invalid JSON. Output: 2.0{
"version_number": "2.0"
}
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 4 (char 3)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 4 (char 3)
2.0<|end_message|>
Failure: Expected '2.0', got '2.0<|end_message|>'
Invalid JSON. Output: 2.0{
"version_number": "2.0"
}
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 4 (char 3)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 4 (char 3)
2.0
The git grep output directly shows the string "version = 2.0" on the feature/v2 branch inside version.txt, which is the version number requested. No parsing, extraction logic, or additional commands are needed; the numeric value is already explicit in the provided line.
Failure: Expected '2.0', got '2.0
The git grep output directly shows the string "version = 2.0" on the feature/v2 branch inside version.txt, which is the version number requested. No parsing, extraction logic, or additional commands are needed; the numeric value is already explicit in the provided line.'
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid structured output. Output: 2.0
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object