Show diff between two commits
Tests ability to show diff between two commits. Evaluates comparative-commit inspection.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
printf 'line1 line2 line3 ' > file.txt - 05
git add file.txt - 06
git commit -m 'Original three lines' - 07
printf 'line1 modified line3 ' > file.txt - 08
git add file.txt - 09
git commit -m 'Changed line2'
Prompt
Using git show --stat, how many files were changed in the commit 'Changed line2'? Output ONLY the number, nothing else.
Expected
1
Fixture Metrics
Scoped model quality, cost, API time, and token usage for git_show/f006.
Loading...
Raw Attempt Evidence
Loading raw attempt evidence…
Model Outputs (546)
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
1
1
1
1
JSON Schema
Structured Output
(raw) {
"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw)
{ "count": 1 }
1
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
1
1
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
1
1
JSON Schema
Structured Output
(raw) {"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) {"count":1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {"count": 1}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) {
"count": 1
}
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
1
1
JSON Schema
Structured Output
(raw) { "count": 1 }
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output: {"{
"count":1}
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Invalid control character at: line 1 column 4 (char 3)
Failure: Failed to parse structured JSON response: Invalid control character at: line 1 column 4 (char 3)
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
(empty output)
Failure: Expected '1', got ''
(empty output)
Failure: Expected '1', got ''
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
1文件数量
1
Failure: Expected '1', got '1文件数量
1'
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-f4182d48-6910-96ff-b89a-9c4986d675be","request_id":"f4182d48-6910-96ff-b89a-9c4986d675be"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-30dbadad-632e-96d9-ac0e-10efcb07ee00","request_id":"30dbadad-632e-96d9-ac0e-10efcb07ee00"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a6ef5a6a-22fb-9988-b206-8bf8ebc5e895","request_id":"a6ef5a6a-22fb-9988-b206-8bf8ebc5e895"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-1cd62f6d-650d-91d3-8671-ed41f0837993","request_id":"1cd62f6d-650d-91d3-8671-ed41f0837993"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a9d8e9f2-33c2-986d-9681-abfee2812d21","request_id":"a9d8e9f2-33c2-986d-9681-abfee2812d21"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-08887dc0-f45a-9da3-8a88-ef951cd82195"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-095d1c35-5b76-957e-be6e-928649c4ea28"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-348a3244-730a-9049-a487-f6c66ccf04b5"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-299c662f-7d93-9c40-91f4-b04dffc52b6b"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-1962be44-7644-96e6-9a2e-0bf25d50ff65"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': 'data: {"error":{"code":"invalid_parameter_error","param":null,"message":"\'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error"},"id":"chatcmpl-909b7320-1a1a-9519-9d9f-6f89edaf0d56"}\n\n', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c2c53902-cdd9-97cc-90a3-8f3aad80b77c","request_id":"c2c53902-cdd9-97cc-90a3-8f3aad80b77c"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8861ee77-5a93-9f65-82a0-c09560d8b6ca","request_id":"8861ee77-5a93-9f65-82a0-c09560d8b6ca"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-471141a7-b6cf-94c5-9e18-4685565ea1cc","request_id":"471141a7-b6cf-94c5-9e18-4685565ea1cc"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-3d0bfdb5-7d62-95de-b08f-422c98e9fe8a","request_id":"3d0bfdb5-7d62-95de-b08f-422c98e9fe8a"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-dd8b8d34-c485-9e4c-b0bd-c02d2b9f3c46","request_id":"dd8b8d34-c485-9e4c-b0bd-c02d2b9f3c46"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-6d063a89-f807-9aed-adbc-49d7bdc7124b","request_id":"6d063a89-f807-9aed-adbc-49d7bdc7124b"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-8df1577f-5d6b-9bb0-9c0c-eec8156d77b9","request_id":"8df1577f-5d6b-9bb0-9c0c-eec8156d77b9"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-a51fbb26-439d-9332-81dc-e47641c376ec","request_id":"a51fbb26-439d-9332-81dc-e47641c376ec"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-ac78eefd-7677-9066-98ea-2cf16aeeb51f","request_id":"ac78eefd-7677-9066-98ea-2cf16aeeb51f"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
(empty output)
JSON Schema
Structured Output
Failure: Error code: 400 - {'error': {'message': 'Provider returned error', 'code': 400, 'metadata': {'raw': '{"error":{"message":"<400> InternalError.Algo.InvalidParameter: \'messages\' must contain the word \'json\' in some form, to use \'response_format\' of type \'json_object\'.","type":"invalid_request_error","param":null,"code":"invalid_parameter_error"},"id":"chatcmpl-c40e2a66-922d-924c-a2dd-219feff41c09","request_id":"c40e2a66-922d-924c-a2dd-219feff41c09"}', 'provider_name': 'Alibaba', 'is_byok': False}}, 'user_id': 'user_3FrVBkmFqVIuZ58dEh7AU8vhXUV'}
1<|end_message|>
Failure: Expected '1', got '1<|end_message|>'
Invalid JSON. Output: 1{ "count": 1 }
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
Invalid JSON. Output: 1{
"count": 1
}
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object
1
The commit message 'Changed line2' matches the single provided commit, and its --stat output explicitly indicates "1 file changed". No other files appear in the diff or stat summary. Therefore the count of changed files is exactly 1.
Failure: Expected '1', got '1
The commit message 'Changed line2' matches the single provided commit, and its --stat output explicitly indicates "1 file changed". No other files appear in the diff or stat summary. Therefore the count of changed files is exactly 1.'
1
The provided output is exactly what `git show --stat` (or `git show --stat 6560d099`) emits for the commit whose subject is "Changed line2".
The line
1 file changed, 1 insertion(+), 1 deletion(-)
directly answers the question; therefore the numeric value that must be emitted is 1.
Failure: Expected '1', got '1
The provided output is exactly what `git show --stat` (or `git show --stat 6560d099`) emits for the commit whose subject is "Changed line2".
The line
1 file changed, 1 insertion(+), 1 deletion(-)
directly answers the question; therefore the numeric value that must be emitted is 1.'
Invalid JSON. Output:
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Failure: Failed to parse structured JSON response: Expecting value: line 1 column 1 (char 0)
Invalid structured output. Output: 1
JSON Schema
Structured Output
Structured Output Error
Structured output schema validation failed: $ must be of type object
Failure: Structured output schema validation failed: $ must be of type object