Show file content at a specific revision
Tests ability to show file content at a specific revision. Evaluates historical-file-content retrieval.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
echo 'version one' > doc.txt - 05
git add doc.txt - 06
git commit -m 'First version' - 07
FIRST=$(git rev-parse HEAD) - 08
echo 'version two' > doc.txt - 09
git add doc.txt - 10
git commit -m 'Second version'
Prompt
Using git show HEAD~1:doc.txt, what was the content of doc.txt in the first commit? Output ONLY the file content, nothing else.
Expected
version one
Campaign Evidence
Loading campaign evidence…
Model Outputs (14)
version one
version one
JSON Schema
Structured Output
(raw) { "content": "version one" }
version one
JSON Schema
Structured Output
(raw) {
"content": "version one"
}
version one
version one
JSON Schema
Structured Output
(raw) {"content": "version one"}
version one
version one
JSON Schema
Structured Output
(raw) {
"content": "version one"
}
version one
version one
JSON Schema
Structured Output
(raw) {
"content": "version one\n"
}
version one
version one
JSON Schema
Structured Output
(raw) {"content":"version one\n"}
(empty output)
Failure: Model 'deepseek/deepseek-v4-flash:none' violated the reasoning_level=none invariant: provider reported 84 reasoning token(s)
version two
Failure: Expected 'version one', got 'version two'
version two
JSON Schema
Structured Output
(raw) {
"content": "version two"
}
Failure: Expected 'version one', got 'version two'