Show which files changed in a specific commit
Tests ability to identify which files changed in a specific commit. Evaluates file-list extraction from git show.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
echo 'alpha' > a.txt - 05
echo 'beta' > b.txt - 06
git add a.txt b.txt - 07
git commit -m 'Add both files' - 08
echo 'changed' > a.txt - 09
git add a.txt - 10
git commit -m 'Modify only a.txt' - 11
echo 'changed' >> b.txt - 12
git add b.txt - 13
git commit -m 'Modify only b.txt'
Prompt
Using git show --stat, which file was modified in the commit with message 'Modify only a.txt'? Output ONLY the filename, nothing else.
Expected
a.txt
Campaign Evidence
Loading campaign evidence…
Model Outputs (14)
a.txt
JSON Schema
Structured Output
(raw) {"filename": "a.txt"}
a.txt
a.txt
JSON Schema
Structured Output
(raw) { "filename": "a.txt" }
a.txt
a.txt
JSON Schema
Structured Output
(raw) {
"filename": "a.txt"
}
b.txt
Failure: Expected 'a.txt', got 'b.txt'
The commit with the message "Modify only a.txt" is not shown in the provided output. The only commit shown is for "Modify only b.txt" which modified `b.txt`.
If you're looking for the commit with message "Modify only a.txt", you would need to check the full git history. Based on the information given, there is no commit with that message in the provided output.
However, if you're asking about the commit shown (which modifies b.txt), the file modified is:
`b.txt`
Failure: Expected 'a.txt', got 'The commit with the message "Modify only a.txt" is not shown in the provided output. The only commit shown is for "Modify only b.txt" which modified `b.txt`.
If you're looking for the commit with message "Modify only a.txt", you would need to check the full git history. Based on the information given, there is no commit with that message in the provided output.
However, if you're asking about the commit shown (which modifies b.txt), the file modified is:
`b.txt`'
b.txt
JSON Schema
Structured Output
(raw) {"filename": "b.txt"}
Failure: Expected 'a.txt', got 'b.txt'
b.txt
Failure: Expected 'a.txt', got 'b.txt'
b.txt
JSON Schema
Structured Output
(raw) {
"filename": "b.txt"
}
Failure: Expected 'a.txt', got 'b.txt'
I don't see a commit with message 'Modify only a.txt' in the provided information. The commit shown has the message 'Modify only b.txt' and modified the file `b.txt`.
Failure: Expected 'a.txt', got '
I don't see a commit with message 'Modify only a.txt' in the provided information. The commit shown has the message 'Modify only b.txt' and modified the file `b.txt`.
'
b.txt
JSON Schema
Structured Output
(raw) {
"filename": "b.txt"
}
Failure: Expected 'a.txt', got 'b.txt'
I don't see a commit with the message 'Modify only a.txt' in the provided commits. The only commit shown has the message 'Modify only b.txt' and modified file b.txt.
Failure: Expected 'a.txt', got '
I don't see a commit with the message 'Modify only a.txt' in the provided commits. The only commit shown has the message 'Modify only b.txt' and modified file b.txt.
'
b.txt
JSON Schema
Structured Output
(raw) {"filename": "b.txt"}
Failure: Expected 'a.txt', got 'b.txt'