Bad commit changes index.js and styles.css - model identifies multi-file bad commit
Tests ability to identify a multi-file bad commit (index.js + styles.css) via bisect. Evaluates bisect for frontend multi-file regressions.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
echo '#!/bin/bash' > test.sh - 05
echo 'grep -q "color: blue" styles.css && grep -q "blue" index.js' >> test.sh - 06
chmod +x test.sh - 07
echo 'body { color: blue; }' > styles.css && echo 'document.body.style.color = "blue";' > index.js - 08
git add test.sh styles.css index.js && git commit -m 'Initial commit' - 09
echo 'body { background: white; }' >> styles.css - 10
git add styles.css index.js && git commit -m 'add blue styling' - 11
echo 'body { color: red; }' > styles.css && echo 'document.body.style.color = "red";' > index.js - 12
git add styles.css index.js && git commit -m 'change to red' - 13
echo 'document.title = "Theme";' >> index.js - 14
git add index.js && git commit -m 'add theme title'
Prompt
You are debugging a git repository. The following commit history has one bad commit that broke the build. Identify the bad commit hash (first 7 characters) or commit subject line that introduced the bug. Use the git log and test output to determine which commit is bad.
Your answer should be the commit hash (e.g., 'abc1234') or the commit subject line.
Expected
change to red
Campaign Evidence
Loading campaign evidence…
Model Outputs (14)
5275de4
Based on the test results, the first commit that fails is `8b38db7 change to red`. Since all commits before it pass, and it introduces the failure, this is the bad commit.
Answer: 8b38db7
c4448e0
JSON Schema
Structured Output
(raw) {"commit":"c4448e0"}
Based on the test results, the build started failing at commit `e44b549 change to red` and continued to fail in subsequent commits. The commit that introduced the bug is therefore:
`e44b549` (or "change to red")
f365e23
JSON Schema
Structured Output
(raw) {"commit": "f365e23"}
82525cb change to red
f9d0606
JSON Schema
Structured Output
(raw) {
"commit": "f9d0606"
}
Based on the test results, I can see the pattern of when the build broke:
- `d6caf71 Initial commit`: **PASS**
- `d7966b1 add blue styling`: **PASS**
- `5b6e300 change to red`: **FAIL**
- `fdcbce5 add theme title`: **FAIL**
The first commit that introduced the failure was `5b6e300` with the subject "change to red". This commit changed the styling from blue to red, which broke the build.
**Bad commit: `5b6e300`** (or "change to red")
21d8295
JSON Schema
Structured Output
(raw) {"commit": "21d8295"}
Looking at the test results, I can see that:
- Commit `4d6d8d4` (Initial commit): PASS
- Commit `3fd1bb5` (add blue styling): PASS
- Commit `081243f` (change to red): FAIL
- Commit `640f885` (add theme title): FAIL
The bug was introduced in commit `081243f` "change to red" since that's the first commit where tests started failing.
081243f
837b959
JSON Schema
Structured Output
(raw) {"commit": "837b959"}
Invalid JSON. Output: 3a28791
JSON Schema
Structured Output
Structured Output Error
Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
Failure: Failed to parse structured JSON response: Extra data: line 1 column 2 (char 1)
bfc5bd3
Failure: Expected target hash/prefix or subject 'change to red'
efea947
JSON Schema
Structured Output
(raw) {
"commit": "efea947"
}
Failure: Expected target hash/prefix or subject 'change to red'