Count commits in a date range
Tests ability to count commits in a date range. Evaluates combining time filters with aggregation.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
GIT_AUTHOR_DATE='2025-01-15T10:00:00' GIT_COMMITTER_DATE='2025-01-15T10:00:00' bash -c 'echo x > f.txt && git add f.txt && git commit -m "Mid-Jan commit"' - 05
GIT_AUTHOR_DATE='2025-02-10T10:00:00' GIT_COMMITTER_DATE='2025-02-10T10:00:00' bash -c 'echo y > f.txt && git add f.txt && git commit -m "Mid-Feb commit"' - 06
GIT_AUTHOR_DATE='2025-02-20T10:00:00' GIT_COMMITTER_DATE='2025-02-20T10:00:00' bash -c 'echo z > f.txt && git add f.txt && git commit -m "Late-Feb commit"' - 07
GIT_AUTHOR_DATE='2025-03-05T10:00:00' GIT_COMMITTER_DATE='2025-03-05T10:00:00' bash -c 'echo w > f.txt && git add f.txt && git commit -m "Early-Mar commit"' - 08
GIT_AUTHOR_DATE='2025-04-01T10:00:00' GIT_COMMITTER_DATE='2025-04-01T10:00:00' bash -c 'echo v > f.txt && git add f.txt && git commit -m "April commit"'
Prompt
How many commits were made between 2025-02-01 and 2025-03-31 (inclusive)? Output ONLY the number, nothing else.
Expected
3
Campaign Evidence
Loading campaign evidence…
Model Outputs (14)
3
3
JSON Schema
Structured Output
(raw) { "count": 3 }
3
JSON Schema
Structured Output
(raw) { "count": 3 }
3
JSON Schema
Structured Output
(raw) {"count": 3}
3
3
JSON Schema
Structured Output
(raw) {
"count": 3
}
3
3
JSON Schema
Structured Output
(raw) {
"count": 3
}
3
3
JSON Schema
Structured Output
(raw) {"count": 3}
2
Failure: Expected '3', got '2'
2
Failure: Expected '3', got '2'
2
JSON Schema
Structured Output
(raw) {
"count": 2
}
Failure: Expected '3', got '2'