Search with line numbers using git grep -n
Tests ability to search with line numbers using git grep -n. Evaluates output-format options for search results.
Baseline Repository
These commands set up the repo before the model sees the prompt. They define the starting file structure, staged changes, and Git history.
- 01
git init - 02
git config user.email 'test@test.com' - 03
git config user.name 'Test User' - 04
mkdir -p src - 05
printf 'import os import sys import logging logger = logging.getLogger(__name__) def main(): logger.info("Starting") os.environ["MODE"] = "prod" sys.exit(0) if __name__ == "__main__": main() ' > src/app.py - 06
git add . - 07
git commit -m 'Add application entry point' - 08
echo 'git grep -n import' > .grep_command - 09
git add .grep_command - 10
git commit -m 'Add grep sentinel'
Prompt
Here is the output of a git grep -n command showing line numbers. On which line numbers does 'import' appear in src/app.py? Output ONLY the line numbers separated by commas, nothing else.
Expected
1,2,3
Campaign Evidence
Loading campaign evidence…
Model Outputs (14)
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) { "line_numbers": "1,2,3" }
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) { "line_numbers": "1,2,3" }
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) {"line_numbers": "1,2,3"}
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) { "line_numbers": "1,2,3" }
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) { "line_numbers" : "1,2,3" }
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) {"line_numbers": "1,2,3"}
1,2,3
1,2,3
JSON Schema
Structured Output
(raw) {
"line_numbers": "1,2,3"
}