Prism-ml / ternary-bonsai-2-27b
max
94.1%
192 / 204 fixtures · 1 run(s)
49,778 input / 137,521 total output / 132,916 reasoning within output tokens $0.07249385
Reliability by Benchmark (Text)
Loading reliability summary…
Text vs JSON Schema Comparison
Pass Rate Delta
-3.9%
Text: 94.1% →
JSON: 90.2%
+4
Gained
JSON pass / text fail
−12
Lost
Text pass / JSON fail
180
Unchanged Pass
Both pass
8
Unchanged Fail
Both fail
Fixture Reliability Delta
| Fixture | Text | JSON | Delta |
|---|
Benchmark Deltas
| Benchmark | Text | JSON | Delta |
|---|---|---|---|
| branch_cleanup | 100% | 75% | -25% |
| blame_forensics | 100% | 83.3% | -16.7% |
| git_clean | 75% | 91.7% | + 16.7% |
| git_grep | 100% | 83.3% | -16.7% |
| cherry_pick | 83.3% | 75% | -8.3% |
| commit_squash | 100% | 91.7% | -8.3% |
| submodule_usage | 75% | 83.3% | + 8.3% |
| tag_management | 100% | 91.7% | -8.3% |
| worktree_usage | 83.3% | 75% | -8.3% |
| commit_messages | 100% | 100% | + 0% |
| git_bisect | 100% | 100% | + 0% |
| git_log_format | 100% | 100% | + 0% |
| git_show | 100% | 100% | + 0% |
| merge_conflicts | 91.7% | 91.7% | + 0% |
| rebase | 91.7% | 91.7% | + 0% |
| reflog | 100% | 100% | + 0% |
| stash_recovery | 100% | 100% | + 0% |
Changed Fixtures (16)
Fixture Gallery (204)
blame_forensics f001
100%
Passed
Tokens
296
Input
237
Total output
229
Reasoning within output
blame_forensics f002
100%
Passed
Tokens
295
Input
400
Total output
393
Reasoning within output
blame_forensics f003
100%
Passed
Tokens
329
Input
113
Total output
104
Reasoning within output
blame_forensics f004
100%
Passed
Tokens
243
Input
125
Total output
117
Reasoning within output
blame_forensics f005
100%
Passed
Tokens
309
Input
1,038
Total output
1,032
Reasoning within output
blame_forensics f006
100%
Passed
Tokens
290
Input
509
Total output
503
Reasoning within output
blame_forensics f007
100%
Passed
Tokens
270
Input
120
Total output
112
Reasoning within output
blame_forensics f008
100%
Passed
Tokens
300
Input
374
Total output
367
Reasoning within output
blame_forensics f009
100%
Passed
Tokens
256
Input
278
Total output
271
Reasoning within output
blame_forensics f010
100%
Passed
Tokens
495
Input
991
Total output
985
Reasoning within output
blame_forensics f011
100%
Passed
Tokens
306
Input
471
Total output
465
Reasoning within output
blame_forensics f012
100%
Passed
Tokens
196
Input
168
Total output
160
Reasoning within output
branch_cleanup f001
100%
Passed
Tokens
149
Input
206
Total output
200
Reasoning within output
branch_cleanup f002
100%
Passed
Tokens
168
Input
193
Total output
185
Reasoning within output
branch_cleanup f003
100%
Passed
Tokens
146
Input
379
Total output
375
Reasoning within output
branch_cleanup f004
100%
Passed
Tokens
184
Input
82
Total output
74
Reasoning within output
branch_cleanup f005
100%
Passed
Tokens
154
Input
179
Total output
173
Reasoning within output
branch_cleanup f006
100%
Passed
Tokens
178
Input
221
Total output
209
Reasoning within output
branch_cleanup f007
100%
Passed
Tokens
148
Input
155
Total output
147
Reasoning within output
branch_cleanup f008
100%
Passed
Tokens
175
Input
182
Total output
170
Reasoning within output
branch_cleanup f009
100%
Passed
Tokens
131
Input
64
Total output
60
Reasoning within output
branch_cleanup f010
100%
Passed
Tokens
165
Input
183
Total output
175
Reasoning within output
branch_cleanup f011
100%
Passed
Tokens
117
Input
150
Total output
146
Reasoning within output
branch_cleanup f012
100%
Passed
Tokens
208
Input
252
Total output
239
Reasoning within output
cherry_pick f001
100%
Passed
Tokens
143
Input
690
Total output
683
Reasoning within output
cherry_pick f002
0%
Failed
Tokens
161
Input
537
Total output
527
Reasoning within output
cherry_pick f003
100%
Passed
Tokens
134
Input
1,099
Total output
1,092
Reasoning within output
cherry_pick f004
100%
Passed
Tokens
205
Input
1,830
Total output
1,802
Reasoning within output
cherry_pick f005
100%
Passed
Tokens
195
Input
121
Total output
100
Reasoning within output
cherry_pick f006
100%
Passed
Tokens
190
Input
380
Total output
355
Reasoning within output
cherry_pick f007
100%
Passed
Tokens
180
Input
916
Total output
903
Reasoning within output
cherry_pick f008
100%
Passed
Tokens
170
Input
82
Total output
73
Reasoning within output
cherry_pick f009
100%
Passed
Tokens
154
Input
143
Total output
135
Reasoning within output
cherry_pick f010
100%
Passed
Tokens
266
Input
528
Total output
497
Reasoning within output
cherry_pick f011
100%
Passed
Tokens
213
Input
292
Total output
265
Reasoning within output
cherry_pick f012
0%
Failed
Tokens
323
Input
952
Total output
906
Reasoning within output
commit_messages f001
92.5%
Passed
Tokens
159
Input
50
Total output
44
Reasoning within output
commit_messages f002
89%
Passed
Tokens
275
Input
123
Total output
116
Reasoning within output
commit_messages f003
95%
Passed
Tokens
130
Input
169
Total output
158
Reasoning within output
commit_messages f004
92.5%
Passed
Tokens
167
Input
111
Total output
105
Reasoning within output
commit_messages f005
92.5%
Passed
Tokens
151
Input
69
Total output
63
Reasoning within output
commit_messages f006
90%
Passed
Tokens
198
Input
280
Total output
271
Reasoning within output
commit_messages f007
92.5%
Passed
Tokens
239
Input
162
Total output
153
Reasoning within output
commit_messages f008
92.5%
Passed
Tokens
123
Input
112
Total output
105
Reasoning within output
commit_messages f009
90%
Passed
Tokens
177
Input
170
Total output
160
Reasoning within output
commit_messages f010
87.5%
Passed
Tokens
347
Input
289
Total output
280
Reasoning within output
commit_messages f011
82.5%
Passed
Tokens
196
Input
1,455
Total output
1,444
Reasoning within output
commit_messages f012
92.5%
Passed
Tokens
177
Input
250
Total output
242
Reasoning within output
commit_squash f001
100%
Passed
Tokens
190
Input
2,638
Total output
2,623
Reasoning within output
commit_squash f002
100%
Passed
Tokens
167
Input
132
Total output
121
Reasoning within output
commit_squash f003
100%
Passed
Tokens
178
Input
2,888
Total output
2,873
Reasoning within output
commit_squash f004
100%
Passed
Tokens
169
Input
308
Total output
294
Reasoning within output
commit_squash f005
100%
Passed
Tokens
164
Input
1,061
Total output
1,051
Reasoning within output
commit_squash f006
100%
Passed
Tokens
147
Input
721
Total output
707
Reasoning within output
commit_squash f007
100%
Passed
Tokens
155
Input
214
Total output
199
Reasoning within output
commit_squash f008
100%
Passed
Tokens
167
Input
181
Total output
166
Reasoning within output
commit_squash f009
100%
Passed
Tokens
156
Input
1,581
Total output
1,567
Reasoning within output
commit_squash f010
100%
Passed
Tokens
174
Input
149
Total output
131
Reasoning within output
commit_squash f011
100%
Passed
Tokens
177
Input
148
Total output
130
Reasoning within output
commit_squash f012
100%
Passed
Tokens
180
Input
170
Total output
147
Reasoning within output
git_bisect f001
100%
Passed
Tokens
234
Input
110
Total output
101
Reasoning within output
git_bisect f002
100%
Passed
Tokens
264
Input
114
Total output
104
Reasoning within output
git_bisect f003
100%
Passed
Tokens
250
Input
137
Total output
130
Reasoning within output
git_bisect f004
100%
Passed
Tokens
258
Input
166
Total output
157
Reasoning within output
git_bisect f005
100%
Passed
Tokens
262
Input
101
Total output
91
Reasoning within output
git_bisect f006
100%
Passed
Tokens
290
Input
227
Total output
217
Reasoning within output
git_bisect f007
100%
Passed
Tokens
260
Input
161
Total output
152
Reasoning within output
git_bisect f008
100%
Passed
Tokens
258
Input
122
Total output
114
Reasoning within output
git_bisect f009
100%
Passed
Tokens
260
Input
221
Total output
207
Reasoning within output
git_bisect f010
100%
Passed
Tokens
258
Input
134
Total output
125
Reasoning within output
git_bisect f011
100%
Passed
Tokens
264
Input
130
Total output
120
Reasoning within output
git_bisect f012
100%
Passed
Tokens
290
Input
178
Total output
169
Reasoning within output
git_clean f001
100%
Passed
Tokens
97
Input
584
Total output
574
Reasoning within output
git_clean f002
100%
Passed
Tokens
108
Input
147
Total output
140
Reasoning within output
git_clean f003
100%
Passed
Tokens
107
Input
115
Total output
108
Reasoning within output
git_clean f004
33.3%
Failed
Tokens
104
Input
9,137
Total output
9,128
Reasoning within output
git_clean f005
100%
Passed
Tokens
96
Input
622
Total output
613
Reasoning within output
git_clean f006
100%
Passed
Tokens
121
Input
2,719
Total output
2,710
Reasoning within output
git_clean f007
100%
Passed
Tokens
112
Input
461
Total output
450
Reasoning within output
git_clean f008
100%
Passed
Tokens
124
Input
167
Total output
160
Reasoning within output
git_clean f009
100%
Passed
Tokens
115
Input
547
Total output
537
Reasoning within output
git_clean f010
100%
Passed
Tokens
110
Input
196
Total output
189
Reasoning within output
git_clean f011
40%
Failed
Tokens
108
Input
3,527
Total output
3,518
Reasoning within output
git_clean f012
50%
Failed
Tokens
110
Input
1,658
Total output
1,644
Reasoning within output
git_grep f001
100%
Passed
Tokens
96
Input
60
Total output
54
Reasoning within output
git_grep f002
100%
Passed
Tokens
112
Input
71
Total output
67
Reasoning within output
git_grep f003
100%
Passed
Tokens
131
Input
34
Total output
30
Reasoning within output
git_grep f004
100%
Passed
Tokens
120
Input
76
Total output
68
Reasoning within output
git_grep f005
100%
Passed
Tokens
161
Input
170
Total output
166
Reasoning within output
git_grep f006
100%
Passed
Tokens
85
Input
166
Total output
162
Reasoning within output
git_grep f007
100%
Passed
Tokens
213
Input
2,026
Total output
2,021
Reasoning within output
git_grep f008
100%
Passed
Tokens
102
Input
60
Total output
56
Reasoning within output
git_grep f009
100%
Passed
Tokens
98
Input
86
Total output
80
Reasoning within output
git_grep f010
100%
Passed
Tokens
108
Input
675
Total output
671
Reasoning within output
git_grep f011
100%
Passed
Tokens
156
Input
129
Total output
125
Reasoning within output
git_grep f012
100%
Passed
Tokens
109
Input
74
Total output
63
Reasoning within output
git_log_format f001
100%
Passed
Tokens
954
Input
73
Total output
69
Reasoning within output
git_log_format f002
100%
Passed
Tokens
946
Input
63
Total output
53
Reasoning within output
git_log_format f003
100%
Passed
Tokens
778
Input
194
Total output
182
Reasoning within output
git_log_format f004
100%
Passed
Tokens
925
Input
67
Total output
63
Reasoning within output
git_log_format f005
100%
Passed
Tokens
781
Input
1,579
Total output
1,571
Reasoning within output
git_log_format f006
100%
Passed
Tokens
955
Input
99
Total output
95
Reasoning within output
git_log_format f007
100%
Passed
Tokens
961
Input
47
Total output
39
Reasoning within output
git_log_format f008
100%
Passed
Tokens
1,252
Input
38
Total output
34
Reasoning within output
git_log_format f009
100%
Passed
Tokens
788
Input
58
Total output
54
Reasoning within output
git_log_format f010
100%
Passed
Tokens
1,469
Input
76
Total output
72
Reasoning within output
git_log_format f011
100%
Passed
Tokens
494
Input
108
Total output
103
Reasoning within output
git_log_format f012
100%
Passed
Tokens
439
Input
71
Total output
67
Reasoning within output
git_show f001
100%
Passed
Tokens
238
Input
72
Total output
66
Reasoning within output
git_show f002
100%
Passed
Tokens
246
Input
367
Total output
362
Reasoning within output
git_show f003
100%
Passed
Tokens
735
Input
107
Total output
98
Reasoning within output
git_show f004
100%
Passed
Tokens
514
Input
87
Total output
78
Reasoning within output
git_show f005
100%
Passed
Tokens
250
Input
282
Total output
277
Reasoning within output
git_show f006
100%
Passed
Tokens
255
Input
45
Total output
41
Reasoning within output
git_show f007
100%
Passed
Tokens
246
Input
102
Total output
95
Reasoning within output
git_show f008
100%
Passed
Tokens
257
Input
111
Total output
70
Reasoning within output
git_show f009
100%
Passed
Tokens
365
Input
289
Total output
285
Reasoning within output
git_show f010
100%
Passed
Tokens
243
Input
154
Total output
150
Reasoning within output
git_show f011
100%
Passed
Tokens
272
Input
64
Total output
59
Reasoning within output
git_show f012
100%
Passed
Tokens
388
Input
72
Total output
68
Reasoning within output
merge_conflicts f001
100%
Passed
Tokens
130
Input
2,129
Total output
2,123
Reasoning within output
merge_conflicts f002
100%
Passed
Tokens
148
Input
234
Total output
224
Reasoning within output
merge_conflicts f003
100%
Passed
Tokens
123
Input
969
Total output
962
Reasoning within output
merge_conflicts f004
100%
Passed
Tokens
190
Input
545
Total output
517
Reasoning within output
merge_conflicts f005
100%
Passed
Tokens
182
Input
185
Total output
164
Reasoning within output
merge_conflicts f006
100%
Passed
Tokens
174
Input
412
Total output
383
Reasoning within output
merge_conflicts f007
100%
Passed
Tokens
178
Input
255
Total output
242
Reasoning within output
merge_conflicts f008
100%
Passed
Tokens
150
Input
171
Total output
162
Reasoning within output
merge_conflicts f009
100%
Passed
Tokens
147
Input
217
Total output
209
Reasoning within output
merge_conflicts f010
100%
Passed
Tokens
227
Input
587
Total output
556
Reasoning within output
merge_conflicts f011
100%
Passed
Tokens
191
Input
255
Total output
228
Reasoning within output
merge_conflicts f012
0%
Failed
Tokens
282
Input
615
Total output
569
Reasoning within output
rebase f001
100%
Passed
Tokens
142
Input
471
Total output
465
Reasoning within output
rebase f002
100%
Passed
Tokens
171
Input
160
Total output
150
Reasoning within output
rebase f003
100%
Passed
Tokens
148
Input
145
Total output
139
Reasoning within output
rebase f004
100%
Passed
Tokens
204
Input
1,523
Total output
1,495
Reasoning within output
rebase f005
100%
Passed
Tokens
188
Input
304
Total output
286
Reasoning within output
rebase f006
100%
Passed
Tokens
188
Input
344
Total output
319
Reasoning within output
rebase f007
100%
Passed
Tokens
182
Input
190
Total output
177
Reasoning within output
rebase f008
100%
Passed
Tokens
164
Input
95
Total output
86
Reasoning within output
rebase f009
100%
Passed
Tokens
149
Input
250
Total output
242
Reasoning within output
rebase f010
100%
Passed
Tokens
258
Input
431
Total output
400
Reasoning within output
rebase f011
100%
Passed
Tokens
203
Input
393
Total output
366
Reasoning within output
rebase f012
0%
Failed
Tokens
315
Input
2,903
Total output
2,857
Reasoning within output
reflog f001
100%
Passed
Tokens
299
Input
215
Total output
176
Reasoning within output
reflog f002
100%
Passed
Tokens
299
Input
384
Total output
248
Reasoning within output
reflog f003
100%
Passed
Tokens
256
Input
351
Total output
206
Reasoning within output
reflog f004
100%
Passed
Tokens
409
Input
6,125
Total output
5,882
Reasoning within output
reflog f005
100%
Passed
Tokens
463
Input
218
Total output
97
Reasoning within output
reflog f006
100%
Passed
Tokens
312
Input
1,051
Total output
779
Reasoning within output
reflog f007
100%
Passed
Tokens
351
Input
1,176
Total output
1,065
Reasoning within output
reflog f008
100%
Passed
Tokens
445
Input
1,961
Total output
1,764
Reasoning within output
reflog f009
100%
Passed
Tokens
383
Input
856
Total output
693
Reasoning within output
reflog f010
100%
Passed
Tokens
351
Input
1,470
Total output
1,290
Reasoning within output
reflog f011
100%
Passed
Tokens
408
Input
7,139
Total output
6,819
Reasoning within output
reflog f012
100%
Passed
Tokens
392
Input
5,672
Total output
5,307
Reasoning within output
stash_recovery f001
100%
Passed
Tokens
184
Input
67
Total output
59
Reasoning within output
stash_recovery f002
100%
Passed
Tokens
353
Input
99
Total output
91
Reasoning within output
stash_recovery f003
100%
Passed
Tokens
171
Input
132
Total output
123
Reasoning within output
stash_recovery f004
100%
Passed
Tokens
166
Input
67
Total output
59
Reasoning within output
stash_recovery f005
100%
Passed
Tokens
178
Input
219
Total output
200
Reasoning within output
stash_recovery f006
100%
Passed
Tokens
230
Input
121
Total output
112
Reasoning within output
stash_recovery f007
100%
Passed
Tokens
263
Input
191
Total output
183
Reasoning within output
stash_recovery f008
100%
Passed
Tokens
228
Input
142
Total output
126
Reasoning within output
stash_recovery f009
100%
Passed
Tokens
258
Input
204
Total output
152
Reasoning within output
stash_recovery f010
100%
Passed
Tokens
175
Input
124
Total output
115
Reasoning within output
stash_recovery f011
100%
Passed
Tokens
170
Input
155
Total output
146
Reasoning within output
stash_recovery f012
100%
Passed
Tokens
181
Input
131
Total output
110
Reasoning within output
submodule_usage f001
100%
Passed
Tokens
88
Input
135
Total output
124
Reasoning within output
submodule_usage f002
0%
Failed
Tokens
149
Input
577
Total output
564
Reasoning within output
submodule_usage f003
0%
Failed
submodule_usage f004
100%
Passed
Tokens
142
Input
158
Total output
152
Reasoning within output
submodule_usage f005
100%
Passed
Tokens
95
Input
218
Total output
189
Reasoning within output
submodule_usage f006
0%
Failed
Tokens
141
Input
173
Total output
167
Reasoning within output
submodule_usage f007
100%
Passed
Tokens
95
Input
127
Total output
107
Reasoning within output
submodule_usage f008
100%
Passed
Tokens
147
Input
6,748
Total output
6,736
Reasoning within output
submodule_usage f009
100%
Passed
Tokens
143
Input
2,800
Total output
2,793
Reasoning within output
submodule_usage f010
100%
Passed
Tokens
148
Input
2,430
Total output
2,419
Reasoning within output
submodule_usage f011
100%
Passed
Tokens
154
Input
80
Total output
73
Reasoning within output
submodule_usage f012
100%
Passed
Tokens
93
Input
129
Total output
115
Reasoning within output
tag_management f001
100%
Passed
Tokens
107
Input
109
Total output
99
Reasoning within output
tag_management f002
100%
Passed
Tokens
105
Input
138
Total output
117
Reasoning within output
tag_management f003
100%
Passed
Tokens
109
Input
77
Total output
65
Reasoning within output
tag_management f004
100%
Passed
Tokens
118
Input
216
Total output
209
Reasoning within output
tag_management f005
100%
Passed
Tokens
134
Input
220
Total output
210
Reasoning within output
tag_management f006
100%
Passed
Tokens
116
Input
143
Total output
126
Reasoning within output
tag_management f007
100%
Passed
Tokens
112
Input
9,752
Total output
9,735
Reasoning within output
tag_management f008
100%
Passed
Tokens
115
Input
532
Total output
505
Reasoning within output
tag_management f009
100%
Passed
Tokens
119
Input
118
Total output
100
Reasoning within output
tag_management f010
100%
Passed
Tokens
96
Input
64
Total output
54
Reasoning within output
tag_management f011
100%
Passed
Tokens
96
Input
288
Total output
280
Reasoning within output
tag_management f012
100%
Passed
Tokens
139
Input
3,375
Total output
3,363
Reasoning within output
worktree_usage f001
100%
Passed
Tokens
190
Input
631
Total output
619
Reasoning within output
worktree_usage f002
0%
Failed
Tokens
199
Input
449
Total output
421
Reasoning within output
worktree_usage f003
100%
Passed
Tokens
270
Input
83
Total output
72
Reasoning within output
worktree_usage f004
100%
Passed
Tokens
273
Input
118
Total output
111
Reasoning within output
worktree_usage f005
100%
Passed
Tokens
183
Input
136
Total output
122
Reasoning within output
worktree_usage f006
100%
Passed
Tokens
188
Input
284
Total output
269
Reasoning within output
worktree_usage f007
0%
Failed
Tokens
189
Input
4,727
Total output
4,701
Reasoning within output
worktree_usage f008
100%
Passed
Tokens
212
Input
1,694
Total output
1,642
Reasoning within output
worktree_usage f009
100%
Passed
Tokens
305
Input
487
Total output
480
Reasoning within output
worktree_usage f010
100%
Passed
Tokens
282
Input
220
Total output
202
Reasoning within output
worktree_usage f011
100%
Passed
Tokens
270
Input
262
Total output
251
Reasoning within output
worktree_usage f012
100%
Passed
Tokens
190
Input
166
Total output
150
Reasoning within output