Prism-ml / ternary-bonsai-2-27b
xhigh
92.7%
189 / 204 fixtures · 1 run(s)
48,708 input / 147,679 total output / 143,765 reasoning within output tokens $0.07749260
Reliability by Benchmark (Text)
Loading reliability summary…
Text vs JSON Schema Comparison
Pass Rate Delta
-4.9%
Text: 92.7% →
JSON: 87.8%
+6
Gained
JSON pass / text fail
−16
Lost
Text pass / JSON fail
173
Unchanged Pass
Both pass
9
Unchanged Fail
Both fail
Fixture Reliability Delta
| Fixture | Text | JSON | Delta |
|---|
Benchmark Deltas
| Benchmark | Text | JSON | Delta |
|---|---|---|---|
| commit_squash | 100% | 50% | -50% |
| branch_cleanup | 100% | 75% | -25% |
| git_grep | 100% | 75% | -25% |
| git_clean | 100% | 83.3% | -16.7% |
| submodule_usage | 66.7% | 83.3% | + 16.7% |
| worktree_usage | 66.7% | 83.3% | + 16.7% |
| blame_forensics | 100% | 91.7% | -8.3% |
| git_show | 91.7% | 100% | + 8.3% |
| cherry_pick | 75% | 75% | + 0% |
| commit_messages | 100% | 100% | + 0% |
| git_bisect | 100% | 100% | + 0% |
| git_log_format | 100% | 100% | + 0% |
| merge_conflicts | 91.7% | 91.7% | + 0% |
| rebase | 91.7% | 91.7% | + 0% |
| reflog | 100% | 100% | + 0% |
| stash_recovery | 100% | 100% | + 0% |
| tag_management | 91.7% | 91.7% | + 0% |
Changed Fixtures (22)
Fixture Gallery (204)
blame_forensics f001
100%
Passed
Tokens
296
Input
377
Total output
369
Reasoning within output
blame_forensics f002
100%
Passed
Tokens
297
Input
369
Total output
362
Reasoning within output
blame_forensics f003
100%
Passed
Tokens
325
Input
122
Total output
113
Reasoning within output
blame_forensics f004
100%
Passed
Tokens
246
Input
143
Total output
135
Reasoning within output
blame_forensics f005
100%
Passed
Tokens
309
Input
2,136
Total output
2,130
Reasoning within output
blame_forensics f006
100%
Passed
Tokens
289
Input
639
Total output
633
Reasoning within output
blame_forensics f007
100%
Passed
Tokens
261
Input
158
Total output
150
Reasoning within output
blame_forensics f008
100%
Passed
Tokens
300
Input
286
Total output
279
Reasoning within output
blame_forensics f009
100%
Passed
Tokens
254
Input
205
Total output
198
Reasoning within output
blame_forensics f010
100%
Passed
Tokens
484
Input
323
Total output
317
Reasoning within output
blame_forensics f011
100%
Passed
Tokens
304
Input
157
Total output
151
Reasoning within output
blame_forensics f012
100%
Passed
Tokens
197
Input
483
Total output
475
Reasoning within output
branch_cleanup f001
100%
Passed
Tokens
152
Input
192
Total output
186
Reasoning within output
branch_cleanup f002
100%
Passed
Tokens
165
Input
139
Total output
131
Reasoning within output
branch_cleanup f003
100%
Passed
Tokens
146
Input
202
Total output
198
Reasoning within output
branch_cleanup f004
100%
Passed
Tokens
184
Input
203
Total output
195
Reasoning within output
branch_cleanup f005
100%
Passed
Tokens
157
Input
111
Total output
105
Reasoning within output
branch_cleanup f006
100%
Passed
Tokens
183
Input
112
Total output
100
Reasoning within output
branch_cleanup f007
100%
Passed
Tokens
148
Input
86
Total output
78
Reasoning within output
branch_cleanup f008
100%
Passed
Tokens
172
Input
106
Total output
94
Reasoning within output
branch_cleanup f009
100%
Passed
Tokens
133
Input
308
Total output
304
Reasoning within output
branch_cleanup f010
100%
Passed
Tokens
169
Input
183
Total output
175
Reasoning within output
branch_cleanup f011
100%
Passed
Tokens
119
Input
162
Total output
158
Reasoning within output
branch_cleanup f012
100%
Passed
Tokens
214
Input
269
Total output
256
Reasoning within output
cherry_pick f001
100%
Passed
Tokens
143
Input
858
Total output
852
Reasoning within output
cherry_pick f002
0%
Failed
Tokens
163
Input
5,945
Total output
5,934
Reasoning within output
cherry_pick f003
100%
Passed
Tokens
136
Input
748
Total output
741
Reasoning within output
cherry_pick f004
0%
Failed
Tokens
206
Input
1,940
Total output
1,915
Reasoning within output
cherry_pick f005
100%
Passed
Tokens
198
Input
124
Total output
103
Reasoning within output
cherry_pick f006
100%
Passed
Tokens
190
Input
208
Total output
183
Reasoning within output
cherry_pick f007
100%
Passed
Tokens
178
Input
262
Total output
249
Reasoning within output
cherry_pick f008
100%
Passed
Tokens
168
Input
63
Total output
54
Reasoning within output
cherry_pick f009
100%
Passed
Tokens
152
Input
44
Total output
36
Reasoning within output
cherry_pick f010
100%
Passed
Tokens
266
Input
228
Total output
197
Reasoning within output
cherry_pick f011
100%
Passed
Tokens
211
Input
188
Total output
161
Reasoning within output
cherry_pick f012
0%
Failed
Tokens
323
Input
531
Total output
485
Reasoning within output
commit_messages f001
94%
Passed
Tokens
159
Input
84
Total output
78
Reasoning within output
commit_messages f002
80%
Passed
Tokens
275
Input
451
Total output
438
Reasoning within output
commit_messages f003
92.5%
Passed
Tokens
130
Input
222
Total output
211
Reasoning within output
commit_messages f004
95%
Passed
Tokens
167
Input
127
Total output
121
Reasoning within output
commit_messages f005
82.5%
Passed
Tokens
151
Input
124
Total output
118
Reasoning within output
commit_messages f006
100%
Passed
Tokens
198
Input
152
Total output
143
Reasoning within output
commit_messages f007
90%
Passed
Tokens
239
Input
171
Total output
162
Reasoning within output
commit_messages f008
92.5%
Passed
Tokens
123
Input
202
Total output
195
Reasoning within output
commit_messages f009
87.5%
Passed
Tokens
177
Input
170
Total output
160
Reasoning within output
commit_messages f010
89%
Passed
Tokens
347
Input
304
Total output
295
Reasoning within output
commit_messages f011
84%
Passed
Tokens
196
Input
993
Total output
982
Reasoning within output
commit_messages f012
93%
Passed
Tokens
177
Input
221
Total output
213
Reasoning within output
commit_squash f001
100%
Passed
Tokens
189
Input
454
Total output
438
Reasoning within output
commit_squash f002
100%
Passed
Tokens
164
Input
145
Total output
134
Reasoning within output
commit_squash f003
100%
Passed
Tokens
177
Input
2,268
Total output
2,253
Reasoning within output
commit_squash f004
100%
Passed
Tokens
165
Input
672
Total output
658
Reasoning within output
commit_squash f005
100%
Passed
Tokens
165
Input
2,459
Total output
2,449
Reasoning within output
commit_squash f006
100%
Passed
Tokens
148
Input
411
Total output
397
Reasoning within output
commit_squash f007
100%
Passed
Tokens
152
Input
396
Total output
381
Reasoning within output
commit_squash f008
100%
Passed
Tokens
165
Input
206
Total output
191
Reasoning within output
commit_squash f009
100%
Passed
Tokens
155
Input
403
Total output
389
Reasoning within output
commit_squash f010
100%
Passed
Tokens
169
Input
137
Total output
119
Reasoning within output
commit_squash f011
100%
Passed
Tokens
180
Input
160
Total output
142
Reasoning within output
commit_squash f012
100%
Passed
Tokens
178
Input
318
Total output
295
Reasoning within output
git_bisect f001
100%
Passed
Tokens
232
Input
150
Total output
140
Reasoning within output
git_bisect f002
100%
Passed
Tokens
268
Input
180
Total output
170
Reasoning within output
git_bisect f003
100%
Passed
Tokens
256
Input
98
Total output
88
Reasoning within output
git_bisect f004
100%
Passed
Tokens
268
Input
195
Total output
185
Reasoning within output
git_bisect f005
100%
Passed
Tokens
262
Input
129
Total output
121
Reasoning within output
git_bisect f006
100%
Passed
Tokens
290
Input
213
Total output
204
Reasoning within output
git_bisect f007
100%
Passed
Tokens
262
Input
187
Total output
177
Reasoning within output
git_bisect f008
100%
Passed
Tokens
262
Input
49
Total output
40
Reasoning within output
git_bisect f009
100%
Passed
Tokens
258
Input
228
Total output
218
Reasoning within output
git_bisect f010
100%
Passed
Tokens
266
Input
182
Total output
169
Reasoning within output
git_bisect f011
100%
Passed
Tokens
260
Input
88
Total output
78
Reasoning within output
git_bisect f012
100%
Passed
Tokens
284
Input
207
Total output
198
Reasoning within output
git_clean f001
100%
Passed
Tokens
97
Input
608
Total output
598
Reasoning within output
git_clean f002
100%
Passed
Tokens
108
Input
129
Total output
122
Reasoning within output
git_clean f003
100%
Passed
Tokens
107
Input
116
Total output
109
Reasoning within output
git_clean f004
100%
Passed
Tokens
104
Input
6,933
Total output
6,922
Reasoning within output
git_clean f005
100%
Passed
Tokens
96
Input
1,033
Total output
1,026
Reasoning within output
git_clean f006
100%
Passed
Tokens
121
Input
3,298
Total output
3,289
Reasoning within output
git_clean f007
100%
Passed
Tokens
112
Input
295
Total output
283
Reasoning within output
git_clean f008
100%
Passed
Tokens
124
Input
4,739
Total output
4,731
Reasoning within output
git_clean f009
100%
Passed
Tokens
115
Input
529
Total output
519
Reasoning within output
git_clean f010
100%
Passed
Tokens
110
Input
364
Total output
355
Reasoning within output
git_clean f011
100%
Passed
Tokens
108
Input
2,444
Total output
2,432
Reasoning within output
git_clean f012
100%
Passed
Tokens
110
Input
402
Total output
391
Reasoning within output
git_grep f001
100%
Passed
Tokens
96
Input
108
Total output
102
Reasoning within output
git_grep f002
100%
Passed
Tokens
111
Input
79
Total output
75
Reasoning within output
git_grep f003
100%
Passed
Tokens
131
Input
109
Total output
105
Reasoning within output
git_grep f004
100%
Passed
Tokens
120
Input
107
Total output
99
Reasoning within output
git_grep f005
100%
Passed
Tokens
161
Input
128
Total output
124
Reasoning within output
git_grep f006
100%
Passed
Tokens
85
Input
167
Total output
163
Reasoning within output
git_grep f007
100%
Passed
Tokens
213
Input
325
Total output
320
Reasoning within output
git_grep f008
100%
Passed
Tokens
102
Input
70
Total output
66
Reasoning within output
git_grep f009
100%
Passed
Tokens
98
Input
59
Total output
53
Reasoning within output
git_grep f010
100%
Passed
Tokens
108
Input
558
Total output
554
Reasoning within output
git_grep f011
100%
Passed
Tokens
156
Input
110
Total output
106
Reasoning within output
git_grep f012
100%
Passed
Tokens
109
Input
77
Total output
66
Reasoning within output
git_log_format f001
100%
Passed
Tokens
946
Input
91
Total output
87
Reasoning within output
git_log_format f002
100%
Passed
Tokens
946
Input
150
Total output
140
Reasoning within output
git_log_format f003
100%
Passed
Tokens
760
Input
126
Total output
114
Reasoning within output
git_log_format f004
100%
Passed
Tokens
954
Input
104
Total output
100
Reasoning within output
git_log_format f005
100%
Passed
Tokens
781
Input
115
Total output
107
Reasoning within output
git_log_format f006
100%
Passed
Tokens
955
Input
154
Total output
150
Reasoning within output
git_log_format f007
100%
Passed
Tokens
938
Input
124
Total output
114
Reasoning within output
git_log_format f008
100%
Passed
Tokens
1,260
Input
48
Total output
44
Reasoning within output
git_log_format f009
100%
Passed
Tokens
763
Input
58
Total output
54
Reasoning within output
git_log_format f010
100%
Passed
Tokens
1,415
Input
48
Total output
44
Reasoning within output
git_log_format f011
100%
Passed
Tokens
482
Input
101
Total output
96
Reasoning within output
git_log_format f012
100%
Passed
Tokens
456
Input
92
Total output
88
Reasoning within output
git_show f001
100%
Passed
Tokens
244
Input
43
Total output
37
Reasoning within output
git_show f002
0%
Failed
Tokens
245
Input
3,799
Total output
3,794
Reasoning within output
git_show f003
100%
Passed
Tokens
737
Input
142
Total output
133
Reasoning within output
git_show f004
100%
Passed
Tokens
514
Input
119
Total output
110
Reasoning within output
git_show f005
100%
Passed
Tokens
246
Input
149
Total output
144
Reasoning within output
git_show f006
100%
Passed
Tokens
254
Input
116
Total output
112
Reasoning within output
git_show f007
100%
Passed
Tokens
246
Input
97
Total output
90
Reasoning within output
git_show f008
100%
Passed
Tokens
255
Input
117
Total output
78
Reasoning within output
git_show f009
100%
Passed
Tokens
365
Input
229
Total output
225
Reasoning within output
git_show f010
100%
Passed
Tokens
244
Input
73
Total output
69
Reasoning within output
git_show f011
100%
Passed
Tokens
272
Input
60
Total output
55
Reasoning within output
git_show f012
100%
Passed
Tokens
383
Input
84
Total output
80
Reasoning within output
merge_conflicts f001
100%
Passed
Tokens
130
Input
965
Total output
958
Reasoning within output
merge_conflicts f002
100%
Passed
Tokens
148
Input
224
Total output
214
Reasoning within output
merge_conflicts f003
100%
Passed
Tokens
123
Input
2,330
Total output
2,323
Reasoning within output
merge_conflicts f004
100%
Passed
Tokens
190
Input
440
Total output
412
Reasoning within output
merge_conflicts f005
100%
Passed
Tokens
182
Input
260
Total output
239
Reasoning within output
merge_conflicts f006
100%
Passed
Tokens
174
Input
856
Total output
831
Reasoning within output
merge_conflicts f007
100%
Passed
Tokens
178
Input
571
Total output
558
Reasoning within output
merge_conflicts f008
100%
Passed
Tokens
150
Input
63
Total output
54
Reasoning within output
merge_conflicts f009
100%
Passed
Tokens
147
Input
234
Total output
226
Reasoning within output
merge_conflicts f010
100%
Passed
Tokens
227
Input
298
Total output
267
Reasoning within output
merge_conflicts f011
100%
Passed
Tokens
191
Input
359
Total output
332
Reasoning within output
merge_conflicts f012
0%
Failed
Tokens
282
Input
956
Total output
910
Reasoning within output
rebase f001
100%
Passed
Tokens
143
Input
3,149
Total output
3,143
Reasoning within output
rebase f002
100%
Passed
Tokens
170
Input
150
Total output
140
Reasoning within output
rebase f003
100%
Passed
Tokens
150
Input
463
Total output
457
Reasoning within output
rebase f004
100%
Passed
Tokens
204
Input
786
Total output
758
Reasoning within output
rebase f005
100%
Passed
Tokens
188
Input
111
Total output
93
Reasoning within output
rebase f006
100%
Passed
Tokens
189
Input
1,101
Total output
1,076
Reasoning within output
rebase f007
100%
Passed
Tokens
181
Input
109
Total output
96
Reasoning within output
rebase f008
100%
Passed
Tokens
165
Input
199
Total output
190
Reasoning within output
rebase f009
100%
Passed
Tokens
148
Input
168
Total output
160
Reasoning within output
rebase f010
100%
Passed
Tokens
256
Input
690
Total output
659
Reasoning within output
rebase f011
100%
Passed
Tokens
202
Input
215
Total output
188
Reasoning within output
rebase f012
0%
Failed
Tokens
319
Input
8,526
Total output
8,480
Reasoning within output
reflog f001
100%
Passed
Tokens
300
Input
474
Total output
434
Reasoning within output
reflog f002
100%
Passed
Tokens
301
Input
231
Total output
125
Reasoning within output
reflog f003
100%
Passed
Tokens
259
Input
407
Total output
297
Reasoning within output
reflog f004
100%
Passed
Tokens
395
Input
3,805
Total output
3,655
Reasoning within output
reflog f005
100%
Passed
Tokens
451
Input
1,634
Total output
1,519
Reasoning within output
reflog f006
100%
Passed
Tokens
313
Input
513
Total output
353
Reasoning within output
reflog f007
100%
Passed
Tokens
352
Input
144
Total output
102
Reasoning within output
reflog f008
100%
Passed
Tokens
432
Input
2,577
Total output
2,418
Reasoning within output
reflog f009
100%
Passed
Tokens
387
Input
1,234
Total output
1,067
Reasoning within output
reflog f010
100%
Passed
Tokens
329
Input
903
Total output
801
Reasoning within output
reflog f011
100%
Passed
Tokens
409
Input
5,617
Total output
5,475
Reasoning within output
reflog f012
100%
Passed
Tokens
404
Input
9,810
Total output
9,420
Reasoning within output
stash_recovery f001
100%
Passed
Tokens
182
Input
90
Total output
82
Reasoning within output
stash_recovery f002
100%
Passed
Tokens
350
Input
109
Total output
95
Reasoning within output
stash_recovery f003
100%
Passed
Tokens
171
Input
90
Total output
81
Reasoning within output
stash_recovery f004
100%
Passed
Tokens
166
Input
117
Total output
109
Reasoning within output
stash_recovery f005
100%
Passed
Tokens
176
Input
219
Total output
210
Reasoning within output
stash_recovery f006
100%
Passed
Tokens
230
Input
94
Total output
85
Reasoning within output
stash_recovery f007
100%
Passed
Tokens
261
Input
84
Total output
76
Reasoning within output
stash_recovery f008
100%
Passed
Tokens
228
Input
118
Total output
99
Reasoning within output
stash_recovery f009
100%
Passed
Tokens
258
Input
192
Total output
183
Reasoning within output
stash_recovery f010
100%
Passed
Tokens
175
Input
92
Total output
83
Reasoning within output
stash_recovery f011
100%
Passed
Tokens
169
Input
191
Total output
183
Reasoning within output
stash_recovery f012
100%
Passed
Tokens
181
Input
71
Total output
52
Reasoning within output
submodule_usage f001
100%
Passed
Tokens
88
Input
83
Total output
72
Reasoning within output
submodule_usage f002
100%
Passed
Tokens
148
Input
2,756
Total output
2,745
Reasoning within output
submodule_usage f003
0%
Failed
submodule_usage f004
100%
Passed
Tokens
141
Input
177
Total output
171
Reasoning within output
submodule_usage f005
100%
Passed
Tokens
95
Input
493
Total output
465
Reasoning within output
submodule_usage f006
0%
Failed
Tokens
136
Input
244
Total output
238
Reasoning within output
submodule_usage f007
100%
Passed
Tokens
95
Input
210
Total output
190
Reasoning within output
submodule_usage f008
33.3%
Failed
Tokens
146
Input
1,345
Total output
1,338
Reasoning within output
submodule_usage f009
100%
Passed
Tokens
141
Input
236
Total output
230
Reasoning within output
submodule_usage f010
100%
Passed
Tokens
152
Input
6,462
Total output
6,451
Reasoning within output
submodule_usage f011
100%
Passed
Tokens
151
Input
66
Total output
59
Reasoning within output
submodule_usage f012
0%
Failed
Tokens
93
Input
213
Total output
199
Reasoning within output
tag_management f001
100%
Passed
Tokens
109
Input
97
Total output
88
Reasoning within output
tag_management f002
100%
Passed
Tokens
104
Input
173
Total output
152
Reasoning within output
tag_management f003
100%
Passed
Tokens
110
Input
65
Total output
53
Reasoning within output
tag_management f004
100%
Passed
Tokens
115
Input
153
Total output
148
Reasoning within output
tag_management f005
100%
Passed
Tokens
133
Input
431
Total output
421
Reasoning within output
tag_management f006
100%
Passed
Tokens
112
Input
629
Total output
614
Reasoning within output
tag_management f007
66.7%
Failed
Tokens
111
Input
4,701
Total output
4,684
Reasoning within output
tag_management f008
100%
Passed
Tokens
115
Input
1,530
Total output
1,502
Reasoning within output
tag_management f009
100%
Passed
Tokens
119
Input
151
Total output
132
Reasoning within output
tag_management f010
100%
Passed
Tokens
96
Input
89
Total output
79
Reasoning within output
tag_management f011
100%
Passed
Tokens
98
Input
188
Total output
180
Reasoning within output
tag_management f012
100%
Passed
Tokens
138
Input
3,263
Total output
3,252
Reasoning within output
worktree_usage f001
100%
Passed
Tokens
195
Input
154
Total output
142
Reasoning within output
worktree_usage f002
100%
Passed
Tokens
196
Input
8,271
Total output
8,226
Reasoning within output
worktree_usage f003
0%
Failed
worktree_usage f004
0%
Failed
worktree_usage f005
0%
Failed
worktree_usage f006
0%
Failed
worktree_usage f007
100%
Passed
Tokens
191
Input
1,171
Total output
1,149
Reasoning within output
worktree_usage f008
100%
Passed
Tokens
214
Input
507
Total output
452
Reasoning within output
worktree_usage f009
100%
Passed
Tokens
301
Input
448
Total output
441
Reasoning within output
worktree_usage f010
100%
Passed
Tokens
282
Input
172
Total output
154
Reasoning within output
worktree_usage f011
100%
Passed
Tokens
272
Input
173
Total output
162
Reasoning within output
worktree_usage f012
100%
Passed
Tokens
180
Input
197
Total output
181
Reasoning within output