A Prompt For Honest Model Evaluation Reports

Evaluation writeups generated top-down come out as marketing. The headline number goes first and everything after it is written to support the headline.

Invert the order:

"Write an evaluation report in this order:

1. Every failure case, grouped by cause, with a concrete example of each

1. What the eval set does not cover

1. Where the metric disagrees with human judgement, with examples

1. Only then: the aggregate numbers

1. One paragraph on whether this is ready to ship, given 1-4"

Section 2 is the one that gets skipped when a human writes this, because you have to admit what you did not test. Asking for it explicitly gets you an honest list instead of silence.

The reports are less pleasant to read. They have also stopped me shipping two models that looked fine on the aggregate.