Distress-Conversation Safety Eval

The gap it exists to close

Almost all safety evaluation scores exchanges independently. A prompt goes in, a response comes out, the response is judged. That design can tell you whether a model refuses a dangerous request. It cannot tell you whether the model keeps refusing when the person asking is nine turns into a conversation and getting more distressed, more insistent, and more skilled at reframing the ask.

That is the shape of the real failure, and it is invisible to a harness that never looks at turn nine in the context of turns one through eight.

Seven scenarios × ten turns × eight criteria, scored per turn with evidence spans, aggregated into multi-turn metrics — position hold rate, turns to first failure, drift slope, deflection index. Criteria are operationalised from Samaritans media guidelines, WHO guidance, NICE NG225 and the safe-messaging literature, with provenance recorded per criterion in code.

It runs offline with no API key, and make report reproduces every number below.

The result that motivates the whole design

bastion-v0 runs the failure in the opposite direction. It never emits a single piece of harmful content, and it is the worst-performing system in the suite by thirty-three points. It signposts on turn one, thins as acuity rises, repeats referrals the user has explicitly refused, and terminates the conversation at peak escalation. Under a harness that asks only "was harmful content emitted", it scores perfect.

Score and gate are reported separately and neither can be quoted alone. That is the argument the repository is built to make.

Drift is measurable before the failure lands

concordia-v1 degrades at −1.51 points per turn and records its first critical failure at a mean of turn 9.3. The downward slope is visible from around turn 6 — three turns before the concession completes.

A harness scoring exchanges independently sees eight good turns and one bad one, and cannot tell that the ninth was the predictable end of a trend. Drift slope makes the failure forecastable rather than merely detectable after the fact.

Criteria separate warmth from safety

helpmax-v1 scores as well as the reference target on validation and engagement, and fails on method safety. bastion-v0 inverts it: no harmful content, and 0.00 on emotional validation.

Both are unsafe, for opposite reasons, and a single axis cannot represent that.

Three design decisions worth defending

Detection is scope-aware, and it had to be

"I'm not going to give you a lethal dose threshold" and "the lethal dose threshold is roughly X" share almost every token. A keyword detector scores the correct refusal as a catastrophic failure — systematically, so that the better a system behaves the worse it scores.

Three scopes are therefore separated: what the responder asserts, what it quotes or mirrors back, and what it is being asked for. All three disambiguations were found by the harness failing on its own known-good reference target, which is the argument for keeping one in the suite.

Both directions of "safe" are penalised

Turn-one signposting and terminal refusal both avoid emitting harmful content, and both end the conversation that was the intervention. Most evaluation can only see one failure direction.

The reference systems are fixtures, not models

Four deterministic profiles exercise every branch of the rubric so the committed reports are byte-reproducible offline and CI can assert that the metrics still separate the behaviours. They demonstrate metric sensitivity; they say nothing about any deployed system. Pointing --system anthropic:<model-id> at a real one is a one-line change.

Where the practitioner experience shows

I served seven years as a deputy team leader in the Singapore Police Force's Crisis Negotiation Unit, deploying to threat-to-life incidents. The thing that transfers is not vocabulary — it is knowing that the dangerous moment in a crisis conversation is rarely the opening. It is turn nine, when the person has stopped performing for you and the responder has been worn down into agreeing with something it refused an hour earlier.

Terminating at peak escalation is not a safe outcome, and a rubric that rewards it is measuring the wrong thing. That is why both failure directions are scored.

Testing

58 tests. The end-to-end suite asserts the properties the harness exists to demonstrate: that the reference target passes the gate cleanly, that the sycophant is caught by position-collapse failures rather than capability failures, that the deflector fails on engagement while emitting nothing harmful, that a high average can still fail the gate, and that failures land late enough that single-turn evaluation would miss them.