The dangerous failure mode in clinical AI is not the obvious error. It is the suggestion that reads exactly like something a competent colleague would say, arrives at the end of a long day, and is wrong in a way that takes ten minutes to notice and three visits to undo.
Design for the tired clinician, not the careful one
Every safety argument that depends on the user reading carefully is an argument that will fail at 7pm. So the constraint we set is structural rather than behavioural: nothing a model produces is saved, sent or billed until a person takes an explicit action to accept it. Not a default. Not a timeout. An action.
Three things every suggestion carries
- The source. Which note, radiograph or guideline the draft was drawn from, openable in one tap.
- The confidence. Stated plainly, and low confidence is shown as low, not hidden behind hedged language.
- The disagreement path. Editing or rejecting is one gesture, and it is never slower than accepting.
That last one matters more than it looks. If accepting is one tap and correcting is a form, you have not built a review step. You have built a nudge toward agreement.
Disagreement as the training signal
When a clinician edits a draft, we capture the diff. Those edits are the most valuable data the system produces, because they are a practising dentist telling us precisely where the model was wrong, in context, for free. They feed an evaluation set reviewed by clinicians — not a leaderboard, a list of cases we are still failing.
A capability claim without an error rate is marketing. We would rather publish where the model is weak than describe where it is strong.
What we will not do
We will not let a model send a message to a patient unreviewed. We will not train on identifiable patient data. We will not present a generated summary as a clinical record. And we will not ship a feature whose failure mode we cannot describe in a sentence to the clinician who will carry the consequence of it.