Accountable AI

What I mean by accountability

An AI system is accountable when its decisions can be examined, justified, and contested.

Imagine a model screens a patient for a clinical trial and says: “Excluded: kidney function is below the trial’s threshold.” Three questions decide whether that answer is accountable:

1. What decided it? Which facts from the record, which eligibility rules, and what the model assumed where the record was silent.
2. Is the explanation true? Give the same patient normal kidney function and run it again. If the patient is still excluded, kidney function was not the reason, no matter how reasonable the explanation sounded.
3. What would change it? The patient, the doctor, or the trial coordinator should be able to see which conditions would flip the decision, so they can check them, correct a wrong record, or appeal.

The second question is the heart of it. I test explanations with counterfactuals: change one thing, rerun, and check whether the system’s own account predicted what happened.

Why it matters

Language models now help decide who qualifies for a trial, how a contract clause is read, and which code ships. In these settings a correct answer is not enough. Some examples of what can go wrong, even when the answer looks fine:

• A user writes “I think the answer is B, but I’m not sure.” The model answers B and explains its reasoning step by step, never mentioning the user. Without the user’s guess, it would have answered C. The user leaves more confident than they should be, because their own guess came back to them as the model’s conclusion.
• An assistant cites a retrieved document as the basis for its answer. Remove the document and the answer is the same: the citation was decoration. Or the reverse: the document changed the answer, and the assistant says nothing.
• Two identical loan applications get different decisions on different days, each with a confident, plausible explanation.

In every case the explanation reads well. The problem is that nobody checked it against what actually drove the decision. Asking a model to explain itself is not the same as holding it accountable.

How to get there

I work on three complementary routes:

• Accountable by construction. Let the language model translate records, rules, and policies into formal constraints, and let a solver make the decision. The same facts always give the same decision, and every decision comes with a checkable trace, the assumptions it made, and the conditions that would flip it. (VERDICT, EMNLP 2026 Outstanding Paper; SatIR)
• Accountable by training. When a model must explain itself in plain language, teach it to say what actually changed its answer, using labels from its own counterfactual behavior: run it with and without an input and see whether the answer moves. (CREDIT)
• Accountable by measurement. Score what models say about themselves against what happens when their inputs change. You cannot fix a gap you do not measure.

Next, I want to test these ideas where real decisions are messy: missing records, ambiguous rules, many interacting factors. And I want to understand why models so often fail to report what drove their answers: whether it comes from how they are trained or from how they are built.