When an automated decision leads to real harm, we often lump together three important questions: who caused it, who is morally responsible, and who is legally accountable. But these are distinct questions, and confusing them creates a gap in genuine moral responsibility that can remain unaddressed.
Causal responsibility is usually the simplest to identify but often the least helpful: the AI model generated the output, the system ran as intended, and no individual directly made that exact decision.
Legal liability can also be determined, though it may take time through contracts and regulations.
The most challenging, and frequently overlooked, question is moral responsibility: who had the authority, information, and opportunity to prevent the harm but chose a design or oversight approach that allowed it to happen?
This gap emerges when an incident’s cause traces entirely through an automated system, enabling everyone involved to honestly claim, “I didn’t decide that specific outcome.” Yet, the real moral responsibility lies with those who decided to deploy the system under certain conditions, with a certain level of oversight, fully aware of the risks at the time.
That choice is a human one, made long before the harm occurred, and it remains a moral decision even if the damage was delivered by a machine rather than a person’s hand.
To bridge this gap in accountability during incident reviews, we must resist the temptation to end the inquiry with “the system did it.”
The critical question is not only what the AI produced, but who had the power to impose different safeguards, and whether their choices were reasonable based on what could be known then, rather than excusable simply because a machine executed the outcome.
This ethical approach reinforces that behind every AI decision, there are human values and responsibilities that must guide its deployment.