Decision support is not diagnosis: why AI does not replace doctors

The short answer

No. The AI tools entering clinics today are built to support a clinician’s decision, not to make it. That is a deliberate design choice, and it is what international health guidance asks for.

The longer answer is more useful, because it explains what these tools are for and how to tell a well-built one from a risky one.

What clinical decision support is

Clinical decision support is not new. The US government’s health IT office describes it as a digital tool that provides timely, person-specific information to improve care. Its examples include alerts and reminders, clinical guidelines, order sets, patient data summaries, documentation templates and diagnostic support.

Most clinicians already use some of it: an allergy alert while prescribing, or a reminder that a test is due.

AI widens what such tools can read. A language model can take in a long history, a stack of reports or the transcript of a consultation, and bring forward what is relevant. The job has not changed. It is still to get the right information to the clinician at the right moment.

What AI does well, and what it cannot do

AI is good at work that is large in volume and repetitive:

• reading a great deal of text quickly
• summarizing a long record
• noticing a pattern across many results
• doing all of this at the end of a long day as well as at the start

There are things it cannot do:

• Examine the patient. It knows only what is in the data it was given. It does not see the person in the room.
• Know what is missing. A result that was never entered, or a history nobody wrote down, does not exist for the software. It can be confidently wrong as a result.
• Carry responsibility. Professional accountability for a decision rests with the clinician and the organization, not with a program.
• Weigh what matters to this patient. Circumstances, preferences and values decide a great deal in medicine, and most of them are not in the record.

AI can also be wrong in fluent, convincing ways, and it can reflect gaps and biases in the data it was built from.

The real risk is trusting it too much

The danger most people picture is a machine overruling a doctor. The more common one is quieter. People tend to accept a computer’s suggestion, especially when they are busy. This is known as automation bias.

A system that is right most of the time makes its mistakes harder to catch, not easier. Good decision support therefore makes it easy to see what a suggestion is based on, and normal to disagree with it.

What the guidance says

In 2021 the World Health Organization published its first global report on AI in health, with six guiding principles. The first is protecting human autonomy: humans should remain in control of health-care systems and medical decisions. The others include transparency and explainability, and responsibility and accountability.

In India, the Indian Council of Medical Research published ethical guidelines for AI in biomedical research and healthcare in 2023. They are written for developers, clinicians and ethics committees, and cover ethical principles, the ethics review process, governance and informed consent.

Where the line sits in practice

A principle holds when the software enforces it. These are the things to look for:

1. AI output is labelled. Anyone reading the record can tell what a model wrote.
2. A suggestion stays a suggestion. Nothing the AI produces becomes part of the record until a clinician accepts it.
3. The clinician can see why. A suggestion comes with the facts it rests on.
4. The record shows who decided. There is a trail of who confirmed each item and who signed.
5. Disagreeing is easy. Rejecting or correcting a suggestion takes no more effort than accepting it.

Questions to ask a supplier

• Does the product ever act without a clinician’s confirmation?
• How is AI-written content marked on screen and in the record?
• Can a clinician see what a suggestion was based on?
• What does the audit trail record?
• How has the product been tested, and in what kinds of settings?
• What does the supplier say the product must not be used for?

How Medrus approaches it

Medrus is clinical decision support. AI Clinical Intelligence places relevant information in front of a clinician at the moment of a decision, and every item carries its state: written by AI, unconfirmed, confirmed or signed. Everything the AI produces stays a suggestion until a clinician accepts it, and the clinician signs the record.

You can try the principle for yourself on our home page. For the documentation side of the same idea, see our guide How AI scribes work, step by step.