How AI scribes work, step by step

What an AI scribe is

An AI scribe is software that listens to the conversation between a clinician and a patient and prepares a draft of the clinical note. You will also see it called an ambient AI scribe, or ambient clinical documentation. “Ambient” means it works in the background of a normal consultation: the clinician talks to the patient, not to the software.

The idea comes from human medical scribes, who sit in on consultations and write the notes so that the doctor does not have to. The appeal is the same. Documentation takes up much of a clinician’s day, and a lot of it is done facing a screen instead of the patient.

One of the largest published rollouts was at The Permanente Medical Group in California. In about ten weeks, 3,442 physicians used an ambient AI scribe across as many as 303,266 patient encounters. The authors reported less time spent on documentation and favorable feedback from physicians and patients, and they stressed that accuracy needs continuing attention.

Step 1: The patient agrees

Before anything is recorded, the patient is told that an AI tool will help with the notes, and is asked whether they agree. What counts as valid consent depends on the law where you practice and on your organization’s policy, so settle that before the first consultation.

A good product treats consent as a step in the workflow. It keeps a record that consent was given, and it makes it easy to carry on without the tool when a patient says no.

Step 2: The conversation is captured

A microphone in the room, usually on a phone, tablet or computer, captures the consultation. Sound quality matters more than most people expect. Background noise, people talking over each other, masks and distance from the microphone all reduce what the system can hear.

Products differ in what happens to the audio. Some process it and discard it. Others keep it for a period. This is one of the first questions to put to a supplier.

Step 3: Speech becomes text

Speech recognition turns the audio into a written transcript. Most systems also try to work out who said what, so that “I have had chest pain since Monday” is attributed to the patient and not to the doctor.

This is where the first errors appear:

• Drug names, doses and numbers can be misheard.
• Accents, dialects and switching between languages in one sentence are harder for the software than one language spoken clearly.
• A sentence can be attributed to the wrong speaker.

Step 4: A language model drafts the note

A transcript is not a clinical note. A consultation wanders through greetings, small talk and stories about relatives. A large language model reads the transcript and writes a structured draft, typically under headings such as history, examination, assessment and plan.

This step can go wrong in three ways:

• Omission. Something clinically important that was said in the room is missing from the draft.
• Misstatement. The draft says something different from what was said, such as the wrong side of the body, or a symptom the patient denied recorded as present.
• Fabrication. The draft contains something that was never said. This is often called a hallucination.

A fluent, well-formatted draft is not evidence that the draft is accurate.

Step 5: The clinician reviews, corrects and signs

The draft goes to the clinician, who reads it, corrects it and signs it. This is the step that turns a draft into a medical record. Until then it is a suggestion written by software.

The review is the safety step, not a formality. Two things make it work in a busy clinic:

1. The draft is clearly marked as written by AI, so nobody mistakes it for a confirmed record.
2. Checking the draft is quicker than writing the note. If it is not, people stop checking.

The clinician remains responsible for what the signed record says.

What an AI scribe does not do

An AI scribe documents. It does not examine the patient, weigh the evidence or decide what to do next.

Some products add suggestions on top of the note, such as follow-up actions. That is a different function, clinical decision support, and it needs the same rule: a suggestion stays a suggestion until a clinician accepts it. Our guide Decision support is not diagnosis covers where that line sits.

Questions to ask before choosing one

• How is patient consent captured and recorded? What happens when a patient declines?
• Which languages and accents has it been tested with? Can we try it on our own consultations?
• Is the audio kept? Where, for how long, and who can listen to it?
• How does the screen show what was written by AI and what a clinician has confirmed?
• Is there an audit trail of who viewed, changed and signed each record?
• Is our data used to train the supplier’s models?
• What do clinicians do when the tool is unavailable?

How Medrus approaches it

The Medrus AI Clinical Copilot drafts the visit summary from the consultation. A consultation is recorded only after the patient’s consent has been captured. Everything the AI writes is labelled as written by AI and stays a draft until a clinician reviews and confirms it. The clinician signs the record.