Google Research and Google DeepMind have extended AMIE, their research medical AI system, from text-based diagnostic dialogue into real-time video consultations. In a study described as a first of its kind, a version called AMIE (Video) conducted live audio-visual consultations, holding a spoken conversation with a patient while simultaneously reasoning about diagnosis and reading visual cues from the video stream. The work was published in the medical literature and posted as a preprint, and it is a research milestone rather than a product: the consultations were with trained patient actors in a controlled study, not with real patients in a clinic.
What makes it worth examining closely is less the headline and more the engineering underneath it, and the careful reading the results demand. AMIE (Video) is a clean illustration of a design pattern that reaches well beyond medicine, and the study around it is a useful case study in the difference between a controlled evaluation and clinical readiness.
The hard problem: talking fast while thinking slowly
A live medical consultation makes two demands that pull in opposite directions. One is conversational: a clinician has to respond in real time, with natural timing, or the interaction feels broken and the patient disengages. The other is analytical: good diagnosis requires slow, careful reasoning, weighing a differential of possible conditions, deciding what to ask next, and noticing subtle signs. Do the deep reasoning inline and the conversation stalls; keep the conversation fast and shallow and the reasoning suffers.
AMIE (Video) resolves this with an asynchronous, three-agent architecture, and this is the single most transferable idea in the work. A patient-facing Talker agent handles the spoken conversation and keeps it responsive. In the background, a Planner agent continuously refines the clinical reasoning, updating the differential diagnosis and management plan and deciding what to prioritize next, while a Perception agent watches the live audio and video streams for clinically relevant non-verbal cues. The two background agents feed guidance back to the Talker as it becomes available, so the conversation stays fluid while the heavy analysis happens off the critical path. The system is built on Gemini and Project Astra, Google's multimodal and real-time interaction technologies.
How AMIE (Video) splits fast talking from slow thinking
Three agents run in parallel. One keeps the conversation flowing in real time; the other two do the heavier reasoning and perception in the background, so deep analysis does not add conversational delay.
The decoupling is the point: real-time dialogue and slow, careful reasoning usually trade off against each other, and running them as separate parallel agents is how AMIE avoids that trade-off.
The pattern, a fast responder decoupled from slower background reasoning and perception, is not specific to healthcare. It is a general answer to any real-time task where a system must respond immediately yet also reason deeply: live tutoring, customer support, technical troubleshooting, any setting where latency and analytical depth both matter. AMIE happens to be a high-stakes, well-instrumented example, which is what makes it a good teaching case for how to structure real-time AI agents.
What the study actually tested
The evaluation was a randomized study built on the Objective Structured Clinical Examination, or OSCE, a format medical schools use to assess clinicians. In an OSCE, standardized patients, actors trained to present specific conditions consistently, are seen across a series of structured scenarios, and trained evaluators score the encounter on defined competencies. Google's study used 100 clinical scenarios, 15 patient actors, and a comparison group of 30 primary care physicians conducting consultations by video, alongside a text-only version of AMIE.
The use of simulated patients is not a weakness to gloss over; it is the standard and appropriate stage for this kind of research. Testing an experimental medical AI on real patients before it has been validated would be neither ethical nor allowed. Standardized-patient studies let researchers compare systems fairly and safely across identical scenarios. But the same design that makes the study rigorous also bounds what it can conclude: it measures performance in a controlled examination, not outcomes in the messier reality of actual clinical care.
Reading the results carefully
The reported finding is that clinical evaluators rated AMIE (Video) on par with or better than the primary care physicians across core competencies, including history-taking, diagnostic accuracy, management appropriateness, and physical observation and examination. That is a strong result within the study's frame, and it is consistent with a pattern seen across several evaluations of large language models on structured medical tasks: systems that are thorough, consistent, and tireless can match or exceed human performance on the parts of a consultation that reward structure and completeness.
What the simulated study found
A randomized OSCE-style simulation of 100 scenarios with 15 patient actors, comparing AMIE (Video) against 30 primary care physicians consulting by video. Patients were simulated, not real. The findings below are qualitative, not scored percentages.
- ✓History-taking thoroughness
- ✓Diagnostic accuracy
- ✓Management appropriateness
- ✓Physical observation and examination
- •Building rapport
- •Partnership and shared decision-making
A controlled simulation is a research milestone, not evidence of real-world clinical performance. The authors note further research is needed before real-world use.
The more revealing detail is where AMIE did not lead. Patient actors preferred AMIE's video interface over text chat for communicative effectiveness, convenience, and feeling understood, but they still preferred human physicians for building rapport and for partnership. That split is the honest center of the result. It suggests current systems are strongest at the structured, information-gathering, reasoning components of care and weakest at the relational parts, the trust, the sense of being cared for by a person, that are central to medicine and hard to quantify. The authors also named concrete technical limits that remain, including fine anatomical precision, subtle emotional nuance, and fast movements, and stated plainly that further research is needed before real-world translation.
The systems are strongest at the structured, reasoning parts of a consultation and weakest at the relational ones. That split, not the headline score, is the honest center of the result.
On reading the AMIE study
Why 'on par with physicians' is not 'ready to replace them'
It is worth being precise about the gap between this study and clinical deployment, because the headline invites a stronger conclusion than the evidence supports. Several things separate a strong OSCE result from real-world use.
Standardized patients present conditions cleanly and consistently; real patients arrive with ambiguity, incomplete histories, multiple overlapping problems, and the full variability of human presentation. Evaluation scores on defined competencies are not the same as clinical outcomes measured over time. A research system is not a regulated medical device: deploying AI that influences diagnosis or treatment generally requires regulatory review, prospective validation, and clear answers on liability and oversight, none of which a study of this kind is meant to provide. And the rapport gap the study itself found points to something the metrics do not fully capture about what patients want from care.
None of this diminishes the achievement. Demonstrating real-time, multimodal medical dialogue at this level of competence is a genuine advance, and it extends a clear research trajectory: AMIE moved from text diagnostic conversations, to multimodal reasoning over images, to live video, each step a distinct milestone. The point is simply that the correct read is "a research system performed well in a controlled simulation," not "AI is ready to run consultations."
The broader signal for AI systems
Step back from medicine and AMIE illustrates a shift in how capable AI systems are being built. The frontier is increasingly not a single monolithic model asked to do everything at once, but an orchestrated system of specialized agents, each handling a distinct concern, coordinated to produce a result no single call could. AMIE's split of talking, planning, and perceiving into parallel agents is a concrete instance of that architecture, and its performance comes as much from the orchestration as from the underlying model.
That has a practical implication for anyone building AI products rather than research systems. Much of the value is moving into the layer above the model: how work is decomposed across agents, how fast and slow processes are coordinated, how perception, reasoning, and interaction are kept in sync. A platform built to orchestrate models and agents rather than to wrap a single one, as Metir AI is, treats the model as one component in a larger system, which is the same structural insight AMIE embodies. The orchestration layer, not any individual model call, is where a system like this earns its results.
The read-through
AMIE (Video) is a well-constructed research result that deserves to be read on its own terms. Technically, it is an elegant demonstration of how to run real-time conversation and deep reasoning at the same time by decoupling them across parallel agents, a pattern with uses far beyond healthcare. Empirically, it shows a system matching physicians on the structured parts of a simulated consultation while still trailing them on rapport, a nuance more instructive than any single score. And methodologically, it is a reminder that a strong controlled study is the beginning of a long validation path, not the end of one.
For medicine specifically, the near-term implication is less about replacing clinicians and more about where structured, tireless, multimodal AI could eventually assist them, in triage, in thorough history-taking, in surfacing a differential for a clinician to weigh, always under human oversight and subject to the regulatory and safety work that real deployment requires. That is a meaningful direction. It is also several careful steps away from the consulting room, and the research itself is the first to say so.
Sources:
- AMIE, our research medical AI system, demonstrates real-time clinical video consultation capabilities | Google Blog
- Advancing AMIE towards expert-level audio-visual clinical consultations | Google Research
- Towards Expert-level Medical AI for Real-time Video Consultations | arXiv preprint
- Advancing conversational diagnostic AI with multimodal reasoning | Nature Medicine
Image credits
Header and in-body image: a masked clinician conducting a video teleconsultation with a patient visible on the screen, shown to illustrate remote video consultation generally. It does not depict AMIE, which is a research system evaluated with trained patient actors rather than in live clinical use. By Ceibos via Wikimedia Commons, licensed under CC BY-SA 4.0.