Case Study: Deploying AI in Clinical Care with Amol Verma

Main ideas, concepts, and lessons

• AI deployment in clinical care should start from real bedside problems, especially unrecognized patient deterioration (the “crashing patient” scenario).
• Effective AI is not just a predictive model—it requires designing an end-to-end sociotechnical system, including:
  • how alerts reach clinicians,
  • what clinicians are expected to do in response,
  • how teams coordinate across wards/ICU/palliative care,
  • and how the system is evaluated and maintained over time.
• Clinician trust and real-world usefulness are crucial:
  • Validating AI performance in real time against clinician judgments,
  • Showing complementary strengths (AI + clinician input).
• Care goals extend beyond prediction:
  • reduce mortality,
  • improve communication with patients/families,
  • improve coordination across clinical teams,
  • improve end-of-life (palliative) care when escalation is not aligned with patient preferences.
• Implementation requires rigorous quality-improvement (QI) thinking, including iterative testing and safety checks before/after rollout.
• Machine learning has unique deployment challenges that differ from traditional medical devices/medications:
  • Black-box complexity makes individual explanations hard.
  • Ongoing maintenance/updating is required because performance can degrade over time.
  • Hidden bias may exist due to how models learn from historical data and device/system patterns.
  • Data/clinical practice shift over time can make alerts obsolete (e.g., pandemic effects on fever-based sepsis detection).
• The field is maturing, with increasing calls for clinical trial rigor (e.g., pre-registration), but few AI technologies have strong evidence from deployed, well-evaluated trials—especially outside highly integrated data environments.
• Canada’s challenge includes fragmented health data, motivating integrated data/compute initiatives (e.g., Gemini and Canada).

---

Detailed methodology / step-by-step approach (how the AI was developed and deployed)

1) Identify the clinical problem and define outcomes (QI “explore” phase)

• Use real patient stories (de-identified) to illustrate a common acute-care failure mode:
  • unrecognized deterioration leading to urgent ICU transfer or death.
• Define clinical and system goals:
  • Primary outcome: reduce unexpected/non-palliative deaths.
  • Secondary aims:
    • improve communication with families,
    • improve coordination across teams (general medicine ↔ ICU ↔ palliative care),
    • improve end-of-life care consistent with preferences.
• Establish a change theory:
  • earlier alerts create a time window for intervention and care escalation/de-escalation planning.

2) Assess feasibility of data (ML/AI-specific requirement)

• Confirm that available EMR data can support:
  • training a model,
  • measuring outcomes reliably.

3) Design the solution as a full intervention (not only a model)

• Determine the model purpose (prediction window and target event).
• Build the sociotechnical system, including:
  • how model predictions/alerts are communicated to clinicians,
  • how the EMR/workflow embeds the tool,
  • the planned clinical response pathway.
• Develop a clinical pathway (“high-risk care pathway”) specifying what to do after an alert, such as:
  • physician reassessment within ~1 hour of alert,
  • connecting physician with nurse,
  • increased monitoring,
  • clinician judgment about potential causes and mitigation (meds/tests/consults),
  • documenting patient preferences/goals of care (especially for end-of-life alignment),
  • coordinating with palliative care when appropriate,
  • proactive workload balancing to avoid overloading nurses with multiple high-risk patients at shift start.

4) Validate the model in a real-time, clinician-relevant way

• Performance alone wasn’t enough; clinicians needed evidence of usefulness.
• Conduct real-time validation:
  • clinicians and nurses predict outcomes,
  • compare their predictions to the AI model across 3,000+ predictions over ~4 months.
• Key findings supporting deployment:
  • AI is near clinician-level for ICU/death prediction overall,
  • AI is better at detecting ICU transfers,
  • combining AI + clinician predictions improves detection meaningfully without increasing false positives, supporting complementarity.

5) Train and engineer the model for operational actionability

• Example model approach (ChartWatch):
  • Multivariate Adaptive Regression Splines (MARS),
  • time-aware predictions (using trajectories of risk scores),
  • deployment updated hourly (including early design choices on alert frequency to reduce disruption).
• Refine prediction horizon for actionability:
  • initially trained for events at any hospital time,
  • adjusted to 24–48 hour windows to match what clinicians can act on.

6) Implement safely with phased rollout and iterative evaluation (PDCA/PSDA)

• Start with silent testing:
  • deploy to confirm real-time behavior (timestamps/workflow differences from research settings),
  • detect issues specific to production workflows.
• Roll out in phases:
  • begin with a subset of clinical teams/units,
  • hold weekly implementation meetings to collect feedback and refine.
• Use an iterative QI framework consistent with Plan → Do → Study → Act.

7) Evaluate impact using practical study design under constraints

• Deploy on general internal medicine in phases (Aug–Oct 2020), with comparisons including:
  • pre-intervention vs intervention periods,
  • excluding early COVID months for comparability.
• Use a quasi-experimental strategy:
  • compare outcomes on units where ChartWatch was deployed vs sub-specialty units where it wasn’t,
  • focus on primary endpoint: non-palliative mortality.
• Use propensity score matching/weighting to adjust baseline covariates.

8) Investigate “how it worked” (mechanism of effect)

• Examine whether improved outcomes align with changes in care processes.
• Check concerns like excessive false positives driving unnecessary imaging.
• Reported observed differences within 24 hours after alert:
  • antibiotics increased (consistent with sepsis risk),
  • glucocorticoids increased (uncertain cause; possible adrenal insufficiency or respiratory decompensation/COVID-era patterns),
  • vital sign measurement frequency increased substantially (suggesting closer monitoring),
  • imaging and several other metrics showed no significant harmful escalation.

---

What the ChartWatch case study concluded

• Overall system impact (primary outcome):
  • modest absolute reduction in non-palliative mortality on the medicine unit vs pre-period,
  • no significant difference in non-deployed sub-specialty units.
• Stronger signal in “high-risk” subgroup:
  • using the risk score to identify high-risk patients,
  • notable reduction in non-palliative death when ChartWatch was deployed.
• Operational “success” narrative (coordination effect):
  • example: an on-call resident receives a high-risk alert,
  • evaluates promptly,
  • ICU involvement occurs quickly after nurse reports deterioration,
  • patient transferred without a ward respiratory arrest,
  • demonstrates improved coordination even when the underlying diagnosis wasn’t fully specified.

---

Key differences between AI and other regulated clinical tools (lessons learned)

1. Black-box complexity

   • Hard to explain why a specific prediction happened.
   • Explanations may not always help; they can mislead clinicians into diagnostic “rabbit holes.”

2. Maintenance and updating

   • Unlike drugs/devices, models can require frequent updates (monthly or more in some settings).
   • Requires regulatory and operational strategies for ongoing monitoring and updates.

3. Bias can be hidden

   • Bias may emerge from latent patterns, not just obvious demographic factors.
   • Requires deliberate evaluation across subgroups.

4. Data shift over time

   • Clinical practices and populations change.
   • Example: fever-based sepsis alerts became less effective during COVID because fever’s relationship to bacterial infection changed.

---

Broader field insights and future direction

• AI clinical research and deployment are not well-aligned:
  • rigorous research often doesn’t reach deployment,
  • deployed tools may lack equivalent rigor.
• Evidence standards are increasing:
  • calls for pre-registration before publishing AI trials (paralleling other medical research).
• Major infrastructure need:
  • integrated health data + computing + clinical implementation expertise.
• Canada’s data fragmentation is highlighted as a barrier.
• Proposed solution: build AI centers of excellence / integrated networks.
  • The presenter references “Gemini and Canada”, combining multi-hospital data and compute, with remote researcher access.

---

Speakers / sources featured

Speaker

• Dr. Amol Verma — physician and scientist (St Michael’s Hospital, Unity Health Toronto; associate professor, University of Toronto; professor of AI research and education in medicine; also cited leadership roles in Ontario Health/agency work)

Referenced organizations / sources (not speakers)

• Signal (startup company; acquired ChartWatch solution)
• Ontario Health (described by the speaker)
• Canadian Medical Association Journal
• Institute for Healthcare Improvement (IHI) change models
• NEJM AI Journal (mentioned regarding pre-registration standards)
• Nature Medicine (mentioned via a paper on chest X-ray bias)
• Epic / Epic Sepsis detection alert (example of change-over-time failure)
• Kaiser Permanente (example of integrated US data network)
• UK CPRD (“Clinical Practice Research Data Link”) (example of integrated data)
• Gemini and Canada / geminmedicine.com (presenter’s initiative)

Data/model referred to

• ChartWatch — AI early warning system (multivariate adaptive regression splines; time-aware) predicting unplanned ICU transfer and in-hospital death