Lifelong Learning for Therapeutic Endoscopy

PI: Daniel Hashimoto, Galen Leung (GI), Gregory Ginsberg (GI), Eric Eaton (SEAS)

Funding: American Society for Gastrointestinal Endoscopy

PCASO Project Lead: Daniel Hashimoto

While different deep learning approaches have been successful in identifying phases of surgery or instruments, existing approaches are limited to individual operations. However, many procedures share steps or are performed on the same organ(s). Such interconnections offer the potential for improved clinical benefit by incorporating recent advances in lifelong deep learning, which enables transfer of knowledge across procedures and continual improvement over time. This project investigates knowledge transfer across “third space” therapeutic endoscopic procedures.

Transfer Learning for Automated Surgical Feedback

PI: Daniel Hashimoto

Funding: American Surgical Association Foundation

PCASO Project Lead: Daniel Hashimoto

Video-based coaching and assessment offer the opportunity to advance surgical performance. Yet, its potential is limited by difficulties in applying it to a broad audience due to the need for many human coaches. Artificial intelligence (AI) could provide automated surgical video analysis to provide coaching and assessment with reduced human input. However, current surgical AI methods focus on one task or one type of procedure at a time. This approach does not account for the interrelated nature of complex phenomena in an operation, preventing AI from providing actionable automated feedback on performance. In contrast, surgeons learn surgical principles and techniques that translate to different operations. This project develops a multitask AI model that can analyze and interpret a surgeon's technique in a way that is analogous to surgeons learning surgical principles, enabling algorithms to transfer knowledge learned in one operation to another. 

Explainability in Computer Assisted Laparoscopic Surgery

PI: Daniel Hashimoto, Eric Wong (SEAS)

Funding: ASSET and Institute for Biomedical Informatics

PCASO Project Lead: Daniel Hashimoto

The goal of this proposal is to develop transparent surgical AI assistants that can explain why zones are deemed to be safe/unsafe. Such a system requires a tight fusion of expertise in surgical AI and explainable AI. However, existing explanation tools have long been criticized for being unreliable, misleading, and ultimately untrustworthy. Dr. Wong has developed methods for interpretable AI with verifiably correct model explanations to overcome these deficiencies. This collaboration's overarching theme is integrating these verifiable explanation techniques into the surgical AI assistants.

Metrics Revolutions: Clinically Relevant Outcome Measures for Computer Vision in Surgery

PI: Daniel Hashimoto, Lena Maier-Hein (Univ of Heidelberg), Amin Madani (UHN), Stefanie Speidel (Univ of Dresden), Dan Stoyanov (UCL)

PCASO Project Lead: Daniel Hashimoto

Building off the Metrics Reloaded project to provide guidance to researchers on appropriate selection of metrics to evaluate the performance of visual models in biomedical imaging tasks, the Metrics Revolutions project seeks to outline more specific, clinically oriented metrics to assess the performance of models that work with visual surgical data.


Patient Perceptions of Therapeutic AI

PI: Daniel Hashimoto

PCASO Project Lead: Vivek Singh

While much of the effort in applying AI to surgery has focused on developing methods for annotating ground truth in surgery and building visual models for the analysis of surgical video, few efforts have sought to determine how patients feel about the possibility of AI influencing the decisions that their physicians may make during a therapeutic procedure such as surgery. This work surveys the landscape of existing literature and seeks the input of a wide range of patients to understand better how patients understand and perceive the role of AI in their care.

Identification of Critical Structures in Robotic Thoracic Surgery

PI: Matjaz Jogan, Kenta Nakahashi (UHN)

PCASO Project Lead: Matjaz Jogan

Identification of critical structures is important for safe dissection in robotic assisted lobectomy. This work expands previous work on Go and NoGo Zones in cholecystectomy to robotic thoracic surgery. 

Weakly Supervised Learning for Video Object Segmentation

PI: Daniel Hashimoto

PCASO Project Lead: Guiqiu Liao