Projects
Lifelong Learning for Therapeutic Endoscopy
PI: Daniel Hashimoto, Galen Leung (GI), Gregory Ginsberg (GI), Eric Eaton (SEAS)
Funding: American Society for Gastrointestinal Endoscopy
PCASO Project Lead: Daniel Hashimoto
While different deep learning approaches have been successful in identifying phases of surgery or instruments, existing approaches are limited to individual operations. However, many procedures share steps or are performed on the same organ(s). Such interconnections offer the potential for improved clinical benefit by incorporating recent advances in lifelong deep learning, which enables transfer of knowledge across procedures and continual improvement over time. This project investigates knowledge transfer across “third space” therapeutic endoscopic procedures.
Transfer Learning for Automated Surgical Feedback
PI: Daniel Hashimoto
Funding: American Surgical Association Foundation
PCASO Project Lead: Daniel Hashimoto
Video-based coaching and assessment offer the opportunity to advance surgical performance. Yet, its potential is limited by difficulties in applying it to a broad audience due to the need for many human coaches. Artificial intelligence (AI) could provide automated surgical video analysis to provide coaching and assessment with reduced human input. However, current surgical AI methods focus on one task or one type of procedure at a time. This approach does not account for the interrelated nature of complex phenomena in an operation, preventing AI from providing actionable automated feedback on performance. In contrast, surgeons learn surgical principles and techniques that translate to different operations. This project develops a multitask AI model that can analyze and interpret a surgeon's technique in a way that is analogous to surgeons learning surgical principles, enabling algorithms to transfer knowledge learned in one operation to another.
Explainability in Computer Assisted Laparoscopic Surgery
PI: Daniel Hashimoto, Eric Wong (SEAS)
Funding: ASSET and Institute for Biomedical Informatics
PCASO Project Lead: Daniel Hashimoto
The goal of this proposal is to develop transparent surgical AI assistants that can explain why zones are deemed to be safe/unsafe. Such a system requires a tight fusion of expertise in surgical AI and explainable AI. However, existing explanation tools have long been criticized for being unreliable, misleading, and ultimately untrustworthy. Dr. Wong has developed methods for interpretable AI with verifiably correct model explanations to overcome these deficiencies. This collaboration's overarching theme is integrating these verifiable explanation techniques into the surgical AI assistants.
Metrics Revolutions: Clinically Relevant Outcome Measures for Computer Vision in Surgery
PI: Daniel Hashimoto, Lena Maier-Hein (Univ of Heidelberg), Amin Madani (UHN), Stefanie Speidel (Univ of Dresden), Dan Stoyanov (UCL)
PCASO Project Lead: Daniel Hashimoto
Building off the Metrics Reloaded project to provide guidance to researchers on appropriate selection of metrics to evaluate the performance of visual models in biomedical imaging tasks, the Metrics Revolutions project seeks to outline more specific, clinically oriented metrics to assess the performance of models that work with visual surgical data.
Patient Perceptions of Therapeutic AI
PI: Daniel Hashimoto
PCASO Project Lead: Vivek Singh
While much of the effort in applying AI to surgery has focused on developing methods for annotating ground truth in surgery and building visual models for the analysis of surgical video, few efforts have sought to determine how patients feel about the possibility of AI influencing the decisions that their physicians may make during a therapeutic procedure such as surgery. This work surveys the landscape of existing literature and seeks the input of a wide range of patients to understand better how patients understand and perceive the role of AI in their care.
Identification of Critical Structures in Robotic Thoracic Surgery
PI: Matjaz Jogan, Kenta Nakahashi (UHN)
PCASO Project Lead: Matjaz Jogan
Identification of critical structures is important for safe dissection in robotic assisted lobectomy. This work expands previous work on Go and NoGo Zones in cholecystectomy to robotic thoracic surgery.
Weakly Supervised Learning for Video Object Segmentation
PI: Daniel Hashimoto
PCASO Project Lead: Guiqiu Liao