AIOZ AI / Mar 29th / 4 min read

FedEFM - Federated Endovascular Foundation Model with Unseen Data (Part 2)

Data collection process and how the dataset is managed in the training and fine-tuning phases.

In this part, we will outline the data collection process and how the dataset is managed in the training and fine-tuning phases.

Robotic Setup

To collect large-scale X-ray images, we employ a robotic platform and a full-size silicon phantom. A surgeon uses a master device joystick to control a follower robot for cannulating three arteries: the left subclavian (LSA), left common carotid (LCCA), and right common carotid (RCCA). During each catheterization procedure, the surgeon activates the X-ray fluoroscopy using a pedal in the operating room. The experiments are conducted using the Epsilon X-ray Generator. We develop a real-time image grabber to transmit the video feed of the surgical scene to a workstation, a computer-based device equipped with an 8-Core ARM v8.2 64-bit CPU. Overall, we collect and label 4,700 new X-ray images to create our EIPhantom dataset. An overview of our robotic setup is demonstrated in Figure 1.

Figure 1. Data collection with endovascular robot.

Data collection

Apart from X-ray images collected from our real robot, we also collect an EISimulation dataset from the CathSim simulator for simulated X-ray images. We manually label both data from the robot and CathSim simulator to use them in downstream tasks. We note that the datasets used to train the foundation model are not being used in downstream endovascular understanding tasks. Figure 2 provides a detailed summary of the datasets used for training and fine-tuning. Additionally, we also visualize samples from each dataset in Figure 3.

Figure 2. X-ray dataset used in our work.

CathACtion

Vessel 12

DRIVE

SenNet

Medical Decathlon

EI Simulator

EI Phantom

RANZCR

Cath Animal

Figure 3. Visualization of datasets used in our work

Motivation

Our goal is to train a federated foundation model for endovascular intervention that incorporates all available types of X-ray data. However, in practice, each hospital (silo) possesses specific data sources that may not be accessible to others. This results in disparities in data corpora across institutions, meaning that certain datasets are present in one hospital but absent in another. Figure 4 illustrates this challenge, which leads to the unseen data issue—an obstacle that must be addressed to ensure effective federated training.

Federated learning preserves data privacy by preventing direct data sharing while allowing the exchange of model weights among hospital silos. To leverage this feature, we introduce the Federated Endovascular Foundation Model (FedEFM), a multishot federated distillation algorithm that utilizes Earth Mover’s Distance (EMD) to facilitate learning. Our approach enables local silo models to learn from neighboring silos and incorporate the acquired knowledge back into their own models through a distillation process. Unlike traditional methods that require consistent label sets across both local and global models trained within federated silos, devices, or servers, our method ensures seamless federated training without requiring hospitals to share their datasets—further enhancing data privacy. Additionally, once trained, the foundational model’s weights provide a valuable initialization for downstream tasks.