Pillcam CV | Gastrointestinal condition identification using pillcam imagery

Pillcam endoscopy generates thousands of images as the camera travels through a patient's digestive tract. The challenge was to develop a computer vision system that could automatically identify and locate various gastrointestinal conditions from this imagery, effectively functioning as an AI assistant for gastroenterologists.

With 11 different condition classes appearing in the Kvasir-Capsule dataset and significant class imbalance, the model needed to be robust enough to identify both common and rare conditions with high accuracy and precision.

I leveraged the publicly available Kvasir-Capsule dataset, a comprehensive video capsule endoscopy collection containing thousands of labeled images capturing various gastrointestinal conditions. Using this specialized dataset allowed for training a model that could identify conditions specific to endoscopic imagery.

For the model architecture, I selected YOLO v9, the latest iteration of the popular object detection framework, known for its excellent balance of speed and accuracy. The model was trained on Google Colab, utilizing its GPU resources, while Roboflow was used to host and manage both the dataset and the trained model.

The training process leveraged YOLOv9's sophisticated loss function architecture, which was particularly well-suited to the challenges of medical image analysis with unbalanced class distribution:

The Distributed Focal Loss component was particularly crucial for this project. DFL builds on traditional focal loss by reducing the weight of easy-to-detect examples while emphasizing difficult cases. What makes DFL especially powerful is its ability to distribute this weighting across multiple scales and classes simultaneously, allowing the model to focus its learning capacity on the most challenging detection scenarios.

In the context of gastrointestinal condition detection, this meant the model could effectively learn to identify both common conditions with abundant training examples and rare pathologies with limited representation in the dataset. This balanced approach to learning led to the exceptionally high performance metrics achieved despite the inherent class imbalance in medical datasets.

The final solution is a computer vision system capable of identifying and localizing 11 different gastrointestinal conditions from pillcam imagery with remarkable accuracy, despite the challenges of class imbalance in the dataset.

The system takes individual frames from pillcam video feeds and identifies potential abnormalities, highlighting their location within the image. This allows for faster review of pillcam footage and reduces the chance of missed diagnoses.

The implementation process involved several key stages:

The model was trained to recognize and localize 11 different gastrointestinal conditions, including both common findings and rarer pathologies. Despite the unbalanced nature of the dataset, the model achieved excellent results across all classes.

This project demonstrates the potential for AI to augment medical diagnostics, particularly in the field of gastroenterology. With its exceptional performance metrics (95.1% mAP@50, 94.0% precision, and 88.4% recall), the model could significantly reduce the time required for pillcam video analysis while potentially increasing detection rates for abnormalities.

The high precision of the model is particularly noteworthy given the unbalanced nature of the dataset, suggesting robust performance even for less common conditions.

The next phase of development extends beyond condition detection to precise localization of the pillcam itself within the gastrointestinal tract. To support this research, we have 3D printed a full-scale model of the human digestive system that serves as a testing environment.

The localization approach will leverage two complementary technologies:

This dual-technology approach aims to create a comprehensive system that not only identifies gastrointestinal conditions but also maps their precise anatomical location, potentially revolutionizing both diagnosis and treatment planning in gastroenterology.