AI solutions for Visual Recognition of Thyroid Eye Disease

Thyroid Eye Disease (TED) is a debilitating autoimmune condition with clear visual signs, where diagnostic delays often prevent timely treatment during the critical inflammatory phase. This thesis addresses the need for earlier, more accessible screening by developing and evaluating artificial intelligence methodologies for detecting TED signs from facial photographs. A foundational contribution to this effort is the creation of a novel, expert-annotated ophthalmic dataset, developed with a medical specialist via a robust, privacy-preserving pipeline to overcome the scarcity of public data. However, the dataset's limited size (507 images) and severe class imbalance pose a significant challenge, motivating a systematic benchmark of AI strategies to find a reliable classification method in a data-scarce environment. The study evaluated three distinct approaches, assessing the zero-shot performance of Multimodal Large Language Models (MedGemma, Gemma, Qwen-VL) using four different prompting strategies, examining the specialized medical model MedSigLip both zero-shot and through data-efficient fine-tuning of a classification head on its features, and fully fine-tuning a ResNet-18 architecture to establish a baseline. The results revealed a clear performance hierarchy where fine-tuned methods significantly surpassed zero-shot approaches. The fully fine-tuned ResNet-18 delivered the best overall performance across all clinical signs. While fine-tuning MedSigLip's features also produced strong results, the zero-shot capabilities of all models proved too inconsistent and unreliable for this specialized medical application.