Diffusion models for few-shot anomaly detection and classification

Due to the rapid technological advancement, automated anomaly inspection has become a crucial aspect in industrial manufacturing. Despite the importance, the methods used to perform these tasks suffer from a data scarcity problem. The lack of anomalous samples limits the performance of tasks such as anomaly detection, classification and localization. Several approaches have been tried in the literature such as training machine learning algorithms only on nominal images, but these methods cannot handle anomaly classification. Some other approaches based on generative models were proposed, but they still suffer from the same data scarcity problem leading to non realistic anomaly generations. In this work a new approach based on a recently proposed framework dubbed AnomalyControl is presented. A Stable Diffusion Inpainting model is fine-tuned to replace nominal parts of an image with other nominal parts, increasing the variance of the good parts of the generated defective images and addressing the data scarcity problem. Three different pipelines are created integrating this new inpainter to the one of AnomalyControl, and are used to generate defective images using the MVTec-AD dataset. The results obtained in the pipelines demonstrate the importance of this new inpainter. It enhances anomaly classification performance when generating from defective images, meaning it not only fulfills its intended function but also improves classification accuracy. However, when generating from nominal images, it results in a slight reduction in anomaly classification performance, an expected outcome given the abundance of nominal samples that do not require additional variance. Interestingly, it demonstrates unexpectedly strong performance in anomaly detection and localization when generating from nominal images, but not when generating from defective images.