Medical Image Retrieval Using Manifold Ranking with Relevance Feedback

Abstract

Medical image retrieval (MedIR) is a challenging field in Visual information retrieval, due to the multi-dimensional and multi-modal context of the underlying content. Traditional models do not take the intrinsic characteristics of data into consideration and have achieved limited accuracy in application to medical images. Manifold Ranking (MR) is a technique that can be used in further optimizing precision and recall in MedIR applications as it ranks items by traversing a dynamically constructed content-specific information graph. In this paper, a MedIR approach based on Manifold Ranking is proposed. Medical images being multi-dimensional, exhibit underlying cluster and manifold information which enhances semantic relevance and allows for label uniformity. Hence, when adapted for MedIR, MR can help in achieving large-scale ranking across datasets as is the case in most medical imaging applications. In addition, a relevance feedback mechanism was also incorporated to support a learning based system. We show that MR achieved significant improvement in retrieval results with relevance feedback as compared to the Euclidean Distance (ED) rankings. This showcases the importance of analyzing the inherent latent structure in medical image data for better performance over traditional methods. � 2018 IEEE.