Coupled Manifold Learning for Retrieval Across Modalities
Kazi A, Conjeti S, Katouzian A, Navab N (2017)
Publication Type: Conference contribution
Publication year: 2017
Publisher: Institute of Electrical and Electronics Engineers Inc.
Book Volume: 2018-January
Pages Range: 1321-1328
Conference Proceedings Title: Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017
Event location: Venice, ITA
ISBN: 9781538610343
Abstract
Coupled Manifold Learning (CpML) is targeted at aligning data manifolds across two related modalities to facilitate similarity preserving cross-modal retrieval. Local and global topologies of the data cloud reflect intra-class variability and overall heterogeneity respectively making it critical to retain both for meaningful retrieval. Towards this we propose a learning paradigm which simultaneously aligns global topology while preserving local manifold structure. The global topologies are maintained by recovering underlying mapping functions in the joint manifold space by deploying partially corresponding instances. The inter, and intra-modality affinity matrices are then computed to reinforce original data skeleton using perturbed minimum spanning tree (pMST), and maximizing the affinity among similar cross-modal instances, respectively. The performance of proposed algorithm is evaluated upon two benchmark multi-modal image-text datasets (Wikipedia and PascalVOC2012 - Sentence). We further show versatility and interdisciplinary application by extending it to crossmodal retrieval between multi-stain atherosclerosis histology medical image dataset. We exhaustively validate and compare CpML to other joint-manifold learning methods and demonstrate superior performance across datasets and tasks.
Involved external institutions
Germany (DE)
United States (USA) (US)How to cite
APA:
Kazi, A., Conjeti, S., Katouzian, A., & Navab, N. (2017). Coupled Manifold Learning for Retrieval Across Modalities. In Proceedings - 2017 IEEE International Conference on Computer Vision Workshops, ICCVW 2017 (pp. 1321-1328). Venice, ITA: Institute of Electrical and Electronics Engineers Inc..
MLA:
Kazi, Anees, et al. "Coupled Manifold Learning for Retrieval Across Modalities." Proceedings of the 16th IEEE International Conference on Computer Vision Workshops, ICCVW 2017, Venice, ITA Institute of Electrical and Electronics Engineers Inc., 2017. 1321-1328.
BibTeX: Download