Category-Level Segmentation of Repeated Objects using SAM2 Memory System
Wang J, Kayser M, Sun Y (2026)
Publication Type: Conference contribution
Publication year: 2026
Publisher: Association for Computing Machinery, Inc
Pages Range: 197-202
Conference Proceedings Title: ICAAI 2025 - 2025 9th International Conference on Advances in Artificial Intelligence
Event location: Manchester, GBR
ISBN: 9798400721045
Abstract
In this paper, we propose a novel segmentation approach that adapts the memory module of the Segment Anything Model 2 (SAM2) [15] for a category-sensitive segmentation within images. By combining SAM2’s Automatic Mask Generator (AMG) with a cross-modal attention mechanism, we build a dynamic memory bank that enables robust semantic associations of objects across frames. To mitigate challenges such as over-segmentation and reflection artifacts, we introduce an innovative fine-tuning strategy between the image encoder and mask decoder. This strategy substantially improves segmentation robustness while maintaining the model’s zero-shot generalization capability. Comparative experiments on ARMBench and real-world datasets demonstrate that our model performs effectively in both zero-shot and fine-tuned settings, adapting seamlessly to diverse environments. Ultimately, our approach empowers robotic systems with enhanced perceptual capabilities to recognize objects from selected categories within images.
Authors with CRIS profile
Involved external institutions
Germany (DE)How to cite
APA:
Wang, J., Kayser, M., & Sun, Y. (2026). Category-Level Segmentation of Repeated Objects using SAM2 Memory System. In ICAAI 2025 - 2025 9th International Conference on Advances in Artificial Intelligence (pp. 197-202). Manchester, GBR: Association for Computing Machinery, Inc.
MLA:
Wang, Jiayi, Matthias Kayser, and Yipeng Sun. "Category-Level Segmentation of Repeated Objects using SAM2 Memory System." Proceedings of the 2025 9th International Conference on Advances in Artificial Intelligence, ICAAI 2025, Manchester, GBR Association for Computing Machinery, Inc, 2026. 197-202.
BibTeX: Download