Reducing Communication in the Conjugate Gradient Method: A Case Study on High-Order Finite Elements
Karp M, Jansson N, Podobas A, Schlatter P, Markidis S (2022)
Publication Type: Conference contribution
Publication year: 2022
Publisher: Association for Computing Machinery, Inc
Conference Proceedings Title: Proceedings of the Platform for Advanced Scientific Computing Conference, PASC 2022
Event location: Basel, CHE
ISBN: 9781450394109
Abstract
Currently, a major bottleneck for several scientific computations is communication, both communication between different processors, so-called horizontal communication, and vertical communication between different levels of the memory hierarchy. With this bottleneck in mind, we target a notoriously communication-bound solver at the core of many high-performance applications, namely the conjugate gradient method (CG). To reduce the communication we present lower bounds on the vertical data movement in CG and go on to make a CG solver with reduced data movement. Using our theoretical analysis we apply our CG solver on a high-performance discretization used in practice, the spectral element method (SEM). Guided by our analysis, we show that for the Poisson equation on modern GPUs we can improve the performance by 30% by both rematerializing the discrete system and by reformulating the system to work on unique degrees of freedom. In order to investigate how horizontal communication can be reduced, we compare CG to two communication-reducing techniques, namely communication-avoiding and pipelined CG. We strong scale up to 4096 CPU cores and showcase performance improvements of upwards of 70% for pipelined CG compared to standard CG when applied on SEM at scale. We show that in addition to improving the scaling capabilities of the solver, initial measurements indicate that the convergence of SEM is largely unaffected by pipelined CG.
Authors with CRIS profile
Involved external institutions
Sweden (SE)How to cite
APA:
Karp, M., Jansson, N., Podobas, A., Schlatter, P., & Markidis, S. (2022). Reducing Communication in the Conjugate Gradient Method: A Case Study on High-Order Finite Elements. In Proceedings of the Platform for Advanced Scientific Computing Conference, PASC 2022. Basel, CHE: Association for Computing Machinery, Inc.
MLA:
Karp, Martin, et al. "Reducing Communication in the Conjugate Gradient Method: A Case Study on High-Order Finite Elements." Proceedings of the 2022 Platform for Advanced Scientific Computing Conference, PASC 2022, Basel, CHE Association for Computing Machinery, Inc, 2022.
BibTeX: Download