CloverLeaf on Intel Multi-Core CPUs: A Case Study in Write-Allocate Evasion

Laukemann J, Gruber T, Hager G, Oryspayev D, Wellein G (2024)

Publication Language: English

Publication Status: Published

Publication Type: Conference contribution, Conference Contribution

Publication year: 2024

Publisher: Institute of Electrical and Electronics Engineers Inc.

Pages Range: 350-360

Conference Proceedings Title: 2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

Event location: San Francisco, CA US

ISBN: 9798350337662

DOI: 10.1109/IPDPS57955.2024.00038

Abstract

In this paper we analyze the MPI-only version of the CloverLeaf code from the SPEChpc 2021 benchmark suite on recent Intel Xeon "Ice Lake" and "Sapphire Rapids" server CPUs. We observe peculiar breakdowns in performance when the number of processes is prime. Investigating this effect, we create first-principles data traffic models for each of the stencil-like hotspot loops. With application measurements and microbenchmarks to study memory data traffic behavior, we can connect the breakdowns to SpecI2M, a new write-allocate evasion feature in current Intel CPUs. For serial and full-node cases we are able to predict the memory data volume analytically with an error of a few percent. We find that if the number of processes is prime, SpecI2M fails to work properly, which we can attribute to short inner loops emerging from the one-dimensional domain decomposition in this case. We can also rule out other possible causes of the prime number effect, such as breaking layer conditions, MPI communication overhead, and load imbalance.

Authors with CRIS profile

Jan Laukemann Erlangen National High Performance Computing Center (NHR@FAU) Thomas Gruber Erlangen National High Performance Computing Center (NHR@FAU) Georg Hager Regionales Rechenzentrum Erlangen (RRZE) Gerhard Wellein Professur für Höchstleistungsrechnen

Additional Organisation(s)

Erlangen National High Performance Computing Center (NHR@FAU)

Related research project(s)

DatenREduktion für Exascale- Anwendungen in der Fusionsforschung (DaREXA-F) DatenREduktion für Exascale- Anwendungen in der Fusionsforschung Sept. 1, 2022 - Aug. 31, 2025

Involved external institutions

Brookhaven National Laboratory US United States (USA) (US)

How to cite

APA:

Laukemann, J., Gruber, T., Hager, G., Oryspayev, D., & Wellein, G. (2024). CloverLeaf on Intel Multi-Core CPUs: A Case Study in Write-Allocate Evasion. In 2024 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (pp. 350-360). San Francisco, CA, US: Institute of Electrical and Electronics Engineers Inc..

MLA:

Laukemann, Jan, et al. "CloverLeaf on Intel Multi-Core CPUs: A Case Study in Write-Allocate Evasion." Proceedings of the 38th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2024, San Francisco, CA Institute of Electrical and Electronics Engineers Inc., 2024. 350-360.

BibTeX: Download