MPI reduction and broadcast algorithms for computer clusters with multistage interconnection networks
https://doi.org/10.55648/1998-6920-2023-17-3-57-69
Abstract
Algorithms for MPI Allreduce, Bcast and Reduce collective operations have been developed. They are based on the dynamic construction of process groups at all levels of the memory hierarchy within the computing node and levels of the multistage interconnection network. The description of the network topology is loaded from an external file. Аlgorithms are implemented on the basis of the Open MPI library in an isolated collective component. Experiments on a cluster with a two-tier InfiniBand EDR network have shown the effectiveness of the algorithms for messages up to 16KB.
About the Author
M. G. Kurnosov
Siberian State University of Telecommunications and Information Science (SibSUTIS)
Russian Federation
Russian Federation
Mikhail G. Kurnosov - Dr. of Sci. (Engineering), Professor; Professor of the Department of Computer Systems SibSUTIS; Senior Research Scientist, ISP SB RAS.
630102, Novosibirsk, Kirov St. 86), phone: +7 383 2698 272; 630090, Novosibirsk, Lavrenteva ave. 13, phone: +7 383 3305 626
References
1. Thakur R., Rabenseifner R., Gropp W. Optimization of collective communication operations in MPICH. Int. Journal of High Performance Computing Applications, 2005, vol. 19 (1), pp. 49-66.
2. Balaji P., Buntinas D., Goodell D., Gropp W., Hoefler T., Kumar S., Lusk E., Thakur R., Traff J. MPI on Millions of Cores. Parallel Processing Letters, 2011, vol. 21, iss. 1, pp. 45-60.
3. Graham R. Cheetah: A Framework for Scalable Hierarchical Collective Operations. Proc. of the IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, 2011, pp. 73-83.
4. Jain S., Kaleem R., Balmana M., Langer A., Durnov D., Sannikov A. and Garzaran M. Framework for Scalable Intra-Node Collective Operations using Shared Memory. Proc. of the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC-2018), 2018, pp. 374-385.
5. Zhu H., Goodell D., Gropp W., Thakur R. Hierarchical Collectives in MPICH2. Proc. of EuroPVM/MPI, 2009, pp. 325–326.
6. Luo X. HAN: a Hierarchical AutotuNed Collective Communication Framework. Proc. of the IEEE International Conference on Cluster Computing (CLUSTER), 2020, pp. 23-34.
7. Kurokawa M. The K computer: 10 Peta-FLOPS supercomputer. Proc. of the 10th International Conference on Optical Internet (COIN2012), 2012, pp. 1.
8. Kumar S., Mamidala A., Heidelberger P., Chen D., Faraj D. Optimization of MPI Collective Operations on the IBM Blue Gene/Q Supercomputer. Journal of High Performance Computing Applications, 2014, no. 28(4), pp. 450-464.
9. Venkata M., Bloch G., Shainer G., Graham R. Accelerating OpenSHMEM Collectives Using InNetwork Computing Approach. Proc. of the International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), 2019, pp. 212-219.
10. Kurnosov M. G. Ierarhicheskij algoritm bar'ernoj sinhronizacii dlya mnogoprocessornyh sistem s obshchej pamyat'yu [Barrier synchronization hierarchical algorithm for multicore shared-memory systems]. Vestnik SibGUTI, 2022, vol. 16, no. 2(58), pp. 3-11.
Supplementary files
Review
For citations:
Kurnosov M.G. MPI reduction and broadcast algorithms for computer clusters with multistage interconnection networks. The Herald of the Siberian State University of Telecommunications and Information Science. 2023;17(3):57-69. (In Russ.) https://doi.org/10.55648/1998-6920-2023-17-3-57-69
Views:
436
Email this article 
