Alternative TitleResource Contention due to Data Movement on HPC Systems
Note (General)Large-scale high-performance systems (HPC), or supercomputers, are composed of hundreds/thousands of nodes that are interconnected using advanced network topologies. One such topology is the fat-tree topology, which provides high-bandwidth and low-latency communications. However, network communication has become a major source of performance bottleneck, even on fat-tree networks, due to significant growth in the sizes of HPC workloads. Ensuring good communication performance remains challenging because of the complexities of the communication operations, architectures, and workloads, et al. This work introduces an approach to conducting performance analysis of MPI applications running on large-scale networks by efficiently and portably exposing metrics from within MPI the layer via the Peruse interface. The solution features the creation of the ibprof profiler to record InfiniBand traffic data and a generalized performance visualization toolset to accurate combine application-specific information with hardware configuration information of fat-tree networks. ibprof incurs an average communication overhead of 2.06% for the MPI Alltoall microbenchmark, 7.63% for the MPI Bcast microbenchmark, and 0.02% for the NPB FT application benchmark. Furthermore, case studies demonstrate that the visualization toolset can identify communication anomalies in applications as well as communication libraries and can also guide performance optimization strategies. Another important contribution of this work is a characterization of the interference between the MPI and I/O traffic on fat-tree networks. The characterization study was done using the packet-level-accurate network simulations and highlights the effects of varying the message size, communication interval, and job size. The results show that MPI traffic is more sensitive to interference than I/O traffic in all cases, with light I/O traffic exhibiting a 1.9X slowdown due to heavy MPI interference and light MPI traffic exhibiting a 7.6X slowdown due to heavy I/O traffic. Furthermore, a significant feature of I/O traffic patterns was identified and labeled as the I/O-congestion threshold. This threshold is the I/O interval (or frequency of sending I/O requests) where the self-congestion from I/O is more detrimental than the contention due to MPI interference. This threshold varies with the I/O request size and the job size. The network topology-specific feature and the characteristics of the interference are used to demonstrate that, independently, specific node allocation and I/O server placement strategies can completely avoid the I/O-MPI interference. Additionally, throttling I/O traffic based on its I/O-congestion threshold can reduce the slowdown of MPI performance by approximately 200% while incurring only an 18% performance penalty.
identifier:oai:t2r2.star.titech.ac.jp:50421499
Collection (particular)国立国会図書館デジタルコレクション > デジタル化資料 > 博士論文
Date Accepted (W3CDTF)2019-10-04T14:36:02+09:00
Data Provider (Database)国立国会図書館 : 国立国会図書館デジタルコレクション