Back to search

University of Alabama Birmingham

A Visual Guide to Balancing Latency and Bandwidth in MPI All-to-all

Abstract

dc:description.abstract

The complex nature of parallel algorithms in the Message Passing Interface (MPI) standard challenges the ability of trivially visualizing them. Within MPI libaries (e.g., MPICH, Open-MPI), there are many collectives that offer different solutions for communicating data between processes. The collective MPI_Alltoall uses various techniques such as the spread-out and Bruck algorithms to have processes transfer data between all processes. Each technique within MPI_Alltoall is best suited for different scenarios. The spread-out algorithm uses a linear number of iterations, in process count P, while the Bruck algorithm is logarithmic. The Bruck algorithm transfers more data overall, but with fewer communication steps, and is thus better suited for smaller sized (latency-dominated) messages. The Bruck algorithm can be modified to take a parameterized input for the log base (radix). By increasing this radix from the original log base of 2, blocks transferred will decrease while communication steps will increase. MPI libraries use a decision tree to determine which algorithm to use in collectives such as MPI_Alltoall. This thesis details and implements an interactive web-based visualization tool to help a user, such as graduate students studying topics in High Performance Computing (HPC), understand the relation between the log-step Bruck, linear-step spread-out, and log-step tunable radix algorithm (TRA) in the collective MPI_Alltoall. The tool visually illustrates and animates the three algorithms, pointing out key differences such as number of iterations, communication patterns, and whether they are in-place. This work lays the groundwork for developing ways to visualize all algorithms in MPI collectives.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Computer and Information Sciences

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Netterville, Naeris
Contributors dc:contributor
  • <p>Thomas Gilray</p>
  • <p>John Johnstone<br>Sidharth Kumar</p>

Subjects

dc:subject × 3

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
University of Alabama Birmingham
Base URL
digitalcommons.library.uab.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Netterville, Naeris. A Visual Guide to Balancing Latency and Bandwidth in MPI All-to-all. Thesis thesis, https://digitalcommons.library.uab.edu/etd-collection/7638