Workshops and Tutorials at PACT 2026

Reproducible Benchmarking for High-Performance Computing Applications

Scheduling: Monday, October 19 (afternoon)

Tutorial Presenters:

Olga Pearce

Lawrence Livermore National Laboratory

Gregory Becker

Lawrence Livermore National Laboratory

Doug Jacobsen

Google

Stephanie Brink

Lawrence Livermore National Laboratory

Abstract

Benchmarking is integral to HPC system procurement, to communicating center workloads to vendors, and to verifying the performance of delivered systems. Yet HPC benchmarking remains largely manual and challenging across the full workflow, creating a high barrier to entry and limiting reproducibility across systems. This hands-on half-day tutorial introduces Ramble and Benchpark, two open-source tools that help users encode build and run instructions, describe systems and experiments, and conduct reproducible benchmarking studies for HPC applications.

The tutorial will walk attendees through the practical lifecycle of reproducible benchmarking: understanding the abstractions behind Ramble experiments, defining experiment workflows and metrics, learning how Benchpark encodes hardware systems and benchmarking experiments, and applying both tools in hands-on exercises. Participants will run and modify Ramble experiments, configure scaling studies, bootstrap Benchpark, initialize experiments on a target system, and execute reproducible benchmarking workflows. Attendees will leave with foundational skills and reusable patterns for defining, running, sharing, and reproducing HPC benchmarking experiments on current and future systems.

Bios

Olga Pearce is a computer scientist in the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory. She leads the Performance Analysis and Visualization for Exascale project and LLNL’s Advanced Technology Systems Benchmarking effort. Olga holds a joint appointment in the Department of Computer Science and Engineering at Texas A&M University, where she teaches Parallel Computing and advises students. Her research interests include parallel algorithms and optimizations, performance modeling and optimization, application load balancing, parallel programming models, communication on heterogeneous architectures, performance tools, and system benchmarking. She has been at LLNL since 2007 and has received multiple Weapons Simulation Codes and Livermore Computing awards for her work on RAJA, application porting and optimization on Sierra, and Advanced Technology Systems procurements. She received her Ph.D. in Computer Science from Texas A&M University in 2014.

Gregory Becker is a computer scientist at Lawrence Livermore National Laboratory. His work focuses on bridging the gap between research and production software at LLNL. His software productization work has included contributions to Spack, a package manager for high-performance computing, as well as scalable I/O formats for performance tools. Gregory has been at LLNL since 2015. He received his B.A. in Computer Science and Mathematics from Williams College in 2015.

Doug Jacobsen is a software engineer at Google. He received his Ph.D. and M.S. degrees from Florida State University in 2011 and 2009, respectively, both in Scientific Computing with a focus on geophysical fluid dynamics. His work focuses on the performance and optimization of high-performance computing applications and workloads. He created the Ramble experimentation framework to accelerate the exploration of performance-affecting changes in workloads.

Stephanie Brink is a computer scientist at Lawrence Livermore National Laboratory. She received her Ph.D. and M.S. degrees from the University of Oregon in 2019 and 2016, respectively. Her research interests include high-performance computing, power-constrained computing, performance tools, and scientific visualization.

Back to Workshops