Dissertation
Tuning the Tuner: The Art of Automatic Performance Optimization
Modern scientific discovery increasingly depends on High-Performance Computing (HPC) systems that combine traditional processors with specialized accelerators such as GPUs.
- Author
- F.Q. Willemsen
- Date
- 14 April 2026
- Links
- Thesis in Leiden Repository
While these systems offer enormous computational power, achieving high performance is challenging: small decisions in how programs organize data, structure computation, or exploit parallelism can lead to large performance differences. Automatic performance tuning (auto-tuning) addresses this challenge by automatically exploring many program configurations to identify efficient ones. However, the effectiveness of auto-tuning depends strongly on how the tuning process itself is designed.
This thesis improves auto-tuning for GPUs by advancing both the algorithms and the infrastructure used to search large configuration spaces. First, it introduces a new Bayesian optimization method tailored to GPU tuning that outperforms existing approaches. Second, it proposes a fair and reproducible methodology for evaluating and comparing tuning strategies. Third, it presents a high-performance solver for constrained search spaces that can analyze billions of possible configurations in under a second, removing a key bottleneck in many tuning systems. The thesis also demonstrates that tuning the hyperparameters of tuning algorithms themselves can yield performance improvements exceeding 200%, while enabling better energy efficiency through large-scale simulations. In addition, evolutionary algorithms are extended to better handle constraints commonly present in GPU programs. Finally, the work explores whether artificial intelligence can design new optimizers automatically, showing that machine-generated algorithms can outperform human-designed ones.
Together, these contributions demonstrate that improving the design of auto-tuning systems can significantly amplify the performance gains they deliver. By releasing all methods and tools as open-source software, this research aims to make advanced auto-tuning techniques broadly accessible and to support the development of more efficient and adaptive HPC applications, contributing to faster progress in science and technology powered by computers that tune themselves to the challenges of tomorrow.