A Transformation-Based Approach to Developing High-Performance GPU Programs

Abstract

We advocate the use of formal patterns and transformations for programming modern many-core processors like Graphics Processing Units (GPU), as an alternative to the currently used low-level, ad hoc programming approaches like CUDA or OpenCL. Our new contribution is introducing an intermediate level of low-level patterns in order to bridge the abstraction gap between popular high-level patterns (${map}$, fold/reduce, ${zip}$, etc.) and imperative, executable code for many-cores. We define our low-level patterns based on the OpenCL programming model which is portable across parallel architectures of different vendors, and we introduce semantics-preserving rewrite rules that transform programs with high-level patterns into programs with low-level patterns, from which executable OpenCL programs are automatically generated. We show that program design decisions and optimizations, which are usually applied ad-hoc by experts, are systematically expressed in our approach as provably-correct transformations for high- and low-level patterns. We evaluate our approach by systematically deriving several differently optimized OpenCL implementations of parallel reduction that achieve performance competitive with OpenCL programs which are manually written and highly tuned by performance experts.

Appendices

Appendix

A Additional Rewrite Rules

B Proof of a Rewrite Rule

Rewrite rules are proved using equational reasoning. As an example we prove rule (25) which introduces layers in the computation hierarchy of a reduction: first a partial reduction is computed, followed by a reduction combining all temporary results.

Proof

(Reduce-Promotion Variant). Let n be a number divisible by m.

C Derived Low-Level Reduction Programs

Fig. 8.

Two more low-level programs implementing parallel reduction. They are equivalent to the fourth and the (seventh) most optimized version described in [8], correspondingly