UPC Collective Operations Optimization

Abstract

In any parallel programming language; collective communication operations involve more than one thread/process and act on multiple streams of data. The language’s API provides both algorithmic and run-time system support to optimize the performance of these operations. Some developers, however, choose to play clever and start from the language’s primitive operations and write their own versions of the collective operations. The question that always pops up: Are these developers wise? In this paper, we check the case of UPC (Universal Parallel C) and prove that in some circumstances, it is wiser for developers to optimize starting from UPC’s primitive operations. In our testing we found out that optimization using primitive UPC operations by the developers can have better performance than readily available UPC’s collective operations. In this paper, we pin point specific optimizations at both the algorithmic and the runtime support levels that developers could use to uncover missed optimization opportunities. We also propose a novel approach to implementing UPC collective operations across clusters. Under this methodology, performance-critical components are moved close to the network. We argue that this provide unique advantages for performance improvement.