Adaptive load balancing of distributed SPMD computations: a transparent approach

Efficient parallel computing on distributed platforms still presents many obstacles. This paper addresses the important issue of masking the power heterogeneity and variability of non-dedicated nodes. To this purpose, we present a load balancing support that autonomously adapts the workload of Single Program Multiple Data (SPMD) applications to platform conditions. This support checks the load status of the nodes at the beginning and during program execution and, if necessary, carries out data migrations from overloaded to underloaded nodes without requiring the programmer to insert load balancing primitives. As additional important contribution to the transparency and efficiency of the framework, we propose a stochastic model for the automatic choice of the optimum interval of activation of the load balancer. Unlike task migration supports for task parallelism and other data migration frameworks for master/slave-based applications, our load balancer is transparent and works for the en...