The Mini-Grid Framework: Application Programming Support for Ad hoc Volunteer Grids

To harvest idle, unused computational resources in networked environments, researchers have proposed different architectures for desktop grid infrastructure. However, most of the existing research work focus on centralized approach. In this thesis, we present the development and deployment of one such infrastructure, called the Mini-Grid Framework for resource management in ad hoc grids using market-based scheduling and context-based resource and application modeling. The framework proposes peer-to-peer architecture that supports several futures: ease of deployment, decentralized task distribution, small scale ad hoc grid formation, and symmetric resource. Furthermore, users can model and specify non-performance based parameters that influence resource allocation.

We evaluated the framework through simulation experiments at the IT University of Copenhagen (ITU), as well as a pilot deployment at the Interdisciplinary Nanoscience Center (iNano), Aarhus University. For the simulation experiments we used an application that calculates prime numbers between a given range, and another application that searches for a key in a large data set. In the simulation experiments, we studied the technical performance and overhead of the Mini-Grid Framework and compared its performance with other relevant systems. For the pilot deployment, we have integrated a parallelized version of the Basic Local Alignment Search Tool (BLAST) algorithm and the RNA secondary structure prediction algorithm developed by iNano research center with the Mini-Grid Framework. These algorithms have been developed on top of our framework through an integration of the framework with the CLC Bio Workbench, a software suite for bioinformatics algorithms. The pilot deployment studied the resource participation, the deployment efforts needed, and the performance of the framework in a real grid environment.

The main contribution of this thesis are: i) modeling entities such as resources and applications using their context, ii) the context-based auction strategy for dynamic task distribution, iii) scheduling through application specific quality parameters, iv) the definition of an extensible API for ad hoc grid formation and v) enabling symmetric resource participation. The Mini-Grid framework has been designed and developed as proof-of-concept. The Mini-Grid framework has been evaluated using LAB deployment at ITU, and has been deployed at iNano research center using real-life application.