Middleware-level collaborative resource discovery for large clusters

In this paper, we introduce a collaborative resource discovery mechanism to enhance system resource allocation and utilization by the distributed applications. The mechanism involves the collaboration of the software agents to collect, organize and efficiently exchange resources information. The main goal is to provide an accurate single system image for all the resources. The concept of virtual clusters is used to enhance localization of information and minimize exchanged messages. This mechanism is introduced as a part of the run-time support layer in Delmon. Delmon is a three-layer middleware framework for distributed systems, which includes the run-time support layer. This layer utilizes agents that reside on the participating machines and communicate with one another to perform the required functions. The agents hide the details of scheduling, controlling, monitoring, and executing user jobs, while the management of system resources is made transparent to the user. The proposed mechanism for collaborative resource discovery is based on the hierarchical structure of the agents and the organization and update protocols.