Middleware for the Remote Instrumentation e-Infrastructure

The tremendously large number and the heterogeneous potential of Grid resources makes the middleware promoting those resources to the various users in research and production to be a significant effort topic in the e-Infrastructure. Set up of the middleware architecture which facilitates Remote Instrumentation for the applications from the selected e-Science communities is one of the strategic objectives of DORII. Middleware composes an important layer of the e-Infrastructure, intended to promote the Grid resources for the e-Science communities exploiting and providing Remote Instrumentation Services.

This page aims at potential users of the DORII middleware solutions and presents the DORII Middleware Architecture and its main components, followed by an additional information about releases, quality assurance and middleware users' support.

Outline

1. Introduction

The DORII middleware architecture set-up follows the principle of adoption of sustainable outcomes of the already existing developments, consolidating them in a common framework and adapting them to the requirements of the Remote Instrumentation.

The architecture relies on gLite which is a middleware for Grid computing, provided by the (EGEE)-Grid. The architecture extends the gLite services with the solutions for interconnecting instruments with the e-Infrastructure (outcome of the GRIDCC project). Moreover, the architecture covers the end-users' needs in accessing the e-Infrastructure, developing the applications, and facilitating the applications' special features (e.g. enhanced visualization, interactivity providing, and parallel processing), consolidating the user-level solutions, developed in the projects g-Eclipse, GRIDCC, Interactive European Grid (I2G), VLAB, Open MPI and others.

See the description of the middleware solutions adopted by the architecture in the next section Overview of the DORII Middleware Architecture .

2. Overview of the DORII Middleware Architecture

The architecture consists mainly of the tools depicted in the figure below.

center

The architecture is compatible with the gLite middleware, which offers a set of the core Grid services - Information (BDII), Job Management (CE, WMS), Data Management (SE, LFC), and Security (VOMS, MyProxy) Services, deployed on the infrastructure resources included to DORII.

  • For more detailed description of gLite, please refer to the gLite homepage

Complementary to the core gLite service, the architecture provides the solutions listed in the table below.

REMOTE INSTRUMENTATION SERVICES
Component Description Details
DORII Instrument Element Middleware that enables easy integration of the various devices like scientific instrumentation and sensors in the Grid IE
TOOLS FOR USER AND APPLICATION SUPPORT ON THE GRID
Component Description Details
DORII Virtual Control Room Web­ based Grid portal that provides the user interface to the Grid as well as the richly featured collaboratory environment VCR
DORII Workflow Management System Client / Server tool that allows creation and execution of complex workflows for scientific applications WfMS
g-Eclipse An access and development platform whose framework provides tools to customize Grid users' applications, manage Grid resources and support the development cycle of new Grid application g-Eclipse
DORII Common Library The client side to the underlying Grid middleware and related components (currently g-Lite). Built largely upon the g­Eclipse code, the CL serves as the building blocks for the Virtual Control Room and Workflow Management System. CL
PARALLEL APPLICATIONS' SUPPORT
Component Description Details
Open MPI Full MPI-2 standard’s implementation for the support of parallel applications Open MPI
MPI-Start Tool for improved running MPI applications in Grid environments for different MPI libraries, file systems and job managers of the Grid resources. MPI-Start
ADVANCED VISUALIZATION TECHNOLOGY
Component Description Details
GVid Grid-enabled video streaming service that is specifically designed for grid-based visualizations GVid
GLogin Transport layer provider that sets up an interactive bi-directional channel between the grid and the user’s local desktop machine GLogin

Adopting those tools by the common architecture, DORII pursued the development process from existing releases in order to ensure that the full potential of the Remote Instrumentation Infrastructure and Grid technologies is enabled for the e-Science. Guided by users' and applications' requirements and feedback, we significantly improved the user and developer support on the Grid, redesigning the user's Grid frontend and collaborative environment (VCR) as well as providing solutions for construction and management of scientific applications' workflow. Moreover, the Instrument Element solution was reimplemented from the scratch, enabling the remote instrumentation concept for a wide range of scientific devices controlled and operated remotely. Furthermore, addressing the complex issues of the component integration, the DORII developments will set up a flexible environment that provides consistent and enjoyable developer and end­-user experience. The basic element of the environment is a Java Common Library that acts as the client side to the underlying Grid middleware and related components (g-Lite). Whereas the DORII Common Library is intended to be a building block for the VCR and WfMS tools, it will be beneficial for use in other projects dealing with the Remote Instrumentation and generally Grid technologies as well. The best practices of our research and developments will be also promoted to other development communities like g-Eclipse and Open MPI.


3. Public Releases

The goals of the DORII project include among others the release of a number of open-source software components. The user feedback is important for the DORII development as considered to be an important improvement guideline for the middleware components.

The Release 1.0 of the DORII middleware was done at the end of M12 of the project (January 2009) and contained the prototype of integrated products. The prototype allowed for early deployment of the pilot applications on the DORII e-Infrastructure. The feedback, provided by the end users with suggestions of new features and improvements of existing ones for the middleware components, was taken as a guideline for the realization of the Plan for Adaptation of Components and resulted in the second release of the DORII middleware Release 2.0. Release 2.0 is of production quality and serves a development and fast prototyping framework for the DORII applications. Release 3.0 was published in March 2010 and includes all the requested features and changes proposed by the application providers. The latest release (3.1), published in August 2010, contains major bug fixes and additional features proposed for Release 3.0.


The Latest Release (3.1)

Previous Releases

Release and Testing Procedure

DORII Integration Team (IT) has been established as a technical discussion forum for the DORII project. Its activities focus on quality assurance of the adopted software components and tools and their adequate integration. DORII-IT released the procedures document that aims to define and to promote the best practices for the DORII software releases in order to obtain quality software with adequate integration that provides consistent and enjoyable end-user experience. The evaluation results can be found here (accessible only for middleware developers)


4. Source code access, DORII@GForge

The source code of the DORII software solutions is open source and maintained using Subversion - a code version control system. Complementary to the redistributable packages, users can get access to the code using any of SVN clients (see link for details).

The address of the DORII@SVN repository is https://svn.gforge.hlrs.de/svn2/dorii-jra/

To access the DORII@SVN, a user needs a special account, which can be purchased contacting the DORII support team at mailing list


5. User Support and Contact

For user support, DORII utilizes GForge.

Back to top
DORII project receives funding from the EC's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° RI-211693.