Common use of Messaging Clause in Contracts

Messaging. All current middleware implementations lack a standard mechanism for exchanging information among services and between the middleware and higher level consumers like accounting and monitoring tools. Although effort has been put in the past to define how 15 X. Xxxxx, extensible Access Control Markup Language (XACML) Version 2.0, February, 2005, xxxx://xxx.xxxxx-xxxx.xxx/committees/tc_home.php?wg_abbrev=xacml - XACML20 the information must be encoded, there is no common transport service independent of the information content. Such a transport service would provide standardized access to information and the development of common ways of managing policies, configurations and service requests. EMI will add such service to its middleware starting from existing off-the-shelf components like the Apache ActiveMQ or the MRG messaging service from Red Hat, extending them to comply with the standard security mechanisms used by the grid services. • Accounting An efficient accounting mechanism compliant with the distributed usage of the grid service is necessary to provide fair access to the resources and in future establish real business models. Various accounting solutions are presently available, but they all make use of proprietary formats and transport mechanisms. EMI will adapt the most important accounting sensors and collectors to use the common messaging infrastructure and a standard data format based on the UR specification, extending it to provide accounting for more types of operations like data management and providing a uniform foundation for the accounting tools used by the infrastructure providers. • Service monitoring and management Standard service monitoring and management interfaces are currently largely absent from grid middleware, but their necessity is becoming obvious in the context of running large scale production infrastructures. The correct approach to grid service management is still discussed in relevant standardisation forums and no clear recipe exists yet. However, established technologies are available and used in many commercial applications based on standard definitions like the Common Information Model (CIM), developed by IBM, Microsoft and Intel among others, and the QPID Modeling Framework (QMF) recently adopted by the leading Linux provider, Red Hat. EMI will investigate and adapt off-the- shelf solutions and develop sensors to be plugged in industry standard monitoring tools, such as Nagios and standard CIM-based tools. • MPI The Message Passing Interface (MPI) programming paradigm is poorly supported in grid environments. Users face a complex programming environment where too many low level details have to be specified to execute even the simplest parallel jobs. EMI will provide an interoperable MPI execution framework across the different middleware implementations to abstract the user interfaces from the underlying middleware and allow users to execute parallel applications in a uniform way, thus bridging the gap between HPC and HTC. • Virtualization and clouds access Recently, cloud technology emerged as a new business model for managing distributed computing and storage resources. Clouds focus on creating well-controlled uniform environments, predominantly through virtualization techniques, and usually in one administrative domain. This guarantees high service levels and transparent accountability, both of which have proved to be a challenge for Grids. However, in the context of international research communities, the top priority is the sharing of existing heterogeneous resources (computing, data, and information) and the creation of a single administrative domain is often unfeasible. This makes Grid an essential technology for distributed computing infrastructures to carry on research today, but interoperability with cloud-based solutions for increased flexibility and ease of use and the adoption of resource management solutions based on virtualization are becoming strong user requirement. A number of important initiatives have started or are being proposed to strengthen the use of Clouds for research (RESERVOIR, OpenNebula, Platform ISF, INFN WNoDeS16, etc), but a clear path towards bridging the different models in a transparent way for the end users is still missing. EMI will work together with providers and developers of cloud solutions, identifying and implementing missing functionality in the grid services to make them use efficiently such solutions as they become mature. In particular, extensions to the current job definition language (JSDL) and the resource modelling schema (GLUE) to be able to request specific virtualized environments to be instantiated by the underlying resource managers are currently needed. • Interoperability between HTC and HPC Interoperability between high-performance and high-throughput computing is receiving increasing attention as user communities like Fusion, Computational Chemistry or Astronomy need to access both supercomputers and shared grid nodes in their computational and data management workflows. Convergence to common standards across the UNICORE and traditional grid services will help bridge this gap. However, standards conformance alone may not be sufficient, as large differences still exist on the level of policies, resource allocations etc. EMI will investigate in detail the differences and provide documented solutions to exploit the specific capabilities of the two systems in different stages of complex computational tasks. • Portlets Portlets are pluggable components that generate dynamic contents in response to web requests. The content that a portlet produces is a fragment of mark-up code that, in aggregation with other non-overlapping portlet results, contributes to the creation of the portal page. The availability of standard portlet interfaces for the middleware services allows the design and creation of user-level portals and gateways that isolate the users from the underlying complexities of the middleware providing direct, customizable functionality. EMI will provide standard web portal interfaces for its services compliant with specifications like the Java Portlet Specifications (v1.0 JSR 168, v2.0 JSR 286), WSRP (WS for Remote Portlets) or the very recent Enterprise Mashups technologies17 16 INFN OCCI implementation on Grid Infrastructure (xxxx://xxx.xxx.xxx/OGF27/materials/1742/03_INFN_OCCI_implementation.pptx) and will closely collaborate with user communities and projects to support their development of portals and gateways. • Standardization EMI primary commitment is the support and evolution of the grid middleware used by infrastructures and user communities to perform scientific research as part of international collaborative efforts. Such an engagement cannot be undertaken without a parallel commitment to define and adopt standards in collaboration with similar initiatives in Europe and worldwide. EMI will actively participate to the effort of established standards development organizations like the Open Grid Forum18 and OASIS19 helping to define and refine new and existing standards. In particular EMI commits to use the OGF Production Grid Infrastructures Working Group (PGI-WG) as a forum for ensuring that EMI is engaged with the rest of the world‘s production level middleware with special attention to initiatives taking place in the United States in the context of the Open Science Grid (OSG) infrastructure. In this context, the major innovative contribution that EMI will bring is the actual and validated implementation of the standards in its supported services and their transition to production infrastructures in collaboration with EGI, PRACE and other providers.

Messaging. All current middleware implementations lack a standard mechanism for exchanging information among services and between the middleware and higher level consumers like accounting and monitoring tools. Although effort has been put in the past to define how 15 X. Xxxxx, extensible Access Control Markup Language (XACML) Version 2.0, February, 2005, xxxx://xxx.xxxxx-xxxx.xxx/committees/tc_home.php?wg_abbrev=xacml - XACML20 the information must be encoded, there is no common transport service independent of the information content. Such a transport service would provide standardized access to information and the development of common ways of managing policies, configurations and service requests. EMI will add such service to its middleware starting from existing off-the-shelf components like the Apache ActiveMQ or the MRG messaging service from Red Hat, extending them to comply with the standard security mechanisms used by the grid services. •  Accounting An efficient accounting mechanism compliant with the distributed usage of the grid service is necessary to provide fair access to the resources and in future establish real business models. Various accounting solutions are presently available, but they all make use of proprietary formats and transport mechanisms. EMI will adapt the most important accounting sensors and collectors to use the common messaging infrastructure and a standard data format based on the UR specification, extending it to provide accounting for more types of operations like data management and providing a uniform foundation for the accounting tools used by the infrastructure providers. •  Service monitoring and management Standard service monitoring and management interfaces are currently largely absent from grid middleware, but their necessity is becoming obvious in the context of running large scale production infrastructures. The correct approach to grid service management is still discussed in relevant standardisation forums and no clear recipe exists yet. However, established technologies are available and used in many commercial applications based on standard definitions like the Common Information Model (CIM), developed by IBM, Microsoft and Intel among others, and the QPID Modeling Framework (QMF) recently adopted by the leading Linux provider, Red Hat. EMI will investigate and adapt off-the- shelf solutions and develop sensors to be plugged in industry standard monitoring tools, such as Nagios and standard CIM-based tools. •  MPI The Message Passing Interface (MPI) programming paradigm is poorly supported in grid environments. Users face a complex programming environment where too many low level details have to be specified to execute even the simplest parallel jobs. EMI will provide an interoperable MPI execution framework across the different middleware implementations to abstract the user interfaces from the underlying middleware and allow users to execute parallel applications in a uniform way, thus bridging the gap between HPC and HTC. •  Virtualization and clouds access Recently, cloud technology emerged as a new business model for managing distributed computing and storage resources. Clouds focus on creating well-controlled uniform environments, predominantly through virtualization techniques, and usually in one administrative domain. This guarantees high service levels and transparent accountability, both of which have proved to be a challenge for Grids. However, in the context of international research communities, the top priority is the sharing of existing heterogeneous resources (computing, data, and information) and the creation of a single administrative domain is often unfeasible. This makes Grid an essential technology for distributed computing infrastructures to carry on research today, but interoperability with cloud-based solutions for increased flexibility and ease of use and the adoption of resource management solutions based on virtualization are becoming strong user requirement. A number of important initiatives have started or are being proposed to strengthen the use of Clouds for research (RESERVOIR, OpenNebula, Platform ISF, INFN WNoDeS16, etc), but a clear path towards bridging the different models in a transparent way for the end users is still missing. EMI will work together with providers and developers of cloud solutions, identifying and implementing missing functionality in the grid services to make them use efficiently such solutions as they become mature. In particular, extensions to the current job definition language (JSDL) and the resource modelling schema (GLUE) to be able to request specific virtualized environments to be instantiated by the underlying resource managers are currently needed. •  Interoperability between HTC and HPC Interoperability between high-performance and high-throughput computing is receiving increasing attention as user communities like Fusion, Computational Chemistry or Astronomy need to access both supercomputers and shared grid nodes in their computational and data management workflows. Convergence to common standards across the UNICORE and traditional grid services will help bridge this gap. However, standards conformance alone may not be sufficient, as large differences still exist on the level of policies, resource allocations etc. EMI will investigate in detail the differences and provide documented solutions to exploit the specific capabilities of the two systems in different stages of complex computational tasks. •  Portlets Portlets are pluggable components that generate dynamic contents in response to web requests. The content that a portlet produces is a fragment of mark-up code that, in aggregation with other non-overlapping portlet results, contributes to the creation of the portal page. The availability of standard portlet interfaces for the middleware services allows the design and creation of user-level portals and gateways that isolate the users from the underlying complexities of the middleware providing direct, customizable functionality. EMI will provide standard web portal interfaces for its services compliant with specifications like the Java Portlet Specifications (v1.0 JSR 168, v2.0 JSR 286), WSRP (WS for Remote Portlets) or the very recent Enterprise Mashups technologies17 16 INFN OCCI implementation on Grid Infrastructure (xxxx://xxx.xxx.xxx/OGF27/materials/1742/03_INFN_OCCI_implementation.pptx) and will closely collaborate with user communities and projects to support their development of portals and gateways. •  Standardization EMI primary commitment is the support and evolution of the grid middleware used by infrastructures and user communities to perform scientific research as part of international collaborative efforts. Such an engagement cannot be undertaken without a parallel commitment to define and adopt standards in collaboration with similar initiatives in Europe and worldwide. EMI will actively participate to the effort of established standards development organizations like the Open Grid Forum18 and OASIS19 helping to define and refine new and existing standards. In particular EMI commits to use the OGF Production Grid Infrastructures Working Group (PGI-WG) as a forum for ensuring that EMI is engaged with the rest of the world‘s production level middleware with special attention to initiatives taking place in the United States in the context of the Open Science Grid (OSG) infrastructure. In this context, the major innovative contribution that EMI will bring is the actual and validated implementation of the standards in its supported services and their transition to production infrastructures in collaboration with EGI, PRACE and other providers.