CERN openlab for DataGrid applications

Annual Report for 2003
Short contribution to IT Department Annual Report

 

The CERN openlab for DataGrid applications is a framework for evaluating and integrating cutting-edge technologies or services in partnership with industry, focusing on potential solutions for the LCG. The openlab invites members of the industry to join and contribute systems, resources or services, and carry out with CERN large-scale highly-performing evaluation of their solutions in an advanced integrated environment.

In a nutshell, the major achievements in 2003 were: the successful incorporation of two new partners: IBM and Oracle; the consolidation and expansion of the opencluster (a powerful compute and storage farm); the start of the gridification process of the opencluster;  the 10 Gbps challenge where very high transfer rates were achieved over LAN and WAN distances (the latter in collaboration with other groups); the organization of three thematic workshops including one on Total Cost of Ownership; the creation of a new, lighter, category of sponsors called contributors; the implementation of the openlab student programme,  bringing some 11 students in the summer.

Management
The project is formally led by the IT Department Head, seconded by the Associate Head, the Chief Technology Officer and the Communication and Development officer (the latter function will be provided in 2004 by the departmental Strategy and Communication unit).

Industrial Sponsors
The year 2003 started with three sponsors, Enterasys Networks

(Contributing high bit-rate network equipment), Hewlett Packard (computer servers and fellows), Intel Corporation (64-bit processors technology and 10 Gbps Network Interface Cards). In March 2003, IBM joint the openlab (to contribute hardware and software disk storage solution), followed by Oracle Corporation (to contribute Grid technology and fellows).

The annual Board of Sponsors meeting was successfully held the 13th of June, and the annual report issued at this occasion. In addition, three Thematic Workshops were organized (on Storage and Data Management, Fabric Management and Total Cost of Ownership). On the latter topic (TCO), a position paper establishing the facts and figures was produced.

In order to permit time-limited incorporations of sponsors to fulfil specific technical missions, a concept of contributor was devised and proposed to existing sponsored. Contributor status (as opposed to partner status for existing sponsors) implies lower financial commitment and correspondingly lesser benefits in terms of influence and exposure.

Technical progress

The openlab is constructing the opencluster, a pilot compute and storage farm based on HP's dual processors machines, Intel's Itanium Family Processors (IFP) processors, Enterasys's 10-Gbps switches, IBM’s Storage Tank system and Oracle’s 10g Grid solution.

In 2003, the opencluster was first expanded with 32 servers (RX2600) equipped with IPF processors (second generation, 1 GHz) and running Red Hat Linux Enterprise Server 2.1 and open AFS and LSF. In October, 16 servers equipped with IPF’s third generation processors (1.3 GHz) were added. This is complemented with 7 development systems. 

The concept of openlab technical challenge –where tangible objectives are jointly targeted by some or all of the partners- was proposed to the sponsors. The first instantiation was the 10 Gbps Challenge, a common effort by Enterasys, HP, Intel and CERN.  In this context, a first experiment where two Linux-based HP computers with 1GHz IPF processors directly connected (back-to-back through 10 GbE Network Interface Cards) reached 5.7 Gbps for memory-to-memory transfer (single stream). The transfer took place over a 10 km fibre. To extend tests over WAN distances, collaborations took place with the DataTag project and the ATLAS DAQ group. Using openlab IPF-based servers as end-systems, DataTag and Caltech established a new world Internet-2 land-speed record. Extensive tests with Enterasys’s ER16 router demonstrated that 10 Gbps rates could only be achieved through multiple parallel streams. An upgrade strategy, including the use of Enterasys new N7 devices in 2004 was agreed between Enterasys and CERN.

On the front of storage, a 30TB disk sub-system (6 meta-data servers and 8 disk servers) was installed, using IBM’s StorageTank solution. Performance tests will be conducted in 2004.

Porting to IFP of physics applications (in collaboration with EP/SFT) and CERN systems continued in 2003, including for Castor, CLHEP, GEANT4 and ROOT.  Other groups also ported their applications (including ALIROOT by ALICE Collaboration, CMSIM by CMS US). Results of scalability tests with PROOF were reported to the CHEP2003 conference. As another example of collaboration with other groups, twenty of the IPF servers were used by ALICE for their 5th Data Challenge.

The gridification effort culminating with the porting of the LCG middleware (based on VDT and EDG). After some difficulties, porting was almost completed at the end of the year. HP Lab’s SmartFrog monitoring system was evaluated. As first results are promising, the effort will continue in 2004.

Dissemination and Development activities

In addition to the thematic workshops organized in the framework of the technical programme, two papers were published in the Proceedings of the CHEP2003 conference and one article was published in the CERN Courier and 3 joint press releases were issued.  The openlab also hosted at CERN two meetings of the First Tuesday Suisse Romande series, involving active participation of openlab partners.

Following a series of meetings, a document exploring the possibilities for development of the openlab in the field of security was produced.

Based on a pilot programme run in 2002, a CERN openlab student programme was run in the summer 2003, involving 11 students from seven European countries. Four of these students contributed directly to the opencluster activity; the others worked on the Athena experiment and on the development of the Grid Café web site. The later was successfully demonstrated at the Telecom2003 exhibition and at the SIS-Forum, part of the World Summit on Information Technology event.

Resources

The openlab integrates technical and managerial efforts from several IT groups: ADC (Technical Management; opencluster via two fellows who joined in 2003; StorageTank); CS (10 GbE networking); DB (Oracle 10g); DI (Project management, communication)

 

F.Fluckiger