The Worldwide Large Hadron Collider Computing Grid celebrates the start of its crucial data challenge: the analysis and management of more than 15 million Gigabytes of data every year, to be produced from the hundreds of millions of subatomic collisions expected inside the LHC every second. This data-handling feat marks an essential stage in the process of enabling researchers to discover new physics.
The Worldwide LHC Computing Grid combines the IT power of more than 140 computer centres, the result of collaboration between 33 countries.
“Our ability to manage data at this scale is the product of several years of intense testing,” said Ian Bird, leader of the Worldwide LHC Computing Grid project. “Today’s result demonstrates the excellent and successful collaboration we have enjoyed with countries all over the world. Without these international partnerships, such an achievement would be impossible.”
“The Worldwide LHC Computing Grid is a vital pillar of the LHC project,” said Jos Engelen, chief scientific officer for the LHC project. “It is an absolute necessity for analysis of the LHC data. It is the result of a ‘silent revolution’ in large scale computing over the last five years.”
The Worldwide LHC Computing Grid relies on dedicated optical fibre networks to distribute data from CERN to eleven major computer centres in Europe, North America and Asia. From these, data is dispatched to more than 140 centres around the world. Together, these distributed computers provide the power to manage the LHC’s data.
“We can routinely process 250,000 jobs a day,” said Bird, “and we can achieve peaks of 500,000 jobs without problems.” A single job can be a calculation lasting several hours or even several days on a single high performance processor. An estimated 100,000 processors are needed to handle all jobs from the LHC experiments.
Bird says physics is not the only field benefiting from grid computing. “Many other researchers and projects are already benefiting from the lessons learned here. Grid computing is enabling all-new ways of doing science where large data handling and analysis capabilities are required,” Bird said.
“When the LHC starts running at full speed, it will produce enough data to fill about six CDs per second,” said Michael Ernst, director of Brookhaven National Laboratory’s Tier-1 Computing Center.
Computing grids such as Enabling Grids for E-Science (EGEE) in Europe and Open Science Grid (OSG) in the U.S. not only contribute their power to the Worldwide LHC Computing Grid, but also contribute to other scientific projects, covering biology, chemistry, medicine, climate science and more.
CERN acknowledges the continued support of its Member States and other collaborating countries, in particular the additional voluntary funding for computing provided during the development phase of the project. The European Commission and other Funding Agencies around the world have supported grid computing for many years with great benefit to LHC Computing.
“We would also like to acknowledge the excellent partnership CERN has with Hewlett-Packard, Intel and Oracle within CERN openlab,” said von Rüden, “as well as the recent contributions of EDS. These partnerships have contributed to the success of the grid.”