Managing TMI with IT

September 16th, 2016

Forbes magazine recently profiled an overseas physics experiment powering up in an underground lab at the French-Swiss border. The experiment will involve the use of as much electricity as it takes to power the entire city of Geneva.

The article sites the colloquial term “TMI” to describe the challenges of “too much information” for the team of innovators working at the Large Hadron Collider, guided by Ian Bird, a particle physicist and enterprise project leader for the computing grid.
The job of the collider is to accelerate tiny particles, almost to the speed of light, smash them together and take snapshot pictures of the collisions. These images are expected to provide insight into the basic composition of matter and the workings of the universe.

With five collision detectors at the collider, each approximately five stories tall and operating much like  giant digital cameras, together they comprise 150 million sensors, each snapping 40 million pictures a second: thus, too much information, more data than the computing system is equipped to handle. In a fraction of a second, the detectors spin out such a quantity of data that a system for identifying and parsing critical information is essential.

The Scientists at the Large Hadron Collider are learning the processes integral to separating the collider’s essential signals from its noise, in order to accumulate the optimum data, which, says Bird, “is a matter of learning what data you can throw away. Most of it just isn’t interesting.” The article suggests that the operations team, assembled just last year, began with a conservative approach to tossing out extraneous information. As they have gained experience, the thinning of data has improved. Eventually, only the best information will be saved. Even with extraneous data eliminated, the computers will be storing 15 petabytes a year. That’s a stack of 1.7 million DVDs.

Given the billions of dollars invested in what is empirically new territory, in collecting the data the project requires, the certainty of very finely calibrated detectors is paramount. “You need to have confidence that there are no bugs in your data” says Bird, “you need to have lots of young researchers.”

LTI is committed to assembling and retaining the industry’s best and brightest, most innovative talent. Click here to learn more about our world-class High Performance Computing teams.