Software and Data Engineering

Software Engineering is an essential aspect of many of our science projects, from fundamental research, or experimental projects, to operational observatories. Additionally, the continuously increasing volume and complexity of the data involved requires efficient techniques for their manipulation and exploitation.

The Software and Data Division of the ICCUB-Tech has expertise on the highest quality grade software development, from on-ground massive data processing and mining to embedded or on-board systems for satellites.

The following are the main projects where our experts are involved:

Gaia:

The ICCUB has played a key role in the definition, implementation and operation of the data simulation, processing and exploitation of the Gaia space astrometry mission. Our experts have important responsibilities in the Data Processing and Analysis Consortium (DPAC) of Gaia, remarkably: CU2 (simulations), CU3 (core processing), CU5 (photometric processing), CU9 (Catalogue Access) and in the Data Processing Center of Barcelona (comprising BSC and CSUC).

The knowledge acquired during more than a decade of Gaia development and operations cover a variety of techniques and solutions, such as:

DIRAC:

Researchers from the Institute of Cosmos Sciences of the University of Barcelona have developed DIRAC, a software package for managing high-power commercial computing systems that optimizes the use of large-scale resources. The software is already used to manage data processing in one of the main experiments carried out at the Large Hadron Collider (LHC), operated by the European Organization for Nuclear Research. The aim of the DIRAC project –funded under the CPAN project of the National Centre for Particle, Astroparticle and Nuclear Physics as part of the Consolider-Ingenio 2010 program – is to develop the software's capability to manage the computing resources needed by users in all areas of the scientific community. The software has been successfully tested in simulations carried out as part of the Belle experiment (Japan), using 2,000 Amazon Elastic Compute Cloud (EC2) processors.

DIRAC manages the execution of the data processing systems used in the Large Hadron Collider (LHCb) experiment to identify the different types of particles measured. It also controls the execution of the algorithms used to select the most relevant data from the huge volume recorded (10 million collisions are needed to reconstruct a single Beauty Particle, which provides a basis for the study of the asymmetry between matter and antimatter in the universe). DIRAC also distributes the results across the Worldwide LHC Computing Grid (WLCG), a network of more than 300 computing centres in 57 countries across the world, 120 of which contribute to the LHCb project, and retrieves the data required for post-experiment analysis.