2 December 2024

Data Processing

LPC2E’s Applied Computing team develops software for a wide range of needs:

  • data processing chains for the various instruments produced in the laboratory and embedded on satellite missions,
  • command/control tools for piloting and rapidly analyzing data from instruments under development,
  • remote sensing processing and model inter-comparison tools for earth observation satellites (for which the laboratory is not involved in development).
  • data visualization and analysis tools for the scientific teams’ various projects
  • specific tools

These software programs are developed in coordination with in-house scientific and technical teams, and sometimes with international partners such as other laboratories, data centers and space agencies.

Instrument data arrive in raw form (optimized and compressed by on-board electronics). The data may be scientific (values measured by the instruments) or technical (status and operating modes of the instruments). Their processing usually requires additional data from the satellite (from the platform or other instruments) or from elsewhere (ground instruments, other satellites, models, etc.). These data are then reconstructed, processed and referenced with respect to time and geophysical parameters. Calibrations and corrections linked to the instrumental response and the surrounding environment are applied to the data during processing, to take it from its raw state to an intermediate level.

To enable fast, efficient processing, the applied computing team develops ad-hoc software. These software programs are used to quickly identify and analyze scientifically useful events in the data.

The laboratory also provides missions with high-level data, resulting from the analysis of lower-level data. Specific and complex processing chains are implemented to interpret the data and extract these physical parameters.

The applied IT team, in conjunction with researchers, can also develop specific tools such as:

  • Software for visualization and production of quicklooks (rapid data visualization) (Figure 1)

Figure 1 – Example of quicklook for the instrument SCM on the Solar Orbiter mission

 

  • programs for real-time or a posteriori tracking of instrumented balloon and aircraft flight paths (see figure 2)
  • satellite data visualization programs for inter-comparison of balloon- and aircraft-borne instrument data with satellite data

 

Figure 2 — Exemple de trajectoire d’un vol avion lors de la campagne DACCIWA en juillet 2016. La teneur en CO mesurée par l’instrument SPIRIT est indiquée par l’échelle de couleur le long du trajet.

 

Datasets and other products must comply with the various archiving standards of the scientific community (CDF, NetCDF, HDF5, ames…) to ensure their durability and distribution, as well as their interfacing with the archiving centers of agencies such as CNES, ESA and NASA

Processing software programs are produced at LPC2E, but their execution can be integrated into multi-instrument processing chains (consortium) operated by partners, so every piece of software must comply with development rules and software quality standards.