Datacubes for EO and Geosciences
Datacubes address the need for fast and efficient access, analysis and processing of geospatial data along space and time, notably Big Earth Data: Earth Observations, IoT sensor data, environmental variables, simulation data, etc. The European Space Agency has been supporting the development of Datacube solutions such as the Advanced geospatial Data Management platform (ADAM), or the Earth System Data Lab (ESDL), and is looking towards advancing the complete service offering (data, processing, applications) at European Level in a fully transparent way for the user, with the Euro Data Cube initiative. A range of different technical Datacube implementations are in place in Geosciences/Earth Sciences today, looking to address the challenge of Massive Data Processing at Scale. One the one hand, datacube solutions for EO data provide efficient access to remote sensing data natively stored in the original grids, spatial reference systems, file formats and resolutions, avoiding data redundancies and facilitating the direct exploitation of the temporal dimension (e.g. commercial solutions such as SciDB or Rasdaman, but also coordinated efforts such as the CEOS Open Data Cube). On the other hand, datacubes for Earth System Science are built around singular physical datacubes using a common spatiotemporal grid, that typically include a high number of variables as a dimension, along space and time. One example is the Earth System Data Lab which enables applications and global studies related to ecosystem states, interactions between Earth’s sub-systems (e.g. biosphere-atmosphere), modeling of fluxes and process understanding, among others. Further solutions for Earth Sciences developed with ESA support such as the Ocean Virtual Lab or the Climate Change Initiative look to address domain-specific scientific needs in terms of data access, processing and visualisation tools. As implementations are reaching maturity, needs for datacube federation, interoperability and new technical interfaces are emerging, and new ways are sought to effectively integrate Global EO data with other sources and Analysis Ready Products, as well as to balance the drastic data volume increase with data access latency. This evolution furthermore generates the need for standardisation, putting EO & Earth Science datacubes on the agenda of organisations such as the OGC. This session will address these specificities of the datacubes ecosystem with a focus on: Datacube technologies, Processing at Scale, Data-intensive Science and Global ARD challenges, and discussing emerging challenges such as interoperability and federation.
- Generating Global products in Datacubes –challenges and solutions (SINERGISE)
- Harmonisation and Interoperability for ARD (Planet/ OGC)
- EuroDataCube – Cube Service for Earth Observation Data Exploitation (ESA / Brockman)
- Data-Intensive and Reproducible Science in Datacubes (http://project-dare.eu)
- Datacubes for Earth Science –Earth System Data Lab (Miguel)