Fresh Data Pool

The Destination Earth Data Lake - Fresh Data Pool stores a subset of frequently accessed data covering the following types of data:

  • Earth Observation Data (EODATA)

  • Socio-Economic Data / Other Data

Fresh Data Pool can be considered as a kind of internal cache for Destination Earth Data Lake, and is used in the following ways:

Fresh Data Pool (EODATA)

The Destination Earth Data Lake (DEDL) provides access to the Fresh Data Pool (EODATA) via an S3-compatible API. This API follows the same syntax and behavior as standard S3 APIs but uses DEDL’s own object storage backend.

Key Points:

  • Data of type EODATA is accessible via the S3-compatible API.

  • Tools like s3cmd, boto3, and similar can facilitate the use of the S3-Compatible API.

  • Would allow mounting this EODATA data in a VM file system using s3fuse for example

  • Credentials are required, available through My DataLake Services application.

Endpoint: - https://eodata.data.destination-earth.eu

Fresh Data Pool (Socio-Economic, Other)

Accessing Fresh Data Pool through HDA

The Fresh Data Pool accelerates Data Access when using the HDA (Harmonised Data Access) Service.

When using the HDA it will transparently check to see if the data is already available in the Fresh Data Pool.

  • If the data is in the Fresh Data Pool the HDA will serve the data from there

  • If the data is not in the Fresh Data Pool the HDA will access data from the federated providers