API Architecture

Understanding the HDA API structure and how components work together to provide unified access to the DestinE ecosystem.

What is the DEDL HDA API?

The DEDL HDA API (Harmonised Data Access API) is the main access layer of the Destination Earth Data Lake (DEDL). It provides a unified, standardized interface for users and services to:

  • Discover Earth observation (EO), Digital Twin, in-situ, statistical data

  • Request data from diverse collections

  • Retrieve multi-petabyte datasets seamlessly

The API abstracts away the complexity of accessing data stored across different locations, formats, and providers, presenting everything through a single harmonized interface.

API Structure

The HDA API consists of three main endpoints working together:

Services API

/services

Discover cloud-native edge services, processing tools, and computational resources.

No authentication required

STAC API v2

/stac/v2

STAC 1.0.0 compliant interface for data discovery and access.

Recommended

STAC Legacy

/stac

Original STAC implementation for backward compatibility.

Deprecated

Core Components

The HDA API structure includes:

Catalog Endpoints

  • List available collections and metadata

  • STAC and OGC API - Records compliant

  • Collection capabilities and schemas

Search & Query Interface

  • Spatial, temporal, and thematic filtering

  • Free-text search across datasets

  • Dataset queryables and parameters

Data Access & Ordering

  • Synchronous (immediate) downloads

  • Asynchronous (job-based) workflows

  • Job creation, monitoring, and retrieval

Authentication & Authorization

  • OAuth2-based bearer tokens

  • Access control for secured datasets

  • Quotas and rate limits

Distributed Architecture

The DEDL HDA API is distributed by design:

../../../_images/HDAdiagram.png

The API presents a unified interface while operating on a distributed backend architecture for:

Feature

Description

Scalability

  • Data hosted across multiple storage nodes

  • Geographic distribution for global access

  • Load balancing across endpoints

Resilience

  • Redundancy and fault tolerance

  • Service availability even if nodes fail

  • Backup data access paths

Data Locality

  • Access data closer to compute resources

  • Reduced transfer latencies

  • Optimized for Digital Twin workflows

Federation

  • Collections from multiple institutions

  • Cross-cloud provider integration

  • Uniform access despite diverse backends

Role within Destination Earth

The HDA API serves as the interface layer between users and the data lake’s data products.

Component

Description

Users & Applications

  • Scientists and researchers

  • Third-party applications

  • Automated workflows

Data Infrastructure

  • Multi-petabyte storage

  • Distributed data nodes

  • Processing capabilities

Federated Data Providers

  • ECMWF, ESA, EUMETSAT datasets

  • Remote storage integration

  • Cross-provider harmonization

Key Functions

Function

Description

Data Discovery

Browse 200+ collections, search with filters, metadata access

Unified Access

Single API for diverse data sources and formats

Services Discovery

Browse available edge services, processing tools, and computational resources

Access Control

Secure authentication and usage monitoring

FAIR Principles

Findable, Accessible, Interoperable, Reusable data

STAC API Versions

STAC API v2 (Recommended)

Current Interface - /stac/v2

  • STAC 1.0.0 compliant

  • Enhanced ECMWF integration

  • Dedicated /order endpoints

  • Direct /data access endpoints

  • Standardized error handling

STAC Legacy (Deprecated)

Legacy Interface - /stac

  • Original implementation

  • Backward compatibility only

  • Limited feature support

  • Will be removed in the near future

  • Migrate to v2 for new projects

Next Steps

Quick Start Guide

Get started with authentication and first API calls

API Guide

Complete guides for all API functionality