Filtering and queryables API

HDA service allows to filter by attributes in order to create customised data selections. For each collection, a set of attributes is defined to build a filtered query. These attributes can be identified using the /queryables endpoint for the collection of interest.

So, for ‘ERA5-Land monthly averaged data from 1950 to present’ we can use request

https://hda.data.destination-earth.eu/stac/collections/EO.ECMWF.DAT.ERA5_LAND_MONTHLY/queryables

It ends with queryables, meaning it is a /queryables endpoint. They produce JSON response, containing a listing of the attributes which can be filtered on, together with their types collection.

Under the properties key-value pair property, we can see further details of the queryable attributes. This information will include the attribute name, type, title and the default value applied.

In case of the ecmwf:product_type attribute, filtering is based on string type with a default set to monthly_averaged_reanalysis.

For the same attribute, we can see a list of enumerated possible values:

"ecmwf:product_type": {
   "default": "monthly_averaged_reanalysis",
   "enum": [
     "monthly_averaged_reanalysis",
     "monthly_averaged_reanalysis_by_hour_of_day"
   ],
   "title": "product_type",
   "type": "string"
 },

Here is Python code to use the queryables endpoint, sending several HTTP requests in order to build a correct search request for the given dataset.

Setup

We start off by importing the relevant modules for HTTP requests and json handling.

import destinelab as deauth
import requests
import json
from getpass import getpass
from IPython.display import JSON

And defining constants for the API URLs:

# Collection https://hda.data.destination-earth.eu/ui/dataset/EO.ECMWF.DAT.CAMS_EUROPE_AIR_QUALITY_FORECASTS as default
COLLECTION_ID = "EO.ECMWF.DAT.ERA5_LAND_MONTHLY"

# Core API
HDA_API_URL = "https://hda.data.destination-earth.eu"

# STAC API
## Core
STAC_API_URL = f"{HDA_API_URL}/stac"

## Queryables
QUERYABLES_URL = f"{STAC_API_URL}/queryables"
QUERYABLES_BY_COLLECTION_ID = f"{COLLECTIONS_URL}/{COLLECTION_ID}/queryables"
HDA_FILTERS =''

Build the query from the selected values

The QUERYABLES_BY_COLLECTION_ID returns the applicable filters under the section named properties.

The properties section contains

• the name of the filter, title,

• the filter type,

• the possible filter values, enum (conditioned by the values selected for the other filters)

• and the the default (or chosen) default applied

response = requests.get(QUERYABLES_BY_COLLECTION_ID)

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "https://hda.data.destination-earth.eu/stac/collections/EO.ECMWF.DAT.ERA5_LAND_MONTHLY/queryables",
  "type": "object",
  "title": "Queryables for EODAG STAC API",
  "description": "Queryable names for the EODAG STAC API Item Search filter. ",
  "properties": {
    "ecmwf:product_type": {
      "default": "monthly_averaged_reanalysis",
      "enum": [
        "monthly_averaged_reanalysis",
        "monthly_averaged_reanalysis_by_hour_of_day"
      ],
      "title": "product_type",
      "type": "string"
    },
    "ecmwf:variable": {
      "default": "2m_dewpoint_temperature",
      "enum": [
        "10m_u_component_of_wind",
        "10m_v_component_of_wind",
        "2m_dewpoint_temperature",
        "2m_temperature",
        "evaporation_from_bare_soil",
        "evaporation_from_open_water_surfaces_excluding_oceans",
        "evaporation_from_the_top_of_canopy",
        "evaporation_from_vegetation_transpiration",
        "forecast_albedo",
        "geopotential",
        "glacier_mask",
        "high_vegetation_cover",
        "lake_bottom_temperature",
        "lake_cover",

The queryables API presents the appropriate filters for the selected dataset, determined by the chosen values. Selection one of the variables, narrows down the choice for other variables.

params = {
 "ecmwf:product_type":  "monthly_averaged_reanalysis_by_hour_of_day",
 "ecmwf:variable": "evaporation_from_bare_soil"
}

response = requests.get(QUERYABLES_BY_COLLECTION_ID,params)

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$id": "https://hda.data.destination-earth.eu/stac/collections/EO.ECMWF.DAT.ERA5_LAND_MONTHLY/queryables",
  "type": "object",
  "title": "Queryables for EODAG STAC API",
  "description": "Queryable names for the EODAG STAC API Item Search filter. ",
  "properties": {
    "ecmwf:product_type": {
      "default": [
        "monthly_averaged_reanalysis_by_hour_of_day"
      ],
      "enum": [
        "monthly_averaged_reanalysis",
        "monthly_averaged_reanalysis_by_hour_of_day"
      ],
      "title": "product_type",
      "type": "string"
    },
    "ecmwf:variable": {
      "default": [
        "evaporation_from_bare_soil"
      ],
      "enum": [
        "10m_u_component_of_wind",
        "10m_v_component_of_wind",
        "2m_dewpoint_temperature",
        "2m_temperature",
        "evaporation_from_bare_soil",
        "evaporation_from_open_water_surfaces_excluding_oceans",
        "evaporation_from_the_top_of_canopy",
        "evaporation_from_vegetation_transpiration",
        "forecast_albedo",

A JupyterLab notebook to help using the queryables API is available here: DEDL - HDA Tutorial - Queryables