An OCS server on its own

Open Climate Service and no Chap at all: a climate data server for anything that speaks STAC or openEO.

chap-devops init climate --only ocs,s3
cd climate
chap-devops up
chap-devops status
chap-devops open ocs

The first chap-devops up pulls the OCS image, about 3 GB. After that, a start takes some seconds. chap-devops status then shows:

ocs   up   http://localhost:8790   0 datasets   32.0 KB data
s3    up   internal

both components are up

You get OCS on http://localhost:8790, with the S3-compatible store beside it. OCS does not read the store yet: it keeps its data on its own ocs_data volume (The object store says why the store is there). Nothing Chap-specific is created: no compose.yml, and no chap-core release is looked up. It worked when chap-devops status shows ocs as up and chap-devops open ocs opens its web interface.

To leave out the object store, use this init instead of the first one. OCS works the same, on the same volume:

chap-devops init climate --only ocs

Ingesting a first dataset

CHIRPS3 rainfall is public and needs no account. Three days of it, over the instance’s extent:

curl -X POST http://localhost:8790/ingestions \
  -H 'Content-Type: application/json' \
  -d '{"dataset_id": "chirps3_precipitation_daily", "start": "2024-01-01", "end": "2024-01-03", "publish": true}'
curl http://localhost:8790/stac/collections/chirps3_precipitation_daily

The first command waits until the data is in, about 20 seconds. It worked when it answers with "status":"completed" and the second returns the collection; chap-devops status then counts one dataset:

ocs   up   http://localhost:8790   1 dataset   424.0 KB data

The data size can be a little different on your machine.

The extent is Laos until you change it, the country of the DHIS2 demo database chap-devops seeds, so data ingested here covers the provinces that DHIS2 holds case data for. Change it in ocs/climate-service.yaml, or at init with --ocs-name, --ocs-country and --ocs-bbox; curl http://localhost:8790/extent shows the one in use.

Which datasets there are

curl http://localhost:8790/data-sources

lists every data source OCS knows, and "ingestable": true marks the ones /ingestions takes. The ones Chap models use:

Data source credentials says where the two accounts go.

A range worth forecasting with

start and end are written in the dataset’s own period: YYYY-MM-DD for a daily one, YYYY-MM for a monthly one and YYYY for a yearly one. Six years of monthly rainfall, the years the demo DHIS2 has cases for:

curl -X POST http://localhost:8790/ingestions \
  -H 'Content-Type: application/json' \
  -d '{"dataset_id": "chirps3_precipitation_monthly", "start": "2019-01", "end": "2024-12"}'

The request waits until the ingestion is complete. For Laos that took between two and eleven minutes, as the download speed changes, and curl shows nothing in that time. With -H 'Prefer: respond-async' it answers 202 at once instead, with a Location of /ingestions/jobs/ID to poll. Leaving out end ingests up to the current period, and sending the same request again answers at once with the earlier ingestion rather than downloading it twice. curl 'http://localhost:8790/datasets?f=json' lists what has been ingested.

Getting it out

Values per district or province, as the CSV Chap reads: OCS averages the dataset over each area in a GeoJSON FeatureCollection whose feature ids become the location column. A DHIS2 hands out its org units in that shape, so with the demo DHIS2 beside OCS (chap-devops components enable dhis2, then chap-devops up; or --only ocs,s3,dhis2 at init, option 10 on the AI page). chap-devops up returns while DHIS2 still starts. Run chap-devops status again until dhis2 shows up, then run:

curl -u admin:district -o provinces.geojson \
  'http://localhost:8780/api/organisationUnits.geojson?level=2'
python3 - > request.json <<'PY'
import json
areas = json.load(open("provinces.geojson"))
print(json.dumps({"process": {"process_graph": {"agg": {
    "process_id": "aggregate_to_chap_csv",
    "arguments": {"dataset_id": "chirps3_precipitation_monthly",
                  "temporal_extent": ["2024-01-01", "2024-12-31"],
                  "geometries": areas, "method": "mean", "period_type": "month"},
    "result": True}}}}))
PY
curl -X POST http://localhost:8790/result \
  -H 'Content-Type: application/json' -d @request.json -o rainfall.csv

It worked when rainfall.csv starts with time_period,location,precip and has one row per province and month (216 for the 18 provinces over 2024). The values are what the dataset holds: CHIRPS3 is a rate in mm per day.

The whole grid over a bounding box, as NetCDF, for anything that reads it (xarray, QGIS, Panoply):

curl -X POST http://localhost:8790/result -H 'Content-Type: application/json' -o rainfall.nc -d '{
  "process": {"process_graph": {
    "load": {"process_id": "load_collection", "arguments": {
      "id": "chirps3_precipitation_monthly",
      "temporal_extent": ["2024-01-01", "2024-12-31"],
      "spatial_extent": {"west": 100.0, "south": 13.9, "east": 107.7, "north": 22.5}}},
    "save": {"process_id": "save_result",
      "arguments": {"data": {"from_node": "load"}, "format": "NetCDF"}, "result": true}}}}'

GET /datasets/ID/download answers 409 for data OCS keeps in its default storage; /result is the way out. Both calls above still work on a read-only instance, which refuses ingestion but not reading.

As a server

Two settings matter once other people use it:

chap-devops components enable ocs --read-only                          # refuse writes over HTTP
chap-devops restart ocs                                                # apply read-only
chap-devops components enable ocs --base-url https://ocs.example.org   # behind a proxy
chap-devops up                                                         # apply the base URL

--read-only changes ocs/climate-service.yaml. OCS reads that file only when it starts, so the command tells you to restart it:

ocs is now read-only
run `chap-devops restart ocs` to apply

chap-devops up does not apply this change. It leaves OCS as it is, and it prints this warning on stderr. Until you run chap-devops restart ocs, chap-devops status shows the same warning:

warning: `ocs/climate-service.yaml` changed after the ocs container started, so ocs may still use the old settings; run `chap-devops restart ocs` to apply it

chap-devops restart ocs returns before OCS answers again. Run chap-devops status until ocs shows up. It worked when this answers 403:

curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:8790/ingestions -d '{}'

A read-write OCS answers 422 to the same request. The base URL is different: it is in compose.ocs.yml, so chap-devops up applies it. --base-url says what it set:

enabled ocs on http://localhost:8790
set the ocs base URL to https://ocs.example.org
run `chap-devops up` to apply

If you give the same base URL again, nothing changes, and the command prints no chap-devops up line:

ocs was already enabled on http://localhost:8790; nothing changed

It worked when the links in curl http://localhost:8790/stac/catalog.json start with the base URL.

Both have init spellings too, --ocs-read-only and --ocs-base-url. A read-only instance refuses every ingestion over HTTP, so ingest first.

Next: Standalone OCS, Data source credentials, Read-only instances, Behind a reverse proxy.

All shapes: Use cases.