GitOps SDI

Infrastructure as Code inspired Spatial Data Infrastructures

Paul van Genuchten, Tom Kralidis

2025-06-15

Contents

• What is GitOps?
• Why GitOps?
• Metadata at the source
• About the workshop
• Technologies

What is GitOps

• “Git” & “Operations” (like DevOps)

• Git is a source of truth (apps, containers, configuration)

• A GitOps process automatically delivers changes in an online cluster.

• A monitoring process detects difference between desired state and the target cluster

5 principles

• Express the desired state declaratively
• Version control the desired state
• Automatically apply the desired state
• Reconcile the desired state
• Visibility and control

Why GitOps?

At ISRIC - World Soil Informtion

• We run various SDI’s to facilitate projects
• SDI’s should be robust, requiring little maintenance

Metadata at the source

• Researchers use many field and earth observation data to predict distribution of Soil properties

• They use tools like Excel, R and Python, generally not aware of metadata standards

• Add a README.txt to describe the resources in a folder

• Why not structure the content in the README, so also machines can read it?

About the workshop

• A self guided tutorial at https://pvgenuchten.github.io/training-gitops-sdi/
• Two moments to summarize
• Welcoming your feedback at https://forms.gle/J59DjGbTBx1s1Tus7
• Ask questions via https://github.com/pvgenuchten/training-gitops-sdi/discussions/5

Technologies

• MCF
• pygeometa
• pyGeoDataCrawler
• pycsw
• mapserver
• terriajs

Metadata Control File

• A convention of the pygeometa community (Meteo Canada)
• YAML, easy to read by humans and machines
• A subset of the ISO19115 standard for metadata

MCF

pygeometa

• a library to work with MCF
• exports MCF to various metadata models (iso, dcat, stac)
• https://github.com/geopython/pygeometa

pyGeoDataCrawler

• Developed at ISRIC in the last year
• On the shoulders of giants; GDAL, pygeometa, pandas
• Generates MCF files for a file repository
• Imports metadata from external sources
• Extracts MCF files from a repository to load it into a catalogue
• Creates map services configuration for files in the repository

pycsw catalogue

• A catalogue implementation in Python
• Limited funtionality, easy maintenance
• Wide range of supported catalogue standards, serves many communities
• Database backend
• Skin customisable with Jinja templates

Related products; GeoNetwork, CKAN, dataverse

MapServer

• Recent versions of MapServer support the old and new standards
• Features and Maps
• No tiles (mapproxy), No coverages yet
• No records/stac, by design

Related products; GeoServer, QGIS server, pygeopai

TerriaJS

An Open Source javascript WebGIS application

Related products; GeoNode, GeoMoose, MapBender, Oskari, GisQuick

Quarto

• Static site generator from Markdown
• Add diagrams or maps with Python and R scripting
• Other interesting option is the W3C MapML