data

hydromodpy.data orchestrates input data acquisition, validation, and integration. Seventeen variables share the same Variable / Manager / Source pattern, with a DuckDB cache that records every fetched artefact for cache-hit detection and reproducibility.

Sub-modules

• data/base_manager_variable.py – BaseVariableManager ABC for point variables (gauges, observations).

• data/base_manager_field.py – BaseFieldManager ABC for field variables (rasters, gridded forcing).

• data/_base_manager_common.py – shared cache and persistence logic.

• data/sources.py – source registry with @register_source decorator and get_source(variable_type, source_name) lookup.

• data/planner.py and data/plan.py – DataPlanner and immutable DataLoadPlan. The planner merges [data].types with rules that infer extra variables from foreign sections (for example geology if domain.zone_ids mentions geology).

• data/registry/catalog_duckdb.py – DataCatalogDuckDB persisting (variable, source, station_id, bbox, dates, file_path, mtime, sha256) for cache hits and external-mod detection.

• data/contracts/ – record types: PointRecord, FieldRecord, LoadResult, StationLocation.

• data/adapters/ – bridges to other layers (geology, station sets).

• data/common/ – shared helpers (timezone, units, geometry).

• data/schemas/ – Pydantic models reused across variables.

• data/variables/ – one folder per variable.

Variable inventory

Seventeen variables ship today, each in its own folder under data/variables/:

Variable	Sources
hydrometry	custom, Hub’Eau (hubeau).
piezometry	custom, Hub’Eau.
water_quality	custom, Hub’Eau.
intermittency	custom, Hub’Eau.
hydrography	custom, BD TOPAGE (bdtopage), EuHydro (euhydro), OpenStreetMap (osm).
geology	custom, BRGM 1:1M (brgm_1m), BRGM 1:50k (brgm_50k).
dem	custom, IGN Geoplateforme DEM (ign_geoplateforme_dem).
oceanic	custom, SHOM (shom), constant.
recharge	custom, SIM2 (sim2), synthetic.
runoff	custom, SIM2.
precipitation, temperature, etp, humidity, radiation, soil_moisture, wind	custom, SIM2 across the climate stack.

LoadResult contract

Every manager returns a LoadResult:

@dataclass
class LoadResult:
    points: list[PointRecord] = []
    fields: list[FieldRecord] = []
    warnings: list[str] = []

• PointRecord: station_id, variable, source, unit, frequency, data (datetime-indexed DataFrame), date_start / date_end, location (StationLocation), source_unit.

• FieldRecord: variable, source, field_path, crs, shape, metadata.

Key public symbols

• hydromodpy.data.base_manager_variable.BaseVariableManager

• hydromodpy.data.base_manager_field.BaseFieldManager

• hydromodpy.data.sources.{register_source, get_source, list_sources}

• hydromodpy.data.planner.DataPlanner

• hydromodpy.data.plan.DataLoadPlan

• hydromodpy.data.registry.catalog_duckdb.DataCatalogDuckDB

• hydromodpy.data.contracts.results.LoadResult

• hydromodpy.data.contracts.records.{PointRecord, FieldRecord}

Recommended reading path

1. hydromodpy/data/README.md

2. hydromodpy/data/base_manager_variable.py

3. hydromodpy/data/sources.py for the registration model.

4. hydromodpy/data/variables/hydrometry/ for a complete point variable.

5. hydromodpy/data/variables/dem/ for a complete field variable.

6. hydromodpy/data/planner.py for the inference rules.

7. hydromodpy/data/registry/catalog_duckdb.py for the cache schema.

Layer-matrix neighbours

• Allowed targets: core, schema, data, spatial.

• Documented tolerance: data -> results for the read-only cross-DB ATTACH bridge.

• Allowed sources: simulation, calibration, analysis, config, workflow, catalog, project and cli.

See also

• Add a Data Variable and Add a Data Source for contributor recipes.

• Data Loading And Retrieval for the user-facing data loading guide and provider matrix.