hydromodpy.workflow.parallel#

Sweep orchestration and worker-pool helpers.

Hosts three reusable building blocks:

  • expand_parameters() / run_sweep(): the original sweep API consumed by project.sweep.

  • SequentialExecutor and ThreadPoolCohortExecutor: cohort executors used by hydromodpy.workflow.runner.Pipeline when a Kahn DAG sort returns multi-step cohorts.

  • execute_cohorts(): thin helper that walks a list of cohorts and dispatches each to the executor of choice.

Threads are used over processes because every workflow step mutates a live WorkflowContext whose components (zarr stores, DuckDB connections) are not pickle-safe.

Functions

execute_cohorts(cohorts, worker, *[, executor])

Walk Kahn cohorts and dispatch every item through executor.

expand_parameters(parameters, strategy)

Return the list of {param: value} dicts that feed one run each.

run_sweep(project, *, parameters, strategy, ...)

Execute one run per parameter point.

Classes

CohortExecutor(*args, **kwargs)

Backend that runs every item of one Kahn cohort.

SequentialExecutor()

In-process sequential executor for cohort items.

SweepProject(*args, **kwargs)

Minimal project protocol needed by sweep orchestration.

SweepRun(*args, **kwargs)

Minimal run view needed by sweep orchestration.

ThreadPoolCohortExecutor([max_workers])

Cohort executor backed by a ThreadPoolExecutor.