Calibration Benchmarks: Uncertainty And Less Data#

Note

This page and its static assets are auto-generated by python -m tools.doc_gallery. The Sphinx build only reads committed PNG and JSON artifacts.

These calibration benchmarks deliberately reduce information content by using fewer observations and adding uncertainty, so the methods are tested on weaker inverse constraints.

See also

Open the calibration benchmark category for the case-by-case pages and their configuration figures.

Coverage#

  • Cases compared: 1

  • Method rows: 6

  • Timing now separates candidate runtime from calibration-method overhead, and further splits actualize, launcher preparation, runtime patch, simulation, output selection, and objective scoring.

Summary Figures#

Calibration intercomparison figure benchmark_target_success_rates
Calibration intercomparison figure `benchmark_target_success_rates` derived from the curated capability-gallery cases.
Calibration intercomparison figure benchmark_cost_vs_budget
Calibration intercomparison figure `benchmark_cost_vs_budget` derived from the curated capability-gallery cases.
Calibration intercomparison figure benchmark_time_vs_cost
Calibration intercomparison figure `benchmark_time_vs_cost` derived from the curated capability-gallery cases.
Calibration intercomparison figure benchmark_calibration_time_closure
Calibration intercomparison figure `benchmark_calibration_time_closure` derived from the curated capability-gallery cases.
Calibration intercomparison figure benchmark_candidate_timing_breakdown
Calibration intercomparison figure `benchmark_candidate_timing_breakdown` derived from the curated capability-gallery cases.

Linked Cases#

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

Transient modflow6 twin calibration benchmark with K_global, Sy_global.

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

Method Rows#

Case

Method

Metric

Target

Cost

Eval

Calibration (s)

Candidate runtime (s)

Algorithm overhead (s)

Actualize (s)

Launcher prep (s)

Patch (s)

Sim (s)

Output select (s)

Objective score (s)

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

random_search

best_fit_or_distribution

1

0.410218

24

31.46 s

31.1 s

0.3578 s

0 s

0 s

0 s

1.296 s

0 s

0 s

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

optuna

best_fit_or_distribution

1

0.407764

48

105.2 s

103.3 s

1.859 s

0 s

0 s

0 s

2.152 s

0 s

0 s

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

cma_es

best_fit

0

0.407793

56

125.9 s

124.3 s

1.554 s

0 s

0 s

0 s

2.221 s

0 s

0 s

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

scipy_nelder_mead

best_fit

0

0.407768

16

26.37 s

26.04 s

0.331 s

0 s

0 s

0 s

1.627 s

0 s

0 s

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

gp_mapping

best_fit_or_distribution

1

0.415671

20

56.85 s

37.88 s

18.96 s

0 s

0 s

0 s

1.894 s

0 s

0 s

Calibration Twin: Recharge-Step Flux-Only K+Sy 1D

da_mh_gp

best_fit_or_distribution

1

0.649032

12

23.3 s

22.74 s

0.5585 s

0 s

0 s

0 s

1.895 s

0 s

0 s

Artifacts#

  • docs/source/_static/capability_gallery/calibration/intercomparison/calibration_uncertain_less_data/calibration_intercomparison_summary.json

  • docs/source/_static/capability_gallery/calibration/intercomparison/calibration_uncertain_less_data/benchmark_target_success_rates.png

  • docs/source/_static/capability_gallery/calibration/intercomparison/calibration_uncertain_less_data/benchmark_cost_vs_budget.png

  • docs/source/_static/capability_gallery/calibration/intercomparison/calibration_uncertain_less_data/benchmark_time_vs_cost.png

  • docs/source/_static/capability_gallery/calibration/intercomparison/calibration_uncertain_less_data/benchmark_calibration_time_closure.png

  • docs/source/_static/capability_gallery/calibration/intercomparison/calibration_uncertain_less_data/benchmark_candidate_timing_breakdown.png

On this page