Calibration Benchmarks: No Uncertainty, More Data#
Note
This page and its static assets are auto-generated by python -m tools.doc_gallery. The Sphinx build only reads committed PNG and JSON artifacts.
These calibration benchmarks keep observation noise off and rely on richer data sets, typically multiple observable blocks, so the objective surface is better constrained.
See also
Open the calibration benchmark category for the case-by-case pages and their configuration figures.
Coverage#
Cases compared: 2
Method rows: 11
Timing now separates candidate runtime from calibration-method overhead, and further splits actualize, launcher preparation, runtime patch, simulation, output selection, and objective scoring.
Summary Figures#
Linked Cases#
Calibration Twin: Recharge-Step K+Sy 1D
Transient modflow6 twin calibration benchmark with K_global, Sy_global.
Calibration Twin: Piecewise-K 1D
Steady modflow6 twin calibration benchmark with K_west, K_middle, K_east.
Method Rows#
Case |
Method |
Metric |
Target |
Cost |
Eval |
Calibration (s) |
Candidate runtime (s) |
Algorithm overhead (s) |
Actualize (s) |
Launcher prep (s) |
Patch (s) |
Sim (s) |
Output select (s) |
Objective score (s) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
random_search |
best_fit |
0 |
0.529874 |
16 |
24.95 s |
24.53 s |
0.419 s |
0 s |
0 s |
0 s |
1.533 s |
0 s |
0 s |
|
optuna |
best_fit_or_distribution |
1 |
0.206196 |
40 |
68.87 s |
67.64 s |
1.228 s |
0 s |
0 s |
0 s |
1.691 s |
0 s |
0 s |
|
cma_es |
best_fit |
1 |
0.214878 |
40 |
107.8 s |
105.7 s |
2.05 s |
0 s |
0 s |
0 s |
2.644 s |
0 s |
0 s |
|
scipy_nelder_mead |
best_fit |
1 |
0.219763 |
12 |
35.57 s |
35.03 s |
0.5338 s |
0 s |
0 s |
0 s |
2.92 s |
0 s |
0 s |
|
gp_mapping |
best_fit_or_distribution |
0 |
0.569137 |
16 |
32.19 s |
24.77 s |
7.423 s |
0 s |
0 s |
0 s |
1.548 s |
0 s |
0 s |
|
da_mh_gp |
best_fit_or_distribution |
0 |
2.22364 |
10 |
15.55 s |
15.22 s |
0.3297 s |
0 s |
0 s |
0 s |
1.522 s |
0 s |
0 s |
|
random_search_seed017 |
distribution |
1 |
0.0451974 |
48 |
16.22 s |
15.06 s |
1.155 s |
0 s |
0 s |
0 s |
0.3138 s |
0 s |
0 s |
|
random_search_seed029 |
distribution |
0 |
0.0289115 |
48 |
15.98 s |
15.07 s |
0.9074 s |
0 s |
0 s |
0 s |
0.3139 s |
0 s |
0 s |
|
optuna |
best_fit_or_distribution |
1 |
0.0211372 |
48 |
15.97 s |
14.56 s |
1.41 s |
0 s |
0 s |
0 s |
0.3034 s |
0 s |
0 s |
|
cma_es |
best_fit |
0 |
0.0271745 |
48 |
19.68 s |
18.18 s |
1.502 s |
0 s |
0 s |
0 s |
0.3787 s |
0 s |
0 s |
|
scipy_nelder_mead |
best_fit |
1 |
0.00225239 |
48 |
22.09 s |
20.59 s |
1.5 s |
0 s |
0 s |
0 s |
0.4289 s |
0 s |
0 s |
Artifacts#
docs/source/_static/capability_gallery/calibration/intercomparison/calibration_data_rich_no_uncertainty/calibration_intercomparison_summary.jsondocs/source/_static/capability_gallery/calibration/intercomparison/calibration_data_rich_no_uncertainty/benchmark_target_success_rates.pngdocs/source/_static/capability_gallery/calibration/intercomparison/calibration_data_rich_no_uncertainty/benchmark_cost_vs_budget.pngdocs/source/_static/capability_gallery/calibration/intercomparison/calibration_data_rich_no_uncertainty/benchmark_time_vs_cost.pngdocs/source/_static/capability_gallery/calibration/intercomparison/calibration_data_rich_no_uncertainty/benchmark_calibration_time_closure.pngdocs/source/_static/capability_gallery/calibration/intercomparison/calibration_data_rich_no_uncertainty/benchmark_candidate_timing_breakdown.png