gwsource.ensemble#

Ensemble driver: trace one time step of many model realisations that share one grid.

Each realisation is a simulation folder (e.g. one per parameter realisation of a PEST++ IES
ensemble, run by a sweep). For every realisation the driver reads one time step, re-selects
the groups (a dict of select() arguments, or a top-level function step -> groups), runs the
backward or forward trace (and, with porosity, the Eulerian travel times of
gwsource.traveltime), and folds the requested per-cell outputs into running statistics, so only
one realisation's outputs are in memory at a time (plus the quantile store, below).

Every realisation must have the same connectivity: the IA/JA arrays of the binary grid file are
hashed and compared with the first realisation's (the flows, terms and saturation may differ).

Statistics per output, group and cell (NaN values, e.g. travel times where the fraction is 0,
are left out cell by cell; count is the number of realisations where the value is defined):
- min, max, mean;
- exceedance[t]: probability that the value exceeds t, i.e. count(value > t) / count; for
  fraction_through this is the capture probability P(F > t);
- quantiles[q]: numpy's default ("linear") quantile over the defined values.
  Exact: if the float32 stack of all values (realisations x cells x outputs x groups x 4 bytes)
  fits memory_budget, the stack is kept and the quantiles are exact (up to float32 rounding).
  Approximate: otherwise each output keeps a fixed-bin histogram per cell over
  histogram_range[output] (default (0, 1), right for fractions; set it for times or volumes).
  An order statistic is placed uniformly inside its bin and clamped to the cell's [min, max];
  the error of a quantile is at most one bin width, (high - low) / histogram_bins, for values
  inside the range (values outside are counted in the end bins). Memory: histogram_bins x
  cells x 4 bytes per output and group.

Parallel runs (max_workers > 1) trace realisations in a ProcessPoolExecutor; results come back
in realisation order and are folded in that order, so the statistics are identical to a serial
run. Each worker returns plain arrays (the trace results hold an unpicklable factorisation).
class gwsource.ensemble.EnsembleConfig(mode='backward', outputs=None, kstpkper=(0, 0), thresholds=(), quantiles=(0.05, 0.5, 0.95), memory_budget=2000000000.0, histogram_bins=200, histogram_range=None, porosity=None, fraction_threshold=1e-10, max_workers=1, model=None, boundnames=False)[source]#

Bases: object

Options of an ensemble run.

mode: "backward" or "forward".
outputs: per-cell outputs to summarise (default fraction_through / source_fraction); any
    per-cell output of the trace result, or a travel-time output if porosity is set.
kstpkper: the 0-based time step to read from every realisation.
thresholds: exceedance probabilities P(value > t) are computed for each t.
quantiles: probabilities in [0, 1].
memory_budget: bytes allowed for the exact-quantile float32 stack.
histogram_bins, histogram_range: the approximate-quantile histogram ({output: (low, high)};
    outputs not listed use (0, 1)).
porosity: scalar or per-cell; enables travel-time outputs.
fraction_threshold: travel-time outputs are NaN where the fraction (F or G) is at or
    below this (gwsource.traveltime.travel_time).
max_workers: 1 (default) runs serially; > 1 uses that many processes.
model: GWF model name if a simulation has several.
boundnames: load each realisation's boundnames from the package input (FloPy), so groups
    can be selected by boundname; off by default because it reads the model input files.
Parameters:
resolved_outputs()[source]#
Return the outputs, with the mode default if none were given.
Return type:

list[str]

bin_range(output)[source]#
Return the histogram range of an output.
Parameters:

output (str)

Return type:

tuple[float, float]

class gwsource.ensemble.Realisation(ws, grid_hash, values, group_inflow, mean_transit_time)[source]#

Bases: object

Outputs of one realisation: values[(output, group)] per cell, group scalars, grid hash.
Parameters:
class gwsource.ensemble.Accumulator(minimum, maximum, total, count, exceed, stack=None, hist=None, bin_range=(0.0, 1.0))[source]#

Bases: object

Running statistics of one output and group over realisations (arrays per cell).
Parameters:
class gwsource.ensemble.EnsembleResult(config, realisations, groups, outputs, stats, exceedance, quantiles, quantile_method, group_inflow, mean_transit_time=<factory>, bin_width=<factory>)[source]#

Bases: object

Ensemble statistics: stats[name][output][group], exceedance[t][...], quantiles[q][...].

stats names: min, max, mean, count. quantile_method is "exact" or "histogram" (bin_width
is the histogram's bin width per output, the bound on the quantile error). group_inflow and
mean_transit_time hold one value per realisation; group_inflow is the destination (backward)
or source (forward) inflow, which in net accounting counts each passage once
(gwsource.metrics).
Parameters:
gwsource.ensemble.list_realisations(root, pattern='*')[source]#
Return the sorted folders under root matching pattern that hold an mfsim.nam.
Parameters:
Return type:

list[Path]

gwsource.ensemble.grid_hash(ia, ja)[source]#
Return a hash of the connectivity arrays.
Parameters:
Return type:

str

gwsource.ensemble.trace_realisation(ws, groups, config, trace_config)[source]#
Read, trace and reduce one realisation to the requested per-cell outputs.
Parameters:
Return type:

Realisation

gwsource.ensemble.iterate_realisations(workspaces, groups, config, trace_config)[source]#
Yield the Realisation of each workspace in order, serially or with a process pool.
Parameters:
gwsource.ensemble.new_accumulator(n_cells, n_real, n_thresholds, exact, bins, bin_range)[source]#
Return empty accumulators for one output and group.
Parameters:
Return type:

Accumulator

gwsource.ensemble.histogram_bin(values, bin_range, bins)[source]#
Return the histogram bin of each value (values outside the range go to the end bins).
Parameters:
Return type:

ndarray

gwsource.ensemble.fold(acc, values, index, thresholds)[source]#
Fold one realisation's per-cell values (NaN = undefined) into the accumulators.
Parameters:
Return type:

None

gwsource.ensemble.exact_quantiles(stack, probs, chunk=100000)[source]#
Return (len(probs), cells) quantiles over the defined values of a (real, cells) stack.
Parameters:
Return type:

ndarray

gwsource.ensemble.histogram_quantiles(acc, probs, chunk=50000)[source]#
Return (len(probs), cells) approximate quantiles (numpy 'linear' rule) from histograms.
Parameters:
Return type:

ndarray

gwsource.ensemble.finish(acc, thresholds, probs)[source]#
Return the statistics of one output and group from its accumulators.
Parameters:
Return type:

dict

gwsource.ensemble.stack_bytes(n_real, n_cells, n_series)[source]#
Return the size of the exact-quantile float32 stack in bytes.
Parameters:
Return type:

int

gwsource.ensemble.run_ensemble(workspaces, groups, config=None, trace_config=None)[source]#
Trace every realisation and return streaming statistics over the ensemble.
Parameters:
Return type:

EnsembleResult