numerical

numerical

Submodule for generic numerical computation.

This subpackage provides function to reduce activation tensor’s space dimensions, running statistics computation, functions to extract describtive numerical values from lists and runnning statistics, as well as utilities to parse string descriptions of these statistics.

Examples

>>> from monitorch.numerical import RunningMeanVar, extract_point
>>> extract_point([4, 5, 6], 'mean')
5.0
>>> rmv = RunningMeanVar()
>>> rmv.append(4)
>>> rmv.update(5)
>>> rmv.update(6)
>>> extract_point(rmv, 'mean')
5.0

class monitorch.numerical.GeometryComputation(inplace: bool, normalize: bool, correlation: bool, eps: float)

An object used for geometry calculation.

Keeps track of norm (RMS) and optionally correlation between consecutive tensors.

Let \(X_1, ..., X_n\) be sequence of tensors passed to update().

Keeps track of \(n_k = ||X||_2\) or \(n_k' = \frac{1}{\sqrt{\dim(X)}}||X||_2\) (if normalize=True)

Optionally \(r_k = \frac{X_{k-1} \cdot X_k}{n_{k-1}n_k + \epsilon}\)

Parameters:

• inplace (bool) – Flag indicating use of RunningMeanVar

• normalize (bool) – Flag indicating if norm of tensor should be computed as RMS

• correlation (bool) – Flag indicating computation of correlation between consecutive tensors, increases memory consumption by storing copy of previous tensor

• eps (float) – Constant used for numerical stability when computing correlation

update(X: Tensor)

Performs an update step on norm and optionally correlation

Parameters:

X (torch.Tensor) – Tensor to use for update

property value: RunningMeanVar | list[float] | tuple[RunningMeanVar | list[float], RunningMeanVar | list[float]]

Accumulated values (either list or RunningMeanVar)

Returns:

• tuple[norms, products] – if correlation=True

• norms – if correlation=False

class monitorch.numerical.RunningMeanVar(count: int = 0, mean: float = 0, var: float = 0, min_: float = inf, max_: float = -inf)

An object used to keep track of running statistics inplace.

Collects number of elements, mean, uncorected variance, mininimum and maximum of collection through update() or append() calls.

append(new_value: float) → None

Alias for update() method for compatability with list methods.

count: int = 0

Number of update calls on the object. Default is 0.

max_: float = -inf

Maximal value from update calls. Default is float(‘-inf’)

mean: float = 0

Mean calculated through all previous calls. Default is 0.

min_: float = inf

Minimal value from update calls. Default is float(‘+inf’)

update(new_value: float) → None

Updates running statistics with provided value.

Uses Welford’s algorithm to update variance and trivial procedure to update mean, minimum and maximum

Parameters:

new_value (float) – The value to update statistics with.

var: float = 0

Uncorrected variance (i.e. df = 0) calculated from update calls using Welford’s algorithm. Default is 0.

class monitorch.numerical.RunningValue(count: int = 0, value: Any = None)

count: int = 0

value: Any = None

monitorch.numerical.extract_point(raw_val: RunningMeanVar | list[float], method: str) → float

Extracts a single variable from RunningMeanVar or list.

Generic function to work with RunningMeanVar and lists of floats.

Parameters:

• raw_val (RunningMeanVar | list[float]) – Object from which the value must be extracted.

• method (str = {'mean', 'median', 'max', 'min', 'Q1', 'Q2', 'std', 'IQR'}) – Description of value to extract.

Returns:

Extracted value specified by method.

Return type:

float

Raises:

AttributeError – If unknown method was passed. If method is median, Q1, Q2 or IQR and raw_val is RunningMeanVar

monitorch.numerical.extract_range(raw_val: RunningMeanVar | list[float], method) → tuple[float, float]

Extracts a range described by method from provided object.

Generic function to extract ranges (pairs of values) from RunningMeanVar or list.

Parameters:

• raw_val (RunningMeanVar | list[float]) – Object from which the range must be extracted.

• method (str = {'std', 'Q1-Q3', 'min-max'}) – Description of range to extract.

Returns:

Extracted range specified by method.

Return type:

tuple(float, float)

Raises:

AttributeError – If unknown method was passed. If method is Q1-Q3 and raw_val is RunningMeanVar

monitorch.numerical.parse_range_name(name) → tuple[str, str]

Parses string name into matplotlib annotatable pair of strings.

Translates:

'std'     to ('-σ', '+σ')
'Q1-Q3'   to ('Q1', 'Q3')
'min-max' to ('min', 'max')

Parameters:

name (str) – Range name

Returns:

Edge names of range

Return type:

tuple(str, str)

Raises:

AttributeError – If the range name is unknown

monitorch.numerical.reduce_activation_to_activation_rates(act_tensor: Tensor, batch: bool) → Tensor

Reduces (boolean) activation tensor to activation rates per channel/neuron.

Flattens spatial dimensions. For batch input computes mean over batch and spatial dimension, for non-batch input computes mean only over spatial dimensions.

Parameters:

• act_tensor (torch.Tensor) – Tensor to compute activation rates from. Can have any dtype, as it is converted to float in the function if mean is computed.

• batch (bool) – Flag indicating whether act_tensor is a batch of data.

Returns:

If input tensor is one dimensional returns itself, otherwise computes mean accross spatial and batch dimensions, if the last is present. In case of multidimensional input tensor, output’s dtype is float.

Return type:

torch.Tensor