LossMetrics

class monitorch.lens.LossMetrics(*, metrics: Iterable[str] | None = None, evaluation_from_grad: bool = True, separate_loss_and_metrics: bool = True, loss_fn: Module | None = None, loss_fn_inplace: bool = True, loss_line: str | Iterable[str] = 'mean', loss_range: str | Iterable[str] | None = 'std', metrics_line: str | Iterable[str] = 'mean', metrics_range: str | Iterable[str] | None = None)

Bases: AbstractLens

Lens to track loss and metrics.

Tracks losses through push_loss on inspector or loss module object (for example torch.nn.MSELoss or torch.nn.NLLLoss). To keep track of metrics pass metric’s name during initizaliozation and use push_metric on inspector to save data.

Strings associated with loss names are taken from inspector.

Parameters:

• metrics (Iterable[str]|None = None) – Metrics’ names to plot.

• evaluation_from_grad (bool = True) – Flag indicating if evaluation (validation) passes must be decided from torch.is_grad_enabed() or loss_fn.training

• separate_loss_and_metrics (bool = True,) – Flag indicating if loss and metric plots should be separate.

• loss_fn (Module|None = None) – Loss function module, if provided loss values will be extract during forward pass through loss_fn object.

• loss_fn_inplace (bool = True) – Flag indicating if data from loss_fn should be aggregated inplace.

• loss_line (str|Iterable[str] = 'mean') – Aggregation methods for loss lines.

• loss_range (str|Iterable[str]|None = 'std') – Aggregation methods for loss bands.

• metrics_line (str|Iterable[str] = 'mean') – Aggregation methods for metrics’ lines.

• metrics_range (str|Iterable[str]|None = None) – Aggregation methods for metrics’ bands.

Examples

An example of training where loss is explicit pushed into inspector.

>>> inspector = PyTorchInspector(
...     lenses = [
...         LossMetrics(),
...     ],
...     module = mynet,
...     visualizer='matplotlib'
... )
>>>
>>> for epoch in range(N_EPOCHS):
...     for data, label in train_dataloader:
...         optimizer.zero_grad()
...         prediction = mynet(data)
...         loss = F.binary_cross_entropy(prediction, label)
...         loss.backward()
...         optimizer.step()
...
...         inspector.push_loss(loss.item(), train=True)
...     inspector.tick_epoch()
>>>
>>> inspector.visualizer.show_fig()

The same effect can be obtained by providind torch.nn loss object.

>>> loss_fn = nn.BCELoss()
>>> inspector = PyTorchInspector(
...     lenses = [
...         LossMetrics(loss_fn=loss_fn),
...     ],
...     module = mynet,
...     visualizer='matplotlib'
... )
>>>
>>> for epoch in range(N_EPOCHS):
...     for data, label in train_dataloader:
...         optimizer.zero_grad()
...         prediction = mynet(data)
...         loss = loss_fn(prediction, label)
...         loss.backward()
...         optimizer.step()
...     inspector.tick_epoch()
>>>
>>> inspector.visualizer.show_fig()

Metrics must be passed explicitly.

>>> loss_fn = nn.BCELoss()
>>> inspector = PyTorchInspector(
...     lenses = [
...         LossMetrics(
...             metrics=['accuracy'],
...             loss_fn=loss_fn
...         ),
...     ],
...     module = mynet,
...     visualizer='matplotlib'
... )
>>>
>>> for epoch in range(N_EPOCHS):
...     for data, label in train_dataloader:
...         optimizer.zero_grad()
...         prediction = mynet(data)
...         loss = loss_fn(prediction, label)
...         loss.backward()
...         optimizer.step()
...
...         accuracy_score = ((prediction > 0.5).float() == label).mean().item()
...         inspector.push_metric('accuracy', accuracy_score)
...     inspector.tick_epoch()
>>>
>>> inspector.visualizer.show_fig()

detach_from_module()

Does not interact with estimator network. Does not detach from loss function module.

finalize_epoch()

Finalizes loss and metrics computation.

introduce_tags(vizualizer: AbstractVisualizer)

Introduces lens’s plots to visualizer.

Registers one big tag if separate_loss_and_metrics is False named 'Loss & Metrics'. If separate_loss_and_metrics is True, registers two big tags: 'Loss' and 'Metrics'.

Parameters:

visualzier (AbstractVisualizer) – A visualizer object to pass tag attributes to.

loss(*, train: bool, method: str | None = None) → float

Get loss from last finalization (epoch tick).

Parameters:

• train (bool) – Flag indicating if train loss must be returned

• method (str|None = None) – Optional method of loss aggregation, default is the first provided during initialization.

Returns:

loss value

Return type:

float

Raises:

AttributeError – If lens cannot get loss strings, most probably the lens was not registered with ExplicitCall preprocessor.

register_foreign_preprocessor(ext_ppr: AbstractPreprocessor, inspector_state)

Saves an instance of monitorch.preprocessor.ExplicitCall, other predprocessors are ignored.

Parameters:

ext_ppr (AbstractPreprocessor) – External preprocessor to register (or ignore).

register_leaf_module(module: Module, module_name: str, inspector_state)

Does not interact with estimator network.

reset_epoch()

Resets inner state.

Resets data computed during last epoch and resets preprocessors.

vizualize(vizualizer: AbstractVisualizer, epoch: int)

Visualizes loss and metrics.

Passes loss Ordered dictionaries that may look like this

OrderedDict([
    ('Loss', {'train_loss mean' : 0.89, 'val_loss mean' : 0.98})
])

OrderedDict([
    ('Loss', {
        ('train_loss Q1', 'train_loss Q3') : (0.79, 0.99),
        ('val_loss Q1',   'val_loss Q3')   : (0.90, 1.06),
    })
])

Metrics dictionaries are similar, though may use other top level index (i.e. ‘Metrics’ instead of ‘Loss’).

Parameters:

• visualizer (AbstractVisualizer) – The visualizer object responsbile for drawing plots.

• epoch (int) – Computation’s epoch number.