inspector

class monitorch.inspector.PyTorchInspector(lenses: list[AbstractLens], *, visualizer: str | AbstractVisualizer = 'print', module: None | Module = None, depth: int = -1, module_name_prefix: str = '.', train_loss_str='train_loss', non_train_loss_str='val_loss', is_active_fn: int | Callable[[int], bool] = 1)

Bases: object

One class to rule them all.

PyTorchInspector is a class that manages interactions between lenses, visualizers and user defined module.

To use inspector one needs to initialize the inspector and provide it the module to monitor. During initialization lenses from monitorch.lens must be provided. The only thing that is required during training is to call tick_epoch() on the end of each epoch. Optionally one could push additional metrics using push_metric() and push_loss().

If visualizer is 'matplotlib', then 'show_fig()' must be called on visualizer, otherwise the plot will be drawn during training.

Parameters:

• lenses (list[AbstractLens]) – List of objects from monitorch.lens, used to collect and plot data.

• visualizer (str|AbstractVisualizer = 'print') – Visualizer to draw plots, must be either a visualizer object from monitorch.visualizer or a string 'matplotlib', 'tensorboard' or 'print'.

• module (None|torch.nn.Module = None) – Optional neural network to examine, can be added later using attach().

• depth (int = -1) – Depth to unfold neural net injection tree. For example depth=0 returns the model itself, depth=1 returns modules directly contained in module object. Default is depth=-1, that is to unfold until leaf modules are reached.

• module_name_prefix (str = '.') – Delimiter to separate names of parent and child modules.

• 'train_loss' (train_loss_str =) – String to be used for training loss.

• 'val_loss (non_train_loss_str =) – String to be used for validation/testing/development loss.

• is_active_fn (int | Callable[[int], bool] = 1) – Function deciding if inspector is active (collects and visualizes data) for given epoch. Passed directly to InspectorState. Integer values correspond to function epoch % n == 0, where n is passed value.

lenses

List of objects from monitorch.lens, used to collect and plot data. Exatcly the same object as the one provided during initialization.

Type:

list[AbstractLens]

visualizer

Visualizaer object that draws all plots. Can be hot-swapped.

Type:

AbstractVisualizer

state

State object representing inspectors inner state. Weak-referenced by gatherers.

Type:

InspectorState

depth

Depth to unfold module inclusion tree.

Type:

int

module_name_prefix

Delimiter to separate names of parent and child modules.

Type:

str

Examples

Basic usage with 'LossMetrics', 'OutputActivation' and 'ParameterGradientGeometry' may look something like this.

>>> from monitorch.inspector import PyTorchInspector
>>> from monitorch.lens import LossMetrics, OutputActivation, ParameterGradientGeometry
>>>
>>> loss_fn = nn.NLLLoss()
>>>
>>> inspector = PyTorchInspector(
...     lenses = [
...         LossMetrics(loss_fn = loss_fn),
...         OutputActivation(),
...         ParameterGradientGeometry()
...     ],
...     module = mynet,
...     visualizer='matplotlib'
... )
>>>
>>> for epoch in range(N_EPOCHS):
...     for data, label in train_dataloader:
...         optimizer.zero_grad()
...         prediction = mynet(data)
...         loss = loss_fn(prediction, label)
...         loss.backward()
...         optimizer.step()
...
...     with torch.no_grad(): # outputs inside this block are not recorded
...         for data, label in val_dataloader:
...             prediction = mynet(data)
...             loss = loss_fn(prediction, label)
...
...     inspector.tick_epoch() # ticking the epoch
>>>
>>> inspector.visualizer.show_fig()

attach(module: Module) → Self

Attaches inspector to a module.

Unfolds inclusion module tree guided by depth set during initialization. Registers submodules onto every lens.

Parameters:

module (torch.nn.Module) – Neural net to attach to.

Returns:

Builder pattern.

Return type:

Self

detach() → Self

Detaches all lenses from modules.

Returns:

Builder pattern.

Return type:

Self

iter(iterable: Iterable) → Iterable

push_loss(value: float, *, train: bool, running: bool = True)

Pushes loss, that can be accessed by monitorch.lens.LossMetrics.

Parameters:

• value (float) – Loss value.

• train (bool) – Flag indicating if it is training loss.

• running (bool = True) – Flag indicating if metric should be saved in-place (True) or in-memory (False).

push_metric(name: str, value: float, *, running: bool = True)

Pushes metric, that can be accessed by monitorch.lens.LossMetrics.

Parameters:

• name (str) – Name of the metric to save.

• value (float) – Metric’s value.

• running (bool = True) – Flag indicating if metric should be saved in-place (True) or in-memory (False).

range(*args, **kwargs) → Iterable

tick(epoch: int | None = None)

Ticks to postprocess data and draw plots.

Parameters:

epoch (int|None = None) – Optional epoch counter, default ticks state, thus incrementing counter.

tick_epoch(epoch: int | None = None)

Ticks to postprocess data and draw plots.

Parameters:

epoch (int|None = None) – Optional epoch counter, default ticks state, thus incrementing counter.

class monitorch.inspector.inspector_state.InspectorState(is_active_fn: int | Callable[[int], bool] = 1)

Bases: object

Class incapsulating public inspector state

Parameters:

is_active_fn (int|Callable[[int], bool]) – Function to determine if inspector is active from number of ticks. If int is provided is equivalent to lambda n_ticks: n_ticks % is_active_fn == 0

counter

Tick counter. Starts from 1.

Type:

int

attached

Flag indicating if inspector is attached to module

Type:

bool

property is_active: bool

Activation state according to is_active_fn

Return type:

Value of is_active_fn(counter)

tick(n_ticks: int = 1) → int

Increments counter by n_ticks

Parameters:

n_ticks (int) – Number of ticks to increment by

Return type:

The value of counter