gatherer

Submodule containing stateful callbacks to hook on modules.

This module implements classes for automatic data collection from PyTorch modules. Gatherers should be used whenever there is need to keep track of data from several modules, that can be differentiated by names. Data aggregation and preprocessing must be done by monitorch.preprocessor. The submodule provides classes to gather outputs, gradients on each pass and to pass module object to processor on call.

class monitorch.gatherer.AbstractGatherer(inspector_state)

Bases: ABC

An abstract class that parents all gatherers.

abstract detach() → None

Abstract method to detach from module.

Detaches gatherer and all its acompaning preprocessors from module.

static requires_active_inspector_state(fn)

class monitorch.gatherer.BackwardGatherer(module, preprocessors: list[AbstractBackwardPreprocessor], name: str, inspector_state)

Bases: AbstractGatherer

Object responsible for collecting data from torch.nn.Module.register_full_backward_hook.

Registers self to module provided in construction as a backward hook, on call hands over data and module’s name to preprocessors.

Parameters:

• module (torch.nn.Module) – Module to hook onto.

• preprocessors (list[AbstractBackwardPreprocessor]) – List of preprocessors to hand over data when PyTorch calls the hook.

• name (str) – Name of module to hand over to preprocessors.

detach() → None

See base class.

class monitorch.gatherer.CallParameterGatherer(module, parameter: str, preprocessors: list[AbstractTensorPreprocessor], name: str, inspector_state)

Bases: AbstractGatherer

Gatherer to hand over whole module object on call.

Keeps a reference of module to pass it on call to preprocessors with name attached.

Parameters:

• module (torch.nn.Module) – Module to be handed over to preprocessors.

• preprocessors (list[AbstractModulePreprocessor]) – Preprocessors to hand the module over to.

• name (str) – Name of the module.

detach() → None

See base class

class monitorch.gatherer.EpochModuleGatherer(module, preprocessors: list[AbstractModulePreprocessor], name: str, inspector_state)

Bases: AbstractGatherer

Gatherer to hand over whole module object on call.

Keeps a reference of module to pass it on call to preprocessors with name attached.

Parameters:

• module (torch.nn.Module) – Module to be handed over to preprocessors.

• preprocessors (list[AbstractModulePreprocessor]) – Preprocessors to hand the module over to.

• name (str) – Name of the module.

detach() → None

See base class

class monitorch.gatherer.FeedForwardGatherer(module, preprocessors: list[AbstractForwardPreprocessor], name: str, inspector_state)

Bases: AbstractGatherer

Object responsible for collecting data from torch.nn.Module.register_forward_hook.

Registers self to module provided in construction as a forward hook, on call hands over data and module’s name to preprocessors.

Parameters:

• module (torch.nn.Module) – Module to hook onto.

• preprocessors (list[AbstractForwardPreprocessor]) – List of preprocessors to hand over data when PyTorch calls the hook.

• name (str) – Name of module to hand over to preprocessors.

detach() → None

See base class

class monitorch.gatherer.OptimizerStepParameterGatherer(module, optimizer: Optimizer, parameter: str, preprocessors: list[AbstractTensorPreprocessor], name: str, inspector_state)

Bases: AbstractGatherer

Gatherer to hand over whole module object on optimizer step.

Keeps a reference of module and optimizer to pass it on call to preprocessors with name attached.

Calling detach() does not remove reference to optimizer object.

Parameters:

• module (torch.nn.Module) – Module to be handed over to preprocessors.

• preprocessors (list[AbstractModulePreprocessor]) – Preprocessors to hand the module over to.

• name (str) – Name of the module.

detach() → None

See base class

class monitorch.gatherer.ParameterGradientGatherer(parameter: str, module, preprocessors: list[AbstractTensorPreprocessor], name: str, inspector_state)

Bases: AbstractGatherer

Class to collect gradients from attributes of module.

Object of ParameterGradientGatherer gatherer is a stateful callback registered onto torch.Tensor using register_post_accumulate_grad_hook. On call hands over data to preprocessors.

Parameters:

• parameter (str) – Name of learnable parameter in module to gather data from.

• module (torch.nn.Module) – Module from which the learnable parameter is obtained. The data will be collected from that learnable parameter.

• preprocessors (list[AbstractTensorPreprocessor]) – Preprocessors that will aggregate data.

• name (str) – Name of the module.

detach() → None

See base class