deepdow.experiments module

Running experiments.

class History[source]

Bases: object

A shared information database for the training process.

database

Keys are different epochs and values are pd.DataFrame with detailed results.

Type:

dict

add_entry(model=None, metric=None, batch=None, epoch=None, dataloader=None, lookback=None, timestamp=None, value=nan)[source]

Add entry to the internal database.

property metrics

Concatenate metrics over all epochs.

Returns:

df – Each row represents a unique logging sample. Columns are batch, current_time, dataloader, epoch, lookback, metric, model, timestamp, value.

Return type:

pd.DataFrame

metrics_per_epoch(epoch)[source]

Results over a specified epoch.

Returns:

df – Each row represents a unique logging sample. Columns are batch, current_time, dataloader, epoch, lookback, metric, model, timestamp, value.

Return type:

pd.DataFrame

pretty_print(epoch=None)[source]

Print nicely the internal database.

Parameters:

epoch (int or None) – If epoch given, then a results only over this epoch. If epoch is None then print results over all epochs.

class Run(network, loss, train_dataloader, val_dataloaders=None, metrics=None, benchmarks=None, device=None, dtype=None, optimizer=None, callbacks=None)[source]

Bases: object

Represents one experiment.

Parameters:
  • network (nn.Module and Benchmark) – Network.

  • loss (callable) –

    A callable computing per sample loss. Specifically it accepts weights, y and returns torch.Tensor

    of shape (n_samples,).

  • train_dataloader (FlexibleDataLoader or RigidDataLoader) – DataLoader used for training.

  • val_dataloaders (None or dict) – If None no validation is performed. If dict then keys are names and values are instances of RigidDataLoader.

  • metrics (None or dict) – If None the only metric is the loss function. If dict then keys are names and values are instances of Loss.

  • benchmarks (None or dict) – If None then no benchmark models used. If dict then keys are names and values are instances of Benchmark.

  • device (torch.device or None) – Device on which to perform the deep network calculations. If None then torch.device(‘cpu’) used.

  • dtype (torch.dtype or None) – Dtype to use for all torch tensors. If None then torch.double used.

  • optimizer (None or torch.optim.Optimizer) – Optimizer to be used. If None then using Adam with lr=0.01.

  • callbacks (list) – List of callbacks to be used.

metrics

Keys represent metric names and values are callables. Note that it always has an element called ‘loss’ representing the actual loss.

Type:

dict

val_dataloaders

Keys represent dataloaders names and values are RigidDataLoader instances. Note that if empty then no logging is made.

Type:

dict

models

Keys represent model names and values are either Benchmark or torch.nn.Module. Note that it always has an element called main representing the main network.

Type:

dict

callbacks

Full list of callbacks. There are three defaults ones BenchmarkCallback, ValidationCallback and ProgressBarCallback. On top of them there are the manually selected callbacks from callbacks.

Type:

list

property hparams

Collect relevant hyperparamters specifying an experiment.

launch(n_epochs=1)[source]

Launch the training and logging loop.

Parameters:

n_epochs (int) – Number of epochs.

on_batch_begin(metadata=None)[source]

Take actions at the beginning of a batch.

on_batch_end(metadata=None)[source]

Take actions at the end of a batch.

on_epoch_begin(metadata=None)[source]

Take actions at the beginning of an epoch.

on_epoch_end(metadata=None)[source]

Take actions at the end of an epoch.

on_train_begin(metadata=None)[source]

Take actions at the beginning of the training.

on_train_end(metadata=None)[source]

Take actions at the end of the training.

on_train_interrupt(metadata=None)[source]

Take actions when training interrupted.