deepdow.experiments module

Running experiments.

class History

Bases: object

A shared information database for the training process.

database

Keys are different epochs and values are pd.DataFrame with detailed results.

Type

dict

add_entry(model=None, metric=None, batch=None, epoch=None, dataloader=None, lookback=None, timestamp=None, value=nan)

Add entry to the internal database.

property metrics

Concatenate metrics over all epochs.

Returns

df – Each row represents a unique logging sample. Columns are batch, current_time, dataloader, epoch, lookback, metric, model, timestamp, value.

Return type

pd.DataFrame

metrics_per_epoch(epoch)

Results over a specified epoch.

Returns

df – Each row represents a unique logging sample. Columns are batch, current_time, dataloader, epoch, lookback, metric, model, timestamp, value.

Return type

pd.DataFrame

pretty_print(epoch=None)

Print nicely the internal database.

Parameters

epoch (int or None) – If epoch given, then a results only over this epoch. If epoch is None then print results over all epochs.

class Run(network, loss, train_dataloader, val_dataloaders=None, metrics=None, benchmarks=None, device=None, dtype=None, optimizer=None, callbacks=None)

Bases: object

Represents one experiment.

Parameters

• network (nn.Module and Benchmark) – Network.

• loss (callable) –

  A callable computing per sample loss. Specifically it accepts weights, y and returns torch.Tensor

  of shape (n_samples,).

• train_dataloader (FlexibleDataLoader or RigidDataLoader) – DataLoader used for training.

• val_dataloaders (None or dict) – If None no validation is performed. If dict then keys are names and values are instances of RigidDataLoader.

• metrics (None or dict) – If None the only metric is the loss function. If dict then keys are names and values are instances of Loss.

• benchmarks (None or dict) – If None then no benchmark models used. If dict then keys are names and values are instances of Benchmark.

• device (torch.device or None) – Device on which to perform the deep network calculations. If None then torch.device(‘cpu’) used.

• dtype (torch.dtype or None) – Dtype to use for all torch tensors. If None then torch.double used.

• optimizer (None or torch.optim.Optimizer) – Optimizer to be used. If None then using Adam with lr=0.01.

• callbacks (list) – List of callbacks to be used.

metrics

Keys represent metric names and values are callables. Note that it always has an element called ‘loss’ representing the actual loss.

Type

dict

val_dataloaders

Keys represent dataloaders names and values are RigidDataLoader instances. Note that if empty then no logging is made.

Type

dict

models

Keys represent model names and values are either Benchmark or torch.nn.Module. Note that it always has an element called main representing the main network.

Type

dict

callbacks

Full list of callbacks. There are three defaults ones BenchmarkCallback, ValidationCallback and ProgressBarCallback. On top of them there are the manually selected callbacks from callbacks.

Type

list

property hparams

Collect relevant hyperparamters specifying an experiment.

launch(n_epochs=1)

Launch the training and logging loop.

Parameters

n_epochs (int) – Number of epochs.

on_batch_begin(metadata=None)

Take actions at the beginning of a batch.

on_batch_end(metadata=None)

Take actions at the end of a batch.

on_epoch_begin(metadata=None)

Take actions at the beginning of an epoch.

on_epoch_end(metadata=None)

Take actions at the end of an epoch.

on_train_begin(metadata=None)

Take actions at the beginning of the training.

on_train_end(metadata=None)

Take actions at the end of the training.

on_train_interrupt(metadata=None)

Take actions when training interrupted.