osl_dynamics.models.mod_base

Base class for models.

Classes

BaseModelConfig	Base class for settings for all models.
ModelBase	Base class for all models.

Functions

unscale_loss_for_distribution(value)

Module Contents

osl_dynamics.models.mod_base.unscale_loss_for_distribution(value)

class osl_dynamics.models.mod_base.BaseModelConfig

Base class for settings for all models.

model_name: str = None

multiple_dynamics: bool = False

init_method: str = None

n_init: int = None

n_init_epochs: int = None

init_take: float = 1

batch_size: int = None

learning_rate: float = None

lr_decay: float = 0.1

gradient_clip: float = None

n_epochs: int = None

optimizer: tensorflow.keras.optimizers.Optimizer = 'adam'

loss_calc: str = 'mean'

multi_gpu: bool = False

strategy: str = None

best_of: int = 1

n_modes: int = None

n_states: int = None

n_channels: int = None

sequence_length: int = None

validate_training_parameters()

Return type:

None

validate_dimension_parameters()

Return type:

None

class osl_dynamics.models.mod_base.ModelBase(config)

Base class for all models.

Acts as a wrapper for a standard Keras model.

Parameters:

config (BaseModelConfig)

osld_version = None

config_type = None

config

model = None

abstractmethod build_model()

Build a keras model.

Return type:

None

compile(optimizer=None, **kwargs)

Compile the model.

Parameters:

optimizer (str or tf.keras.optimizers.Optimizer) – Optimizer to use when compiling.

Return type:

None

add_metrics_for_loss()

Add a metric for each model output loss.

Return type:

None

skip_non_finite_steps()

Skip any training step whose loss or gradients are not finite.

Without this, a NaN or inf in a single batch (e.g. from an unstable KL term) would be applied to the weights and end the training. The step is skipped instead: the weights and optimizer are left as they were and the batch is left out of the losses reported for the epoch. fit logs how many steps were skipped.

Return type:

None

initialization(*args, method=None, **kwargs)

Wrapper for an initialization method.

Parameters:

• *args (arguments) – Arguments to pass to the initialization method.

• method (str) – Initialization method name.

• **kwargs (keyword arguments) – Keyword arguments to pass to the initialization method.

Returns:

history – Training history for the initialization.

Return type:

dict

fit(*args, use_tqdm=False, tqdm_class=None, save_best_after=None, checkpoint_freq=None, save_filepath=None, **kwargs)

Wrapper for the standard keras fit method.

Adds callbacks and then trains the model.

Parameters:

• args (arguments) – Arguments for keras.Model.fit().

• use_tqdm (bool, optional) – Should we use a tqdm progress bar instead of the usual output from tensorflow.

• tqdm_class (TqdmCallback, optional) – Class for the tqdm progress bar.

• save_best_after (int, optional) – Epoch number after which we should save the best model. The best model is that which achieves the lowest loss.

• checkpoint_freq (int, optional) – Frequency (in epochs) at which to create checkpoints.

• save_filepath (str, optional) – Path to save the best model to (if save_best_after is passed) or directory to save the checkpoints to (if checkpoint_freq is passed). Can’t be passed if both save_best_after and checkpoint_freq are passed.

• additional_callbacks (list, optional) – List of keras callback objects.

• kwargs (keyword arguments, optional) – Keyword arguments for keras.Model.fit().

Returns:

history – The training history.

Return type:

dict

train(*args, best_of=None, save_dir=None, **kwargs)

Wrapper for initializing and fitting the model.

Parameters:

• *args (arguments) – Arguments to pass to both the initialization and fit method.

• best_of (int, optional) – How many runs should we perform? We will return the best run (which is the one with the lowest variational free energy). Defaults to config.best_of.

• save_dir (str, optional) – Directory to save each run to. If None, the models are not saved.

• **kwargs (keyword arguments) – Keyword arguments to pass to both the initialization and fit method.

Return type:

None

load_weights(filepath)

Load weights of the model from a file.

Parameters:

filepath (str) – Path to file containing model weights.

Return type:

None

reset_weights(keep=None)

Re-initialize the weights of the model using their initializers.

Weights with a random initializer are given new random values. Note, if a parameter has been set with update_initializer=True (e.g. set_means, set_covariances or an initialization method such as random_state_time_course_initialization), its initializer is replaced by the value that was set. This parameter is then reset to that value rather than a new random value.

Parameters:

keep (list of str, optional) – Layer names to NOT reset.

Return type:

None

reset()

Reset the model weights and re-compile (creating a new optimizer).

See reset_weights for how the weights are re-initialized.

Return type:

None

make_dataset(inputs, shuffle=False, concatenate=False, step_size=None, drop_last_batch=False)

Converts a Data object into a TensorFlow Dataset.

Parameters:

• inputs (osl_dynamics.data.Data or str or np.ndarray) – Data object. If a str or :np.ndarray: is passed this function will first convert it into a Data object.

• shuffle (bool, optional) – Should we shuffle the data?

• concatenate (bool, optional) – Should we return a single TensorFlow Dataset or a list of Datasets.

• step_size (int, optional) – Number of samples to slide the sequence across the dataset. Default is no overlap.

• drop_last_batch (bool, optional) – Should we drop the last batch if it is smaller than the batch size?

Returns:

dataset – TensorFlow Dataset (or list of Datasets) that can be used for training/evaluating.

Return type:

tf.data.Dataset or list

get_training_time_series(training_data, prepared=True, concatenate=False)

Get the time series used for training from a Data object.

Parameters:

• training_data (osl_dynamics.data.Data) – Data object.

• prepared (bool, optional) – Should we return the prepared data? If not, we return the raw data.

• concatenate (bool, optional) – Should we concatenate the data for each session?

Returns:

training_data – Training data time series.

Return type:

np.ndarray or list

get_static_loss_scaling_factor(n_sequences)

Get scaling factor for static losses.

When calculating loss, we want to approximate the effect of the regularization across the entire training dataset. To do this We divide the regularization by the total number of sequences.

Parameters:

n_sequences (int) – Total number of sequences in the training dataset.

Returns:

scale_factor – Scale factor for ‘static’ losses, i.e. those which are not time varying.

Return type:

float

summary_string()

Return a summary of the model as a string.

This is a modified version of the keras.Model.summary() method which makes the output easier to parse.

Return type:

str

summary_table(renderer)

Return a summary of the model as a table (HTML or LaTeX).

Parameters:

renderer (str) – Renderer to use. Either "html" or "latex".

Returns:

table – Summary of the model as a table.

Return type:

str

html_summary()

Return a summary of the model as an HTML table.

Return type:

str

latex_summary()

Return a summary of the model as a LaTeX table.

Return type:

str

save_config(dirname)

Saves config object as a .yml file.

Parameters:

dirname (str) – Directory to save config.yml.

Return type:

None

save(dirname)

Saves config object and weights of the model.

This is a wrapper for self.save_config and self.model.save_weights.

Parameters:

dirname (str) – Directory to save the config object and weights of the model.

Return type:

None

set_trainable(layers, values)

Context manager to temporarily set the trainable attribute of layers.

Parameters:

• layers (str or list of str) – List of layers to set the trainable attribute of.

• values (bool or list of bool) – Value to set the trainable attribute of the layers to.

Return type:

Generator

static load_config(dirname)

Load a config object from a .yml file.

Parameters:

dirname (str) – Directory to load config.yml from.

Returns:

• config (dict) – Dictionary containing values used to create the config object.

• version (str) – Version used to train the model.

Return type:

Tuple[dict, str]

classmethod load(dirname, from_checkpoint=False, single_gpu=True)

Load model from dirname.

Parameters:

• dirname (str) – Directory where config.yml and weights are stored.

• from_checkpoint (bool, optional) – Should we load the model from the latest checkpoint in <dirname>/checkpoints? Note, the optimizer state is not restored.

• single_gpu (bool, optional) – Should we compile the model on a single GPU?

Returns:

model – Model object.

Return type:

Model

property is_multi_gpu: bool

Returns True if the model’s strategy is MirroredStrategy.

Return type:

bool