osl_dynamics.models.dive
#
DIVE (DyNeMo with Integrated Variability Estimation).
Module Contents#
Classes#
Settings for DIVE. |
|
DIVE. |
Functions#
|
- class osl_dynamics.models.dive.Config[source]#
Bases:
osl_dynamics.models.mod_base.BaseModelConfig
,osl_dynamics.models.inf_mod_base.VariationalInferenceModelConfig
Settings for DIVE.
- Parameters:
model_name (str) – Model name.
n_modes (int) – Number of modes.
n_channels (int) – Number of channels.
sequence_length (int) – Length of sequence passed to the inference network and generative model.
inference_rnn (str) – RNN to use, either
'gru'
or'lstm'
.inference_n_layers (int) – Number of layers.
inference_n_untis (int) – Number of units.
inference_normalization (str) – Type of normalization to use. Either
None
,'batch'
or'layer'
.inference_activation (str) – Type of activation to use after normalization and before dropout. E.g.
'relu'
,'elu'
, etc.inference_dropout (float) – Dropout rate.
inference_regularizer (str) – Regularizer.
model_rnn (str) – RNN to use, either
'gru'
or'lstm'
.model_n_layers (int) – Number of layers.
model_n_units (int) – Number of units.
model_normalization (str) – Type of normalization to use. Either
None
,'batch'
or'layer'
.model_activation (str) – Type of activation to use after normalization and before dropout. E.g.
'relu'
,'elu'
, etc.model_dropout (float) – Dropout rate.
model_regularizer (str) – Regularizer.
theta_normalization (str) – Type of normalization to apply to the posterior samples,
theta
. Either'layer'
,'batch'
orNone
.learn_alpha_temperature (bool) – Should we learn
alpha_temperature
?initial_alpha_temperature (float) – Initial value for
alpha_temperature
.learn_means (bool) – Should we make the mean vectors for each mode trainable?
learn_covariances (bool) – Should we make the covariance matrix for each mode trainable?
initial_means (np.ndarray) – Initialisation for mean vectors.
initial_covariances (np.ndarray) – Initialisation for mode covariances.
covariances_epsilon (float) – Error added to mode covariances for numerical stability.
means_regularizer (tf.keras.regularizers.Regularizer) – Regularizer for group mean vectors.
covariances_regularizer (tf.keras.regularizers.Regularizer) – Regularizer for group covariance matrices.
do_kl_annealing (bool) – Should we use KL annealing during training?
kl_annealing_curve (str) – Type of KL annealing curve. Either
'linear'
or'tanh'
.kl_annealing_sharpness (float) – Parameter to control the shape of the annealing curve if
kl_annealing_curve='tanh'
.n_kl_annealing_epochs (int) – Number of epochs to perform KL annealing.
batch_size (int) – Mini-batch size.
learning_rate (float) – Learning rate.
lr_decay (float) – Decay for learning rate. Default is 0.1. We use
lr = learning_rate * exp(-lr_decay * epoch)
.gradient_clip (float) – Value to clip gradients by. This is the
clipnorm
argument passed to the Keras optimizer. Cannot be used ifmulti_gpu=True
.n_epochs (int) – Number of training epochs.
optimizer (str or tf.keras.optimizers.Optimizer) – Optimizer to use.
'adam'
is recommended.multi_gpu (bool) – Should be use multiple GPUs for training?
strategy (str) – Strategy for distributed learning.
n_sessions (int) – Number of sessions whose observation model parameters can vary.
embeddings_dim (int) – Number of dimensions for the embedding vectors.
spatial_embeddings_dim (int) – Number of dimensions for the spatial embeddings.
unit_norm_embeddings (bool) – Should we normalize the embeddings to have unit norm?
dev_n_layers (int) – Number of layers for the MLP for deviations.
dev_n_units (int) – Number of units for the MLP for deviations.
dev_normalization (str) – Type of normalization for the MLP for deviations.
dev_activation (str) – Type of activation to use for the MLP for deviations.
dev_dropout (float) – Dropout rate for the MLP for deviations.
dev_regularizer (str) – Regularizer for the MLP for deviations.
dev_regularizer_factor (float) – Regularizer factor for the MLP for deviations. This will be scaled by the amount of data.
initial_dev (dict) – Initialisation for dev posterior parameters.
session_labels (List[SessionLabels]) – List of session labels.
- session_labels: List[osl_dynamics.data.SessionLabels][source]#
- class osl_dynamics.models.dive.Model(config)[source]#
Bases:
osl_dynamics.models.inf_mod_base.VariationalInferenceModelBase
DIVE.
- Parameters:
config (osl_dynamics_models.dive.Config) –
- get_group_means()[source]#
Get the group level mode means.
- Returns:
means – Group means. Shape is (n_modes, n_channels).
- Return type:
np.ndarray
- get_group_covariances()[source]#
Get the group level mode covariances.
- Returns:
covariances – Group covariances. Shape is (n_modes, n_channels, n_channels).
- Return type:
np.ndarray
- get_group_means_covariances()[source]#
Get the group level mode means and covariances.
This is a wrapper for
get_group_means
andget_group_covariances
.- Returns:
means (np.ndarray) – Group means. Shape is (n_modes, n_channels).
covariances (np.ndarray) – Group covariances. Shape is (n_modes, n_channels, n_channels).
- get_embedding_weights()[source]#
Get the weights of the embedding layers.
- Returns:
embedding_weights – Weights of the embedding layers.
- Return type:
dict
- get_session_embeddings()[source]#
Get the embedding vectors for sessions for each session label.
- Returns:
embeddings – Embeddings for each session label.
- Return type:
dict
- get_summed_embeddings()[source]#
Get the summed embeddings.
- Returns:
summed_embeddings – Summed embeddings. Shape is (n_sessions, embeddings_dim).
- Return type:
np.ndarray
- get_session_means_covariances()[source]#
Get the means and covariances for each session.
- Returns:
session_means (np.ndarray) – Mode means for each session. Shape is (n_sessions, n_modes, n_channels).
session_covs (np.ndarray) – Mode covariances for each session. Shape is (n_sessions, n_modes, n_channels, n_channels).
- set_regularizers(training_dataset)[source]#
Set the means and covariances regularizer based on the training data.
A multivariate normal prior is applied to the mean vectors with
mu=0
,sigma=diag((range/2)**2)
and an inverse Wishart prior is applied to the covariances matrices withnu=n_channels-1+0.1
andpsi=diag(range)
.- Parameters:
training_dataset (tf.data.Dataset or osl_dynamics.data.Data) – Training dataset.
- set_dev_parameters_initializer(training_data)[source]#
Set the deviance parameters initializer based on training data.
- Parameters:
training_data (osl_dynamics.data.Data or tf.data.Dataset) – The training data.
- set_embeddings_initializer(initial_embeddings)[source]#
Set the embeddings initializer.
- Parameters:
initial_embeddings (dict) – Initial embeddings for each session label.
- set_group_means(group_means, update_initializer=True)[source]#
Set the group means of each mode.
- Parameters:
group_means (np.ndarray) – Group level mode means. Shape is (n_modes, n_channels).
update_initializer (bool, optional) – Do we want to use the passed group means when we re-initialize the model?
- set_group_covariances(group_covariances, update_initializer=True)[source]#
Set the group covariances of each mode.
- Parameters:
group_covariances (np.ndarray) – Group level mode covariances. Shape is (n_modes, n_channels, n_channels).
update_initializer (bool, optional) – Do we want to use the passed group covariances when we re-initialize the model?
- set_group_means_covariances(group_means, group_covariances, update_initializer=True)[source]#
This is a wrapper for
set_group_means
andset_group_covariances
.