osl_dynamics.inference.layers#

Custom Tensorflow layers.

Attributes#

tfb

Classes#

`TFConcatLayer`	Wrapper for tf.concat.
`TFMatMulLayer`	Wrapper for tf.matmul.
`TFRangeLayer`	Wrapper for tf.range.
`TFZerosLayer`	Wrapper for tf.zeros.
`TFBroadcastToLayer`	Wrapper for tf.broadcast_to.
`TFGatherLayer`	Wrapper for tf.gather.
`TFAddLayer`	Wrapper for tf.add.
`TFConstantLayer`	Wrapper for tf.constant.
`DummyLayer`	Returns the inputs without modification.
`InverseCholeskyLayer`	Layer for getting Cholesky vectors from positive definite symmetric matrices.
`BatchSizeLayer`	Layer for getting the batch size.
`SampleGammaDistributionLayer`	Layer for sampling from a Gamma distribution.
`SampleNormalDistributionLayer`	Layer for sampling from a Normal distribution.
`SampleGumbelSoftmaxDistributionLayer`	Layer for sampling from a Gumbel-Softmax distribution.
`SampleOneHotCategoricalDistributionLayer`	Layer for sampling from a Categorical distribution.
`SoftmaxLayer`	Layer for applying a softmax activation function.
`LearnableTensorLayer`	Layer to learn a tensor.
`VectorsLayer`	Layer to learn a set of vectors.
`CovarianceMatricesLayer`	Layer to learn a set of covariance matrices.
`CorrelationMatricesLayer`	Layer to learn a set of correlation matrices.
`DiagonalMatricesLayer`	Layer to learn a set of diagonal matrices.
`OscillatorCovarianceMatricesLayer`	Layer to learn a set of oscillator covariances.
`MatrixLayer`	Layer to learn a matrix.
`MixVectorsLayer`	Mix a set of vectors.
`MixMatricesLayer`	Layer to mix matrices.
`LogLikelihoodLossLayer`	Layer to calculate the negative log-likelihood.
`KLDivergenceLayer`	Layer to calculate a KL divergence between two Normal distributions.
`KLLossLayer`	Layer to calculate the KL loss.
`InferenceRNNLayer`	RNN inference network.
`ModelRNNLayer`	RNN generative model.
`CategoricalKLDivergenceLayer`	Layer to calculate a KL divergence between two categorical distributions.
`CategoricalLogLikelihoodLossLayer`	Layer to calculate the log-likelihood loss assuming a categorical model.
`ConcatEmbeddingsLayer`	Layer for getting the concatenated embeddings.
`SessionParamLayer`	Layer for getting the array specific parameters.
`MixSessionSpecificParametersLayer`	Class for mixing means and covariances.
`GammaExponentialKLDivergenceLayer`	Gamma exponential KL diverengnce layer.
`MultiLayerPerceptronLayer`	Multi-Layer Perceptron layer.
`HiddenMarkovStateInferenceLayer`	Hidden Markov state inference layer.
`SeparateLogLikelihoodLayer`	Layer to calculate the log-likelihood for different HMM states.
`SeparatePoissonLogLikelihoodLayer`	Layer to calculate the log-likelihood for different HMM-Poisson states.
`SumLogLikelihoodLossLayer`	Layer for summing log-likelihoods.
`EmbeddingLayer`	Layer for embeddings.

Functions#

`add_epsilon`(A, epsilon[, diag])	Add epsilon.
`NormalizationLayer`(norm_type, args, *kwargs)	Returns a normalization layer.
`RNNLayer`(rnn_type, args, *kwargs)	Returns a Recurrent Neural Network (RNN) layer.

Module Contents#

osl_dynamics.inference.layers.tfb[source]#

osl_dynamics.inference.layers.add_epsilon(A, epsilon, diag=False)[source]#

Add epsilon.

Adds epsilon the the diagonal of batches of square matrices or all elements of matrices.

Parameters:

A (tf.Tensor) – Batches of square matrices or vectors. Shape is (…, N, N) or (…, N).
epsilon (float) – Small error added to the diagonal of the matrices or every element of the vectors.
diag (bool, optional) – Do we want to add epsilon to the diagonal only?

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.TFConcatLayer(axis, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.concat.

Parameters:

axis (int) – Axis to concatenate along.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

axis[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFMatMulLayer[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.matmul.

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFRangeLayer(limit, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.range.

Parameters:

limit (int) – Upper limit for range.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

limit[source]#

call(inputs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFZerosLayer(shape, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.zeros.

Parameters:

shape (tuple) – Shape of the zeros tensor.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

shape[source]#

call(inputs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFBroadcastToLayer(n_modes, n_channels, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.broadcast_to.

Parameters:

n_modes (int)
n_channels (int)

n_modes[source]#

n_channels[source]#

call(inputs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFGatherLayer(axis, batch_dims=0, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.gather.

Parameters:

axis (int)
batch_dims (int)

axis[source]#

batch_dims = 0[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFAddLayer[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.add.

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.TFConstantLayer(values, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Wrapper for tf.constant.

Parameters:: values (numpy.ndarray)

values[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

osl_dynamics.inference.layers.NormalizationLayer(norm_type, *args, **kwargs)[source]#

Returns a normalization layer.

Parameters:

norm_type (str) – Type of normalization layer. Either 'layer', 'batch' or None.
args (arguments) – Arguments to pass to the normalization layer.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the normalization layer.

Return type:

tensorflow.keras.layers.Layer

osl_dynamics.inference.layers.RNNLayer(rnn_type, *args, **kwargs)[source]#

Returns a Recurrent Neural Network (RNN) layer.

Parameters:

rnn_type (str) – Type of RNN. Either 'lstm' or 'gru'.
args (arguments) – Arguments to pass to the normalization layer.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the normalization layer.

Return type:

tensorflow.keras.layers.Layer

class osl_dynamics.inference.layers.DummyLayer[source]#

Bases: tensorflow.keras.layers.Layer

Returns the inputs without modification.

call(inputs, **kwargs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.InverseCholeskyLayer(epsilon, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for getting Cholesky vectors from positive definite symmetric matrices.

Parameters:

epsilon (float) – Small error added to the diagonal of the matrices.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

epsilon[source]#

bijector[source]#

call(inputs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.BatchSizeLayer[source]#

Bases: tensorflow.keras.layers.Layer

Layer for getting the batch size.

Parameters:: kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

call(inputs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.SampleGammaDistributionLayer(epsilon, do_annealing, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for sampling from a Gamma distribution.

This layer is a wrapper for tfp.distributions.Gamma.

Parameters:

epsilon (float) – Error to add to the shape and rate for numerical stability.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.
do_annealing (bool)

epsilon[source]#

call(inputs, training=None, **kwargs)[source]#

This method accepts the shape and rate and outputs the samples.

Parameters:

inputs (List[tensorflow.Tensor])
training (Optional[bool])

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.SampleNormalDistributionLayer(epsilon, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for sampling from a Normal distribution.

This layer is a wrapper for tfp.distributions.Normal.

Parameters:

epsilon (float) – Error to add to the standard deviations for numerical stability.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

epsilon[source]#

call(inputs, training=None, **kwargs)[source]#

This method accepts the mean and the standard deviation and outputs the samples.

Parameters:

inputs (List[tensorflow.Tensor])
training (Optional[bool])

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.SampleGumbelSoftmaxDistributionLayer(temperature, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for sampling from a Gumbel-Softmax distribution.

This layer is a wrapper for tfp.distributions.RelaxedOneHotCategorical.

Parameters:

temperature (float) – Temperature for the Gumbel-Softmax distribution.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

temperature[source]#

call(inputs, training=None, **kwargs)[source]#

This method accepts logits and outputs samples.

Parameters:

inputs (tensorflow.Tensor)
training (Optional[bool])

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.SampleOneHotCategoricalDistributionLayer[source]#

Bases: tensorflow.keras.layers.Layer

Layer for sampling from a Categorical distribution.

This layer is a wrapper for tfp.distributions.OneHotCategorical.

call(inputs, **kwargs)[source]#

The method accepts logits and outputs samples.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.SoftmaxLayer(initial_temperature, learn_temperature, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for applying a softmax activation function.

Parameters:

initial_temperature (float) – Temperature parameter.
learn_temperature (bool) – Should we learn the alpha temperature?
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.LearnableTensorLayer(shape, learn=True, initializer=None, initial_value=None, regularizer=None, constraint=None, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a tensor.

Parameters:

shape (tuple) – Shape of the tensor.
learn (bool, optional) – Should we learn the tensor?
initializer (tf.keras.initializers.Initializer, optional) – Initializer for the tensor.
initial_value (float, optional) – Initial value for the tensor if initializer is not passed.
regularizer (osl-dynamics regularizer, optional) – Regularizer for the tensor. Must be from osl_dynamics.inference.regularizers.
constraint (tf.keras.constraints.Constraint, optional) – Constraint for the tensor. Limits the values the weights can take.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

learn = True[source]#

shape[source]#

initial_value = None[source]#

regularizer = None[source]#

constraint = None[source]#

add_regularization(tensor)[source]#

Parameters:: tensor (tensorflow.Tensor)
Return type:: None

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, training=None, **kwargs)[source]#

Parameters:

inputs (tensorflow.Tensor)
training (Optional[bool])

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.VectorsLayer(n, m, learn, initial_value, regularizer=None, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a set of vectors.

Parameters:

n (int) – Number of vectors.
m (int) – Number of elements.
learn (bool) – Should we learn the vectors?
initial_value (np.ndarray) – Initial value for the vectors. Shape must be (n, m).
regularizer (osl-dynamics regularizer, optional) – Regularizer for the tensor. Must be from osl_dynamics.inference.regularizers.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.CovarianceMatricesLayer(n, m, learn, initial_value, epsilon, regularizer=None, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a set of covariance matrices.

A cholesky factor is learnt and used to calculate a covariance matrix as \(C = LL^T\), where \(L\) is the cholesky factor. The cholesky factor is learnt as a vector of free parameters.

Parameters:

n (int) – Number of matrices.
m (int) – Number of rows/columns.
learn (bool) – Should the matrices be learnable?
initial_value (np.ndarray) – Initial values for the matrices. Shape must be (n, m, m).
epsilon (float) – Error added to the diagonal of covariances matrices for numerical stability.
regularizer (osl-dynamics regularizer, optional) – Regularizer for the tensor. Must be from osl_dynamics.inference.regularizers.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

epsilon[source]#

bijector[source]#

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.CorrelationMatricesLayer(n, m, learn, initial_value, epsilon, regularizer=None, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a set of correlation matrices.

A cholesky factor is learnt as a vector of free parameters and used to calculate a correlation matrix.

Parameters:

n (int) – Number of matrices.
m (int) – Number of rows/columns.
learn (bool) – Should the matrices be learnable?
initial_value (np.ndarray) – Initial values for the matrices. Shape must be (n, m, m).
epsilon (float) – Error added to the diagonal of correlation matrices for numerical stability.
regularizer (osl-dynamics regularizer, optional) – Regularizer for the tensor. Must be from osl_dynamics.inference.regularizers.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

epsilon[source]#

bijector[source]#

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.DiagonalMatricesLayer(n, m, learn, initial_value, epsilon, regularizer=None, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a set of diagonal matrices.

The diagonal is forced to be positive using a softplus transformation.

Parameters:

n (int) – Number of matrices.
m (int) – Number of rows/columns.
learn (bool) – Should the matrices be learnable?
initial_value (np.ndarray) – Initial values for the matrices. Shape must be (n, m, m) or (n, m).
epsilon (float) – Error added to the diagonal matrices for numerical stability.
regularizer (osl-dynamics regularizer, optional) – Regularizer for the tensor. Must be from osl_dynamics.inference.regularizers.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

epsilon[source]#

bijector[source]#

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.OscillatorCovarianceMatricesLayer(n, m, sampling_frequency, frequency_range, learn, epsilon, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a set of oscillator covariances.

Parameters:

n (int) – Number of matrices.
m (int) – Number of rows/columns.
sampling_frequency (float) – Sampling frequency in Hz.
frequency_range (tuple[float, float]) – Limits for the frequency parameter. Upper limit should not be higher than the Nyquist frequency.
learn (bool) – Should we learn the covariances?
epsilon (float) – Error added to the diagonal matrices for numerical stability.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

m[source]#

n[source]#

sampling_frequency[source]#

epsilon[source]#

tau[source]#

amplitude[source]#

frequency[source]#

variance[source]#

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Retrieve the covariance matrices.

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.MatrixLayer(m, constraint, learn, initial_value, epsilon, regularizer=None, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to learn a matrix.

Parameters:

m (int) – Number of rows/columns.
constraint (str) – Either ‘covariance’ or ‘diagonal’.
learn (bool) – Should the matrix be trainable?
initial_value (np.ndarray) – Initial value for the matrix.
epsilon (float) – Error added to the matrices for numerical stability.
regularizer (osl-dynamics regularizer, optional) – Regularizer for the tensor. Must be from osl_dynamics.inference.regularizers.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

constraint[source]#

layers[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Note

The inputs passed to this method are not used.

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.MixVectorsLayer[source]#

Bases: tensorflow.keras.layers.Layer

Mix a set of vectors.

The mixture is calculated as \(m_t = \displaystyle\sum_j \alpha_{jt} \mu_j\), where \(\mu_j\) are the vectors and \(\alpha_{jt}\) are mixing coefficients.

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.MixMatricesLayer[source]#

Bases: tensorflow.keras.layers.Layer

Layer to mix matrices.

The mixture is calculated as \(C_t = \displaystyle\sum_j \alpha_{jt} D_j\), where \(D_j\) are the matrices and \(\alpha_{jt}\) are mixing coefficients.

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.LogLikelihoodLossLayer(calculation, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate the negative log-likelihood.

We assume a multivariate normal probability density. This layer will add the negative log-likelihood to the loss.

Parameters:

calculation (str) – Operation for reducing the time dimension. Either ‘mean’ or ‘sum’.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

calculation[source]#

call(inputs)[source]#

The method takes the data, mean vector and covariance matrix.

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.KLDivergenceLayer(epsilon, calculation, clip_start=0, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate a KL divergence between two Normal distributions.

Parameters:

epsilon (float) – Error added to the standard deviations for numerical stability.
calculation (str) – Operation for reducing the time dimension. Either ‘mean’ or ‘sum’.
clip_start (int, optional) – Index to clip the sequences inputted to this layer. Default is no clipping.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

clip_start = 0[source]#

epsilon[source]#

calculation[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.KLLossLayer(do_annealing, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate the KL loss.

This layer sums KL divergences if multiple values as passed, applies an annealing factor and adds the value to the loss function.

Parameters:

do_annealing (bool) – Should we perform KL annealing?
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

call(inputs, training=False, **kwargs)[source]#

Parameters:

inputs (Union[tensorflow.Tensor, List[tensorflow.Tensor]])
training (bool)

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.InferenceRNNLayer(rnn_type, norm_type, act_type, n_layers, n_units, drop_rate, reg, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

RNN inference network.

Parameters:

rnn_type (str) – Either 'lstm' or 'gru'.
norm_type (str) – Either 'layer', 'batch' or None.
act_type ('str') – Activation type, e.g. 'relu', 'elu', etc.
n_layers (int) – Number of layers.
n_units (int) – Number of units/neurons per layer.
drop_rate (float) – Dropout rate for the output of each layer.
reg (str) – Regularization for each layer.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

layers = [][source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.ModelRNNLayer(rnn_type, norm_type, act_type, n_layers, n_units, drop_rate, reg, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

RNN generative model.

Parameters:

rnn_type (str) – Either 'lstm' or 'gru'.
norm_type (str) – Either 'layer', 'batch' or None.
act_type ('str') – Activation type, e.g. 'relu', 'elu', etc.
n_layers (int) – Number of layers.
n_units (int) – Number of units/neurons per layer.
drop_rate (float) – Dropout rate for the output of each layer.
reg (str) – Regularization for each layer.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

layers = [][source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.CategoricalKLDivergenceLayer(calculation, clip_start=0, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate a KL divergence between two categorical distributions.

Parameters:

calculation (str) – Operation for reducing the time dimension. Either ‘mean’ or ‘sum’.
clip_start (int, optional) – Index to clip the sequences inputted to this layer. Default is no clipping.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

calculation[source]#

clip_start = 0[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.CategoricalLogLikelihoodLossLayer(n_states, calculation, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate the log-likelihood loss assuming a categorical model.

Parameters:

n_states (int) – Number of states.
calculation (str) – Operation for reducing the time dimension. Either ‘mean’ or ‘sum’.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

n_states[source]#

calculation[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.ConcatEmbeddingsLayer[source]#

Bases: tensorflow.keras.layers.Layer

Layer for getting the concatenated embeddings.

The concatenated embeddings are obtained by concatenating embeddings and spatial embeddings.

call(inputs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.SessionParamLayer(param, epsilon, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for getting the array specific parameters.

This layer adds deviations to the group spatial parameters.

Parameters:

param (str) – Which parameter are we using? Must be 'means' or 'covariances'.
epsilon (float) – Error added to the diagonal of covariances for numerical stability.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

param[source]#

epsilon[source]#

call(inputs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.MixSessionSpecificParametersLayer[source]#

Bases: tensorflow.keras.layers.Layer

Class for mixing means and covariances.

The mixture is calculated as

\(m_t = \displaystyle\sum_j \alpha_{jt} \mu_j^{s_t}\)
\(C_t = \displaystyle\sum_j \alpha_{jt} D_j^{s_t}\)

where \(s_t\) is the array at time \(t\).

call(inputs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: Tuple[tensorflow.Tensor, tensorflow.Tensor]

class osl_dynamics.inference.layers.GammaExponentialKLDivergenceLayer(epsilon, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Gamma exponential KL diverengnce layer.

Layer to calculate KL divergence between Gamma posterior and exponential prior for deviation magnitude.

Parameters:

epsilon (float) – Error added to the standard deviations for numerical stability.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

epsilon[source]#

scale_factor = -1[source]#

call(inputs, training=None, **kwargs)[source]#

Parameters:

inputs (List[tensorflow.Tensor])
training (Optional[bool])

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.MultiLayerPerceptronLayer(n_layers, n_units, norm_type, act_type, drop_rate, regularizer=None, regularizer_strength=1, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Multi-Layer Perceptron layer.

Parameters:

n_layers (int) – Number of layers.
n_units (int) – Number of units/neurons.
norm_type (str) – Normalization layer type. Can be 'layer', 'batch' or None.
act_type (str) – Activation type.
drop_rate (float) – Dropout rate.
regularizer (str, optional) – Regularizer type.
regularizer_strength (float, optional) – Constant to multiply the regularization by.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the base class.

regularizer[source]#

regularizer_strength = 1[source]#

layers = [][source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, training=None, **kwargs)[source]#

Parameters:

inputs (tensorflow.Tensor)
training (Optional[bool])

Return type:

tensorflow.Tensor

class osl_dynamics.inference.layers.HiddenMarkovStateInferenceLayer(n_states, sequence_length, initial_trans_prob, trans_prob_prior, initial_state_probs, learn_trans_prob, learn_initial_state_probs, implementation='rescale', use_stationary_distribution=False, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Hidden Markov state inference layer.

This layer uses the Baum-Welch algorithm to calculate the posterior for the hidden state in a Hidden Markov Model (HMM).

Parameters:

n_states (int) – Number of states.
sequence_length (int) – Length of the sequence.
initial_trans_prob (np.ndarray) – Initial transition probability matrix. Shape must be (n_states, n_states).
trans_prob_prior (np.ndarray) – Dirichlet prior for the transition probability matrix.
initial_state_probs (np.ndarray) – Initial transition probability matrix. Shape must be (n_states,)
learn_trans_prob (bool) – Should we learn the transition probability matrix?
learn_initial_state_probs (bool) – Should we learn the initial state probabilities?
implementation (str, optional) – ‘rescale’ or ‘log’ implementation of the Baum-Welch algorithm.
use_stationary_distribution (bool, optional) – Should we use the stationary distribution (estimated from the transition probability matrix) for the initial state probabilities?
kwargs (keyword arguments, optional) – Keyword arguments to pass to the normalization layer.

n_states[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

get_stationary_distribution()[source]#

Return type:: tensorflow.Tensor

get_initial_state_probs()[source]#

Return type:: tensorflow.Tensor

get_trans_prob()[source]#

Return type:: tensorflow.Tensor

call(log_B, **kwargs)[source]#

Parameters:: log_B (tensorflow.Tensor)
Return type:: Tuple[tensorflow.Tensor, tensorflow.Tensor]

compute_output_shape(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: Tuple[tensorflow.TensorShape, tuple]

class osl_dynamics.inference.layers.SeparateLogLikelihoodLayer(n_states, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate the log-likelihood for different HMM states.

Parameters:

n_states (int) – Number of states.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the keras.layers.Layer.

n_states[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

compute_output_shape(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: tensorflow.TensorShape

class osl_dynamics.inference.layers.SeparatePoissonLogLikelihoodLayer(n_states, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer to calculate the log-likelihood for different HMM-Poisson states.

Parameters:

n_states (int) – Number of states.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the keras.layers.Layer.

n_states[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.SumLogLikelihoodLossLayer(calculation, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for summing log-likelihoods.

Parameters:

calculation (str) – Operation for reducing the time dimension. Either ‘mean’ or ‘sum’.
kwargs (keyword arguments, optional) – Keyword arguments to pass to the keras.layers.Layer.

calculation[source]#

call(inputs, **kwargs)[source]#

Parameters:: inputs (List[tensorflow.Tensor])
Return type:: tensorflow.Tensor

class osl_dynamics.inference.layers.EmbeddingLayer(input_dim, output_dim, unit_norm, **kwargs)[source]#

Bases: tensorflow.keras.layers.Layer

Layer for embeddings.

Parameters:

input_dim (int) – Input dimension.
output_dim (int) – Output dimension.
unit_norm (bool, optional) – Should the embeddings be unit norm?

embedding_layer[source]#

layers[source]#

unit_norm[source]#

build(input_shape)[source]#

Parameters:: input_shape (tensorflow.TensorShape)
Return type:: None

call(inputs, **kwargs)[source]#

Parameters:: inputs (tensorflow.Tensor)
Return type:: tensorflow.Tensor

property embeddings: tensorflow.Tensor[source]#

Return type:: tensorflow.Tensor