ml4cps.nn.rbm module

The module implements the novel DENTA algorithm for the learning of hybrid automata from data.

Author: Nemanja Hranisavljevic, hranisan@hsu-hh.de, nemanja@ai4cps.com

class ml4cps.nn.rbm.BinaryBinaryRBM(n_visible, n_hidden, device='cpu')

Bases: XBinaryRBM

free_energy(x)

Compute the free energy of the visible units. :param x: Visible states :return: Free energy scalar

h2v(h)
sample_x(vis_probs)

Sample hidden states given visible states. :param visible: Input visible states :return: Binary hidden states and probabilities

class ml4cps.nn.rbm.FractionalBinaryRBM(n_visible, n_hidden, n_levels=5, device='cpu')

Bases: BinaryBinaryRBM

decode_input(x)
class ml4cps.nn.rbm.GaussianBinaryRBM(n_visible, n_hidden, device='cpu', replicate_h=1, sigma_learnable=False, sigma=1.0)

Bases: XBinaryRBM

free_energy(x)

Compute the free energy of the visible units. :param x: Visible states. :return: Vector of free energy scalars.

gmm_model()
h2v(hidden)
sample_x(visible_mean)

Sample visible states given hidden states. :param hidden: Input hidden states :return: Gaussian visible states

v2h(visible, **kwargs)
class ml4cps.nn.rbm.PoissonBinaryRBM(n_visible, n_hidden, device='cpu', replicate_h=1)

Bases: XBinaryRBM

free_energy(x)

Compute the free energy for a given visible state. :param x: Input visible state :return: Free energy

h2v(hidden)

Hidden-to-Visible transformation (mean of the Poisson distribution).

sample_x(visible_mean)

Sample visible states given hidden states. :param visible_mean: Poisson mean (rate parameter) :return: Sampled Poisson visible states

v2h(visible, **kwargs)

Visible-to-Hidden transformation.

class ml4cps.nn.rbm.ReLUBinaryRBM(n_visible, n_hidden, device='cpu', replicate_h=1)

Bases: XBinaryRBM

free_energy(visible)

Compute the free energy for a given visible state. :param visible: Input visible state :return: Free energy

h2v(hidden)

Hidden-to-Visible transformation using ReLU visible units.

v2h(visible, **kwargs)

Visible-to-Hidden transformation.

class ml4cps.nn.rbm.ReluBinaryRBM(n_visible, n_hidden, device='cpu')

Bases: XBinaryRBM

free_energy(x)

Compute the free energy of the visible units. :param x: Visible states :return: Free energy scalar

sample_x(hidden)

Sample visible states given hidden states. :param hidden: Input hidden states :return: Gaussian visible states

class ml4cps.nn.rbm.ThreeWayBinaryBinaryRBM(n_visible, n_hidden, n_auxiliary, device='cpu')

Bases: BinaryBinaryRBM

Three-Way Binary RBM with additional auxiliary/temporal units.

a2v(auxiliary, hidden)

Auxiliary-to-Visible transformation (optional).

contrastive_divergence(d, num_gibbs=1, use_probability_last_x_update=False)

Perform Contrastive Divergence (CD-k) to train the RBM. :param visible: Input visible states :param num_gibbs: Number of Gibbs sampling steps :return: CD loss

free_energy(visible, auxiliary)

Compute the free energy with three-way interactions.

generate(num_examples, num_steps=10)
hu2v(hidden, auxiliary)

Hidden-to-Visible transformation with three-way interactions.

sample_u(auxiliary_probs)

Sample auxiliary states given visible and hidden states.

vh2u(v, h)
vu2h(visible, auxiliary, pre_sigmoid=False)

Visible-to-Hidden transformation with three-way interactions.

class ml4cps.nn.rbm.XBinaryRBM(n_visible, n_hidden, device='cpu')

Bases: ABC, Module

It is an “abstract” class for modeling RBM with binary latent units.

contrastive_divergence(d, num_gibbs=1, use_probability_last_x_update=False, auxiliary=None)

Perform Contrastive Divergence (CD-k) to train the RBM. :param visible: Input visible states :param num_gibbs: Number of Gibbs sampling steps :return: CD loss

decode_input(x)
fit(data, epochs=10, batch_size=64, k=1, learning_rate=0.01)

Train the RBM using Contrastive Divergence. :param data: Input data (torch.Tensor) :param epochs: Number of epochs :param batch_size: Batch size :param k: Number of Gibbs sampling steps :param learning_rate: Learning rate

forward(visible)

Forward pass: Gibbs sampling for reconstruction. :param visible: Input visible states :return: Reconstructed visible states

abstractmethod free_energy(x)

Compute the free energy of the visible units. :param x: Visible states. :return: Vector of free energy scalars.

generate(num_examples, num_steps=10)
plot_input_space(data=None, samples=None, show_gaussian_components=False, data_limit=10000, xmin=None, xmax=None, ymin=None, ymax=None, figure_width=600, figure_height=600, show_axis_titles=True, show_energy_contours=False, showlegend=True, show_recon_error_contours=False, ncontours=None, plot_code_positions=True, show_recon_error_heatmap=False, plot_bias_vector=False, show_reconstructions=False, plot_separation_lines=False, samples_opacity=0.2, **kwargs)
plot_learning_curve()
prepare_input(d)
recon(v, round=False)
recon_error(data, input=None, per_point=False, round=False)
sample_h(hidden_probs)

Sample hidden states given visible states. :param visible: Input visible states :return: Binary hidden states and probabilities

abstractmethod sample_x(hidden)
train_rbm(train_data, valid_data, learning_rule='cd', min_epoch=0, max_epoch=10, weight_decay=0.0, lr=0.01, batch_size=128, shuffle=True, num_gibbs=1, verbose=True, early_stopping=False, early_stopping_patience=3, use_probability_last_x_update=True, log_mlflow=False, denoising=False, sparsity_weight=0, sparsity_target=0.1, optimizer='RMSprop')

param: train_data: Training data, list of tensors. param: valid_data: Validation data, list of tensors.

v2h(visible, pre_sigmoid=False)
ml4cps.nn.rbm.mse(x, r, per_point=False)
ml4cps.nn.rbm.sparsity(activations, per_point=False)