hex.deeplearning

Class Neurons

java.lang.Object

hex.deeplearning.Neurons

Direct Known Subclasses:

Neurons.Input, Neurons.Maxout, Neurons.Output, Neurons.Rectifier, Neurons.Tanh

public abstract class Neurons
extends java.lang.Object

This class implements the concept of a Neuron layer in a Neural Network During training, every MRTask2 F/J thread is expected to create these neurons for every map call (Cheap to make). These Neurons are NOT sent over the wire. The weights connecting the neurons are in a separate class (DeepLearningModel.DeepLearningModelInfo), and will be shared per node.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Neurons.Input Input layer of the Neural Network This layer is different from other layers as it has no incoming weights, but instead gets its activation values from the training points.
static class	Neurons.Linear
static class	Neurons.Maxout
static class	Neurons.MaxoutDropout
static class	Neurons.Output
static class	Neurons.Rectifier
static class	Neurons.RectifierDropout
static class	Neurons.Softmax
static class	Neurons.Tanh
static class	Neurons.TanhDropout

Field Summary

Fields

Modifier and Type	Field and Description
double[]	_a Layer state (one per neuron): activity, error
double[]	_b
protected Dropout	_dropout For Dropout training
double[]	_e Layer state (one per neuron): activity, error
Neurons	_previous References for feed-forward connectivity
float[]	_w
static DocGen.FieldDoc[]	DOC_FIELDS
protected DeepLearning	params Parameters (deep-cloned() from the user input, can be modified here, e.g.
protected int	units

Method Summary

Methods

Modifier and Type	Method and Description
protected abstract void	bprop() Back propagation
protected abstract void	fprop(long seed, boolean training) Forward propagation
void	init(Neurons[] neurons, int index, DeepLearning p, DeepLearningModel.DeepLearningModelInfo minfo, boolean training) Initialization of the parameters and connectivity of a Neuron layer
double	momentum(long n) The momentum - real number in [0, 1) Can be a linear ramp from momentum_start to momentum_stable, over momentum_ramp training samples
double	rate(long n) The learning rate
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DOC_FIELDS

public static DocGen.FieldDoc[] DOC_FIELDS

units

protected int units

params

protected DeepLearning params

Parameters (deep-cloned() from the user input, can be modified here, e.g. learning rate decay)

_a

public transient double[] _a

Layer state (one per neuron): activity, error

_e

public transient double[] _e

Layer state (one per neuron): activity, error

_previous

public Neurons _previous

References for feed-forward connectivity

_w

public float[] _w

_b

public double[] _b

_dropout

protected Dropout _dropout

For Dropout training

Method Detail

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

init

public final void init(Neurons[] neurons,
        int index,
        DeepLearning p,
        DeepLearningModel.DeepLearningModelInfo minfo,
        boolean training)

Initialization of the parameters and connectivity of a Neuron layer

Parameters:

neurons - Array of all neuron layers, to establish feed-forward connectivity

index - Which layer am I?

p - User-given parameters (Job parental object hierarchy is not used)

minfo - Model information (weights/biases and their momenta)

training - Whether training is done or just testing (no need for dropout)

fprop

protected abstract void fprop(long seed,
         boolean training)

Forward propagation

Parameters:

seed - For seeding the RNG inside (for dropout)

training - Whether training is done or just testing (no need for dropout)

bprop

protected abstract void bprop()

Back propagation

rate

public double rate(long n)

The learning rate

Parameters:

n - The number of training samples seen so far (for rate_annealing > 0)

Returns:

Learning rate

momentum

public double momentum(long n)

The momentum - real number in [0, 1) Can be a linear ramp from momentum_start to momentum_stable, over momentum_ramp training samples

Parameters:

n - The number of training samples seen so far

Returns:

momentum