hex.deeplearning

Class DeepLearningModel

java.lang.Object

water.Iced<T>

water.Keyed<T>

water.Lockable<M>

hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

hex.deeplearning.DeepLearningModel

All Implemented Interfaces:

hex.Model.DeepFeatures, java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable<DeepLearningModel>

public class DeepLearningModel
extends hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>
implements hex.Model.DeepFeatures

The Deep Learning model It contains a DeepLearningModelInfo with the most up-to-date model, a scoring history, as well as some helpers to indicate the progress

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DeepLearningModel.DeepLearningModelOutput The Deep Learning model output contains a few extra fields in addition to the metrics in Model.Output 1) Scoring history (raw data) 2) weights/biases (raw data) 3) variable importances (TwoDimTable)
static class	DeepLearningModel.DeepLearningScoring

Nested classes/interfaces inherited from class hex.Model

hex.Model.DeepFeatures, hex.Model.Output, hex.Model.Parameters

Nested classes/interfaces inherited from class water.Keyed

water.Keyed.BinarySerializer<X extends water.Keyed>

Field Summary

Fields

Modifier and Type	Field and Description
long	_timeLastScoreEnter
water.Key	actual_best_model_key
long	actual_train_samples_per_iteration
double	epoch_counter
water.Key	model_info_key
long	run_time
boolean	stopped_early
double	time_for_communication_us
long	training_rows
long	tspiGuess
long	validation_rows

Fields inherited from class hex.Model

_output, _parms, _warnings

Fields inherited from class water.Lockable

_lockers

Fields inherited from class water.Keyed

_key, EMPTY_KEY_LIST

Constructor Summary

Constructors

Constructor and Description
DeepLearningModel(water.Key destKey, DeepLearningParameters parms, DeepLearningModel.DeepLearningModelOutput output, water.fvec.Frame train, water.fvec.Frame valid, int nClasses) Regular constructor (from scratch)
DeepLearningModel(water.Key destKey, DeepLearningParameters parms, DeepLearningModel cp, boolean store_best_model, DataInfo dataInfo) Constructor to restart from a checkpointed model

Method Summary

Methods

Modifier and Type	Method and Description
double	calcOutlierThreshold(water.fvec.Vec mse, double quantile) Compute quantile-based threshold (in reconstruction error) to find outliers
protected long	checksum_impl()
double	classification_error()
int	compareTo(DeepLearningModel o)
void	delete()
double	deviance()
double	deviance(double w, double y, double f)
float	error()
DeepLearningParameters	get_params() Get the parameters actually used for model building, not the user-given ones (_parms) They might differ since some defaults are filled in, and some invalid combinations are auto-disabled in modifyParams
double	logloss()
double	loss(DataInfo.Row[] myRows) Compute the loss function
hex.ModelMetrics.MetricBuilder	makeMetricBuilder(java.lang.String[] domain)
DeepLearningModelInfo	model_info()
double	mse()
protected water.fvec.Frame	predictScoreImpl(water.fvec.Frame orig, water.fvec.Frame adaptedFr, java.lang.String destination_key) Make either a prediction or a reconstruction.
water.api.ModelSchema	schema()
protected double[]	score0(double[] data, double[] preds)
double[]	score0(double[] data, double[] preds, double weight, double offset) Predict from raw double values representing the data
water.fvec.Frame	scoreAutoEncoder(water.fvec.Frame frame, water.Key destination_key) Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))
water.fvec.Frame	scoreDeepFeatures(water.fvec.Frame frame, int layer) Score auto-encoded reconstruction (on-the-fly, and materialize the deep features of given layer
DeepLearningModel.DeepLearningScoring[]	scoring_history()
protected boolean	toJavaCheckTooBig()
protected water.util.SB	toJavaInit(water.util.SB sb, water.util.SB fileContextSB)
protected void	toJavaPredictBody(water.util.SB bodySb, water.util.SB classCtxSb, water.util.SB fileCtxSb)
hex.VarImp	varImp()

Methods inherited from class hex.Model

adaptTestForTrain, adaptTestForTrain, addMetrics, addWarning, cleanup_adapt, defaultThreshold, getPublishedKeys, isSupervised, remove_impl, score, score, score, score0, score0, scoreMetrics, testJavaScoring, toJava, toJava, toJavaNCLASSES, toJavaPROB, toJavaSuper, toString

Methods inherited from class water.Lockable

delete_and_lock, delete, delete, read_lock, read_lock, unlock_all, unlock, update, write_lock

Methods inherited from class water.Keyed

checksum, getBinarySerializer, remove, remove, remove, remove

Methods inherited from class water.Iced

clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

run_time

public long run_time

actual_train_samples_per_iteration

public long actual_train_samples_per_iteration

tspiGuess

public long tspiGuess

time_for_communication_us

public double time_for_communication_us

epoch_counter

public double epoch_counter

stopped_early

public boolean stopped_early

training_rows

public long training_rows

validation_rows

public long validation_rows

actual_best_model_key

public water.Key actual_best_model_key

model_info_key

public water.Key model_info_key

_timeLastScoreEnter

public long _timeLastScoreEnter

Constructor Detail

DeepLearningModel

public DeepLearningModel(water.Key destKey,
                 DeepLearningParameters parms,
                 DeepLearningModel cp,
                 boolean store_best_model,
                 DataInfo dataInfo)

Constructor to restart from a checkpointed model

Parameters:

destKey - New destination key for the model

parms - User-given parameters for checkpoint restart

cp - Checkpoint to restart from

store_best_model - Store only the best model instead of the latest one

DeepLearningModel

public DeepLearningModel(water.Key destKey,
                 DeepLearningParameters parms,
                 DeepLearningModel.DeepLearningModelOutput output,
                 water.fvec.Frame train,
                 water.fvec.Frame valid,
                 int nClasses)

Regular constructor (from scratch)

Parameters:

destKey -

parms -

output -

train -

valid -

nClasses -

Method Detail

deviance

public double deviance(double w,
              double y,
              double f)

Overrides:

deviance in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

schema

public water.api.ModelSchema schema()

model_info

public final DeepLearningModelInfo model_info()

varImp

public final hex.VarImp varImp()

scoring_history

public DeepLearningModel.DeepLearningScoring[] scoring_history()

get_params

public final DeepLearningParameters get_params()

Get the parameters actually used for model building, not the user-given ones (_parms) They might differ since some defaults are filled in, and some invalid combinations are auto-disabled in modifyParams

Returns:

actually used parameters

error

public float error()

makeMetricBuilder

public hex.ModelMetrics.MetricBuilder makeMetricBuilder(java.lang.String[] domain)

Specified by:

makeMetricBuilder in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

compareTo

public int compareTo(DeepLearningModel o)

classification_error

public double classification_error()

mse

public double mse()

deviance

public double deviance()

logloss

public double logloss()

predictScoreImpl

protected water.fvec.Frame predictScoreImpl(water.fvec.Frame orig,
                                water.fvec.Frame adaptedFr,
                                java.lang.String destination_key)

Make either a prediction or a reconstruction.

Overrides:

predictScoreImpl in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

Parameters:

orig - Test dataset

adaptedFr - Test dataset, adapted to the model

Returns:

A frame containing the prediction or reconstruction

score0

protected double[] score0(double[] data,
              double[] preds)

Specified by:

score0 in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

loss

public double loss(DataInfo.Row[] myRows)

Compute the loss function

Parameters:

myRows - Mini-Batch Array of denseRow's containing numerical/categorical predictor and response data (standardized)

Returns:

loss

score0

public double[] score0(double[] data,
              double[] preds,
              double weight,
              double offset)

Predict from raw double values representing the data

Overrides:

score0 in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

Parameters:

data - raw array containing categorical values (horizontalized to 1,0,0,1,0,0 etc.) and numerical values (0.35,1.24,5.3234,etc), both can contain NaNs

preds - predicted label and per-class probabilities (for classification), predicted target (regression), can contain NaNs

Returns:

preds, can contain NaNs

scoreAutoEncoder

public water.fvec.Frame scoreAutoEncoder(water.fvec.Frame frame,
                                water.Key destination_key)

Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))

Specified by:

scoreAutoEncoder in interface hex.Model.DeepFeatures

Parameters:

frame - Original data (can contain response, will be ignored)

Returns:

Frame containing one Vec with reconstruction error (MSE) of each reconstructed row, caller is responsible for deletion

scoreDeepFeatures

public water.fvec.Frame scoreDeepFeatures(water.fvec.Frame frame,
                                 int layer)

Score auto-encoded reconstruction (on-the-fly, and materialize the deep features of given layer

Specified by:

scoreDeepFeatures in interface hex.Model.DeepFeatures

Parameters:

frame - Original data (can contain response, will be ignored)

layer - index of the hidden layer for which to extract the features

Returns:

Frame containing the deep features (#cols = hidden[layer])

calcOutlierThreshold

public double calcOutlierThreshold(water.fvec.Vec mse,
                          double quantile)

Compute quantile-based threshold (in reconstruction error) to find outliers

Parameters:

mse - Vector containing reconstruction errors

quantile - Quantile for cut-off

Returns:

Threshold in MSE value for a point to be above the quantile

delete

public void delete()

Overrides:

delete in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

toJavaInit

protected water.util.SB toJavaInit(water.util.SB sb,
                       water.util.SB fileContextSB)

Overrides:

toJavaInit in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

toJavaCheckTooBig

protected boolean toJavaCheckTooBig()

Overrides:

toJavaCheckTooBig in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

toJavaPredictBody

protected void toJavaPredictBody(water.util.SB bodySb,
                     water.util.SB classCtxSb,
                     water.util.SB fileCtxSb)

Overrides:

toJavaPredictBody in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

checksum_impl

protected long checksum_impl()

Overrides:

checksum_impl in class hex.Model<DeepLearningModel,DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>