hex.deeplearning

Class DeepLearningModel

java.lang.Object

water.Iced

water.Lockable<Model>

water.Model

hex.deeplearning.DeepLearningModel

All Implemented Interfaces:

java.lang.Cloneable, java.lang.Comparable<DeepLearningModel>, Freezable

public class DeepLearningModel
extends Model
implements java.lang.Comparable<DeepLearningModel>

The Deep Learning model It contains a DeepLearningModelInfo with the most up-to-date model, a scoring history, as well as some helpers to indicate the progress

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DeepLearningModel.DeepLearningModelInfo
static class	DeepLearningModel.Errors

Nested classes/interfaces inherited from class water.Model

Model.ModelAutobufferSerializer, Model.ModelCategory

Field Summary

Fields

Modifier and Type	Field and Description
long	_timeLastScoreEnter
Key	_validationKey
Key	actual_best_model_key
long	actual_train_samples_per_iteration
static DocGen.FieldDoc[]	DOC_FIELDS
double	epoch_counter
double	time_for_communication_us
long	training_rows
long	validation_rows

Fields inherited from class water.Model

_dataKey, _domains, _have_cv_results, _modelClassDist, _names, _priorClassDist, GEN_BENCHMARK_CODE, training_duration_in_ms, training_start_time, warnings

Fields inherited from class water.Lockable

_key, _lockers

Constructor Summary

Constructors

Constructor and Description
DeepLearningModel(DeepLearningModel cp, Key destKey, Key jobKey, FrameTask.DataInfo dataInfo) Constructor to restart from a checkpointed model
DeepLearningModel(Key destKey, Key jobKey, Key dataKey, FrameTask.DataInfo dinfo, DeepLearning params, float[] priorDist)

Method Summary

Methods

Modifier and Type	Method and Description
double	calcOutlierThreshold(Vec mse, double quantile) Compute quantile-based threshold (in reconstruction error) to find outliers
ConfusionMatrix	cm() for grid search error reporting
int	compareTo(DeepLearningModel o)
void	delete_best_model()
void	delete_xval_models()
float	error()
boolean	generateHTML(java.lang.String title, java.lang.StringBuilder sb)
DeepLearning	get_params()
java.lang.String	getHeader()
Model.ModelAutobufferSerializer	getModelSerializer() Returns a model serializer into AutoBuffer.
boolean	isClassifier()
boolean	isSupervised()
Request2	job()
protected double	missingColumnsType() Type of missing columns during adaptation between train/test datasets Overload this method for models that have sparse data handling.
DeepLearningModel.DeepLearningModelInfo	model_info()
double	mse() Returns mse for validation set.
int	nfeatures() Returns number of input features
Frame	score(Frame frame) This is an overridden version of Model.score().
float[]	score0(double[] data, float[] preds) Predict from raw double values representing the data
Frame	scoreAutoEncoder(Frame frame) Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))
Frame	scoreDeepFeatures(Frame frame, int layer) Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))
DeepLearningModel.Errors[]	scoring_history()
protected void	setCrossValidationError(Job.ValidatedJob job, double cv_error, ConfusionMatrix cm, AUCData auc, HitRatio hr)
protected void	toJavaFillPreds0(SB bodySb) Fill preds[0] based on already filled and unified preds[1,..NCLASSES].
void	toJavaHtml(java.lang.StringBuilder sb)
protected SB	toJavaInit(SB sb, SB fileContextSB)
protected SB	toJavaNCLASSES(SB sb)
protected void	toJavaPredictBody(SB bodySb, SB classCtxSb, SB fileCtxSb)
java.lang.String	toString()
java.lang.String	toStringAll()
VarImp	varimp() Variable importance of individual input features measured by this model.

Methods inherited from class water.Model

adapt, adapt, addWarning, calcError, classNames, delete_impl, errStr, getDomainMapping, getDomainMapping, getModelCategory, getUniqueId, hasCrossValModels, nclasses, printCrossValidationModelsHTML, responseName, score, score, score, score, score, score0, scoreCrossValidation, scoreImpl, setModelClassDistribution, start_training, start_training, stop_training, testJavaScoring, toJava, toJava, toJavaDefaultMaxIters, toJavaInit, toJavaSuper, toJavaUnifyPreds

Methods inherited from class water.Lockable

delete_and_lock, delete, delete, delete, delete, is_unlocked, is_wlocked, read_lock, read_lock, unlock_all, unlock_lockable, unlock, update, write_lock

Methods inherited from class water.Iced

clone, frozenType, init, newInstance, read, toDocField, write, writeJSON, writeJSONFields

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DOC_FIELDS

public static DocGen.FieldDoc[] DOC_FIELDS

_validationKey

@Request.API(help="Validation dataset used for model building",
             json=true)
public final Key _validationKey

actual_train_samples_per_iteration

public long actual_train_samples_per_iteration

time_for_communication_us

public double time_for_communication_us

epoch_counter

@Request.API(help="Number of training epochs",
             json=true)
public double epoch_counter

training_rows

@Request.API(help="Number of rows in training data",
             json=true)
public long training_rows

validation_rows

@Request.API(help="Number of rows in validation data",
             json=true)
public long validation_rows

actual_best_model_key

@Request.API(help="Key to the best model so far (based on overall error on scoring data set)")
public Key actual_best_model_key

_timeLastScoreEnter

public long _timeLastScoreEnter

Constructor Detail

DeepLearningModel

public DeepLearningModel(DeepLearningModel cp,
                 Key destKey,
                 Key jobKey,
                 FrameTask.DataInfo dataInfo)

Constructor to restart from a checkpointed model

Parameters:

cp - Checkpoint to restart from

destKey - New destination key for the model

jobKey - New job key (job which updates the model)

DeepLearningModel

public DeepLearningModel(Key destKey,
                 Key jobKey,
                 Key dataKey,
                 FrameTask.DataInfo dinfo,
                 DeepLearning params,
                 float[] priorDist)

Method Detail

model_info

public final DeepLearningModel.DeepLearningModelInfo model_info()

scoring_history

public DeepLearningModel.Errors[] scoring_history()

get_params

public final DeepLearning get_params()

Overrides:

get_params in class Model

job

public final Request2 job()

Overrides:

job in class Model

missingColumnsType

protected double missingColumnsType()

Description copied from class: Model

Type of missing columns during adaptation between train/test datasets Overload this method for models that have sparse data handling. Otherwise, NaN is used.

Overrides:

missingColumnsType in class Model

Returns:

real-valued number (can be NaN)

error

public float error()

isClassifier

public boolean isClassifier()

Overrides:

isClassifier in class Model

isSupervised

public boolean isSupervised()

Overrides:

isSupervised in class Model

nfeatures

public int nfeatures()

Description copied from class: Model

Returns number of input features

Overrides:

nfeatures in class Model

compareTo

public int compareTo(DeepLearningModel o)

Specified by:

compareTo in interface java.lang.Comparable<DeepLearningModel>

cm

public ConfusionMatrix cm()

for grid search error reporting

Overrides:

cm in class Model

mse

public double mse()

Description copied from class: Model

Returns mse for validation set.

Overrides:

mse in class Model

varimp

public VarImp varimp()

Description copied from class: Model

Variable importance of individual input features measured by this model.

Overrides:

varimp in class Model

setCrossValidationError

protected void setCrossValidationError(Job.ValidatedJob job,
                           double cv_error,
                           ConfusionMatrix cm,
                           AUCData auc,
                           HitRatio hr)

Overrides:

setCrossValidationError in class Model

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

toStringAll

public java.lang.String toStringAll()

getHeader

public java.lang.String getHeader()

score

public Frame score(Frame frame)

This is an overridden version of Model.score(). Make either a prediction or a reconstruction.

Overrides:

score in class Model

Parameters:

frame - Test dataset

Returns:

A frame containing the prediction or reconstruction

See Also:

Model.score(Frame, boolean)

score0

public float[] score0(double[] data,
             float[] preds)

Predict from raw double values representing the data

Specified by:

score0 in class Model

Parameters:

data - raw array containing categorical values (horizontalized to 1,0,0,1,0,0 etc.) and numerical values (0.35,1.24,5.3234,etc), both can contain NaNs

preds - predicted label and per-class probabilities (for classification), predicted target (regression), can contain NaNs

Returns:

preds, can contain NaNs

scoreAutoEncoder

public Frame scoreAutoEncoder(Frame frame)

Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))

Parameters:

frame - Original data (can contain response, will be ignored)

Returns:

Frame containing one Vec with reconstruction error (MSE) of each reconstructed row, caller is responsible for deletion

scoreDeepFeatures

public Frame scoreDeepFeatures(Frame frame,
                      int layer)

Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))

Parameters:

frame - Original data (can contain response, will be ignored)

Returns:

Frame containing one Vec with reconstruction error (MSE) of each reconstructed row, caller is responsible for deletion

calcOutlierThreshold

public double calcOutlierThreshold(Vec mse,
                          double quantile)

Compute quantile-based threshold (in reconstruction error) to find outliers

Parameters:

mse - Vector containing reconstruction errors

quantile - Quantile for cut-off

Returns:

Threshold in MSE value for a point to be above the quantile

getModelSerializer

public Model.ModelAutobufferSerializer getModelSerializer()

Description copied from class: Model

Returns a model serializer into AutoBuffer.

Overrides:

getModelSerializer in class Model

generateHTML

public boolean generateHTML(java.lang.String title,
                   java.lang.StringBuilder sb)

toJavaNCLASSES

protected SB toJavaNCLASSES(SB sb)

Overrides:

toJavaNCLASSES in class Model

toJavaFillPreds0

protected void toJavaFillPreds0(SB bodySb)

Description copied from class: Model

Fill preds[0] based on already filled and unified preds[1,..NCLASSES].

Overrides:

toJavaFillPreds0 in class Model

toJavaHtml

public void toJavaHtml(java.lang.StringBuilder sb)

toJavaInit

protected SB toJavaInit(SB sb,
            SB fileContextSB)

Overrides:

toJavaInit in class Model

toJavaPredictBody

protected void toJavaPredictBody(SB bodySb,
                     SB classCtxSb,
                     SB fileCtxSb)

Overrides:

toJavaPredictBody in class Model

delete_best_model

public void delete_best_model()

delete_xval_models

public void delete_xval_models()