hex.deeplearning

Class DeepLearningModel

java.lang.Object

water.Iced<T>

water.Keyed<T>

water.Lockable<M>

hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

hex.deeplearning.DeepLearningModel

All Implemented Interfaces:

hex.Model.DeepFeatures, java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable<DeepLearningModel>

public class DeepLearningModel
extends hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>
implements hex.Model.DeepFeatures

The Deep Learning model It contains a DeepLearningModelInfo with the most up-to-date model, a scoring history, as well as some helpers to indicate the progress

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DeepLearningModel.DeepLearningModelInfo
static class	DeepLearningModel.DeepLearningModelOutput
static class	DeepLearningModel.DeepLearningParameters
static class	DeepLearningModel.DeepLearningScoring

Nested classes/interfaces inherited from class hex.Model

hex.Model.DeepFeatures, hex.Model.Output, hex.Model.Parameters

Nested classes/interfaces inherited from class water.Keyed

water.Keyed.BinarySerializer<X extends water.Keyed>

Field Summary

Fields

Modifier and Type	Field and Description
long	_timeLastScoreEnter
water.Key	actual_best_model_key
long	actual_train_samples_per_iteration
double	epoch_counter
long	run_time
double	time_for_communication_us
long	training_rows
long	validation_rows

Fields inherited from class hex.Model

_output, _parms, _warnings

Fields inherited from class water.Lockable

_lockers

Fields inherited from class water.Keyed

_key, EMPTY_KEY_LIST

Constructor Summary

Constructors

Constructor and Description
DeepLearningModel(water.Key destKey, DeepLearningModel.DeepLearningParameters parms, DeepLearningModel.DeepLearningModelOutput output, water.fvec.Frame train, water.fvec.Frame valid)
DeepLearningModel(water.Key destKey, DeepLearningModel.DeepLearningParameters parms, DeepLearningModel cp, boolean store_best_model, DataInfo dataInfo) Constructor to restart from a checkpointed model

Method Summary

Methods

Modifier and Type	Method and Description
double	calcOutlierThreshold(water.fvec.Vec mse, double quantile) Compute quantile-based threshold (in reconstruction error) to find outliers
protected long	checksum_impl()
hex.ConfusionMatrix	cm()
int	compareTo(DeepLearningModel o)
void	delete()
float	error()
DeepLearningModel.DeepLearningParameters	get_params() Get the parameters actually used for model building, not the user-given ones (_parms) They might differ since some defaults are filled in, and some invalid combinations are auto-disabled in modifyParams
double	logloss()
hex.ModelMetrics.MetricBuilder	makeMetricBuilder(java.lang.String[] domain)
DeepLearningModel.DeepLearningModelInfo	model_info()
static void	modifyParms(DeepLearningModel.DeepLearningParameters fromParms, DeepLearningModel.DeepLearningParameters toParms, boolean classification) Take user-given parameters and turn them into usable, fully populated parameters (e.g., to be used by Neurons during training)
double	mse()
water.api.ModelSchema	schema()
water.fvec.Frame	score(water.fvec.Frame fr, java.lang.String destination_key)
double[]	score0(double[] data, double[] preds) Predict from raw double values representing the data
water.fvec.Frame	scoreAutoEncoder(water.fvec.Frame frame, water.Key destination_key) Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))
water.fvec.Frame	scoreDeepFeatures(water.fvec.Frame frame, int layer) Score auto-encoded reconstruction (on-the-fly, and materialize the deep features of given layer
protected water.fvec.Frame	scoreImpl(water.fvec.Frame orig, water.fvec.Frame adaptedFr, java.lang.String destination_key) Make either a prediction or a reconstruction.
DeepLearningModel.DeepLearningScoring[]	scoring_history()
protected boolean	toJavaCheckTooBig()
protected water.util.SB	toJavaInit(water.util.SB sb, water.util.SB fileContextSB)
protected void	toJavaPredictBody(water.util.SB bodySb, water.util.SB classCtxSb, water.util.SB fileCtxSb)
java.lang.String	toString()
hex.VarImp	varImp()

Methods inherited from class hex.Model

adaptTestForTrain, adaptTestForTrain, addMetrics, addWarning, cleanup_adapt, defaultThreshold, getPublishedKeys, isSupervised, remove_impl, score, score, score0, score0, score0, testJavaScoring, toJava, toJava, toJavaNCLASSES, toJavaPROB, toJavaSuper

Methods inherited from class water.Lockable

delete_and_lock, delete, delete, read_lock, read_lock, unlock_all, unlock, update, write_lock

Methods inherited from class water.Keyed

checksum, getBinarySerializer, remove, remove, remove, remove

Methods inherited from class water.Iced

clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

run_time

public long run_time

actual_train_samples_per_iteration

public long actual_train_samples_per_iteration

time_for_communication_us

public double time_for_communication_us

epoch_counter

public double epoch_counter

training_rows

public long training_rows

validation_rows

public long validation_rows

actual_best_model_key

public water.Key actual_best_model_key

_timeLastScoreEnter

public long _timeLastScoreEnter

Constructor Detail

DeepLearningModel

public DeepLearningModel(water.Key destKey,
                 DeepLearningModel.DeepLearningParameters parms,
                 DeepLearningModel cp,
                 boolean store_best_model,
                 DataInfo dataInfo)

Constructor to restart from a checkpointed model

Parameters:

destKey - New destination key for the model

parms - User-given parameters for checkpoint restart

cp - Checkpoint to restart from

store_best_model - Store only the best model instead of the latest one

DeepLearningModel

public DeepLearningModel(water.Key destKey,
                 DeepLearningModel.DeepLearningParameters parms,
                 DeepLearningModel.DeepLearningModelOutput output,
                 water.fvec.Frame train,
                 water.fvec.Frame valid)

Method Detail

schema

public water.api.ModelSchema schema()

model_info

public final DeepLearningModel.DeepLearningModelInfo model_info()

varImp

public final hex.VarImp varImp()

scoring_history

public DeepLearningModel.DeepLearningScoring[] scoring_history()

get_params

public final DeepLearningModel.DeepLearningParameters get_params()

Get the parameters actually used for model building, not the user-given ones (_parms) They might differ since some defaults are filled in, and some invalid combinations are auto-disabled in modifyParams

Returns:

actually used parameters

error

public float error()

makeMetricBuilder

public hex.ModelMetrics.MetricBuilder makeMetricBuilder(java.lang.String[] domain)

Specified by:

makeMetricBuilder in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

compareTo

public int compareTo(DeepLearningModel o)

cm

public hex.ConfusionMatrix cm()

mse

public double mse()

logloss

public double logloss()

modifyParms

public static void modifyParms(DeepLearningModel.DeepLearningParameters fromParms,
               DeepLearningModel.DeepLearningParameters toParms,
               boolean classification)

Take user-given parameters and turn them into usable, fully populated parameters (e.g., to be used by Neurons during training)

Parameters:

fromParms - raw user-given parameters from the REST API

toParms - modified set of parameters, with defaults filled in

classification -

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

scoreImpl

protected water.fvec.Frame scoreImpl(water.fvec.Frame orig,
                         water.fvec.Frame adaptedFr,
                         java.lang.String destination_key)

Make either a prediction or a reconstruction.

Overrides:

scoreImpl in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

Parameters:

orig - Test dataset

adaptedFr - Test dataset, adapted to the model

Returns:

A frame containing the prediction or reconstruction

score0

public double[] score0(double[] data,
              double[] preds)

Predict from raw double values representing the data

Specified by:

score0 in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

Parameters:

data - raw array containing categorical values (horizontalized to 1,0,0,1,0,0 etc.) and numerical values (0.35,1.24,5.3234,etc), both can contain NaNs

preds - predicted label and per-class probabilities (for classification), predicted target (regression), can contain NaNs

Returns:

preds, can contain NaNs

scoreAutoEncoder

public water.fvec.Frame scoreAutoEncoder(water.fvec.Frame frame,
                                water.Key destination_key)

Score auto-encoded reconstruction (on-the-fly, without allocating the reconstruction as done in Frame score(Frame fr))

Specified by:

scoreAutoEncoder in interface hex.Model.DeepFeatures

Parameters:

frame - Original data (can contain response, will be ignored)

Returns:

Frame containing one Vec with reconstruction error (MSE) of each reconstructed row, caller is responsible for deletion

score

public water.fvec.Frame score(water.fvec.Frame fr,
                     java.lang.String destination_key)

Overrides:

score in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

scoreDeepFeatures

public water.fvec.Frame scoreDeepFeatures(water.fvec.Frame frame,
                                 int layer)

Score auto-encoded reconstruction (on-the-fly, and materialize the deep features of given layer

Specified by:

scoreDeepFeatures in interface hex.Model.DeepFeatures

Parameters:

frame - Original data (can contain response, will be ignored)

layer - index of the hidden layer for which to extract the features

Returns:

Frame containing the deep features (#cols = hidden[layer])

calcOutlierThreshold

public double calcOutlierThreshold(water.fvec.Vec mse,
                          double quantile)

Compute quantile-based threshold (in reconstruction error) to find outliers

Parameters:

mse - Vector containing reconstruction errors

quantile - Quantile for cut-off

Returns:

Threshold in MSE value for a point to be above the quantile

delete

public void delete()

Overrides:

delete in class water.Lockable<DeepLearningModel>

toJavaInit

protected water.util.SB toJavaInit(water.util.SB sb,
                       water.util.SB fileContextSB)

Overrides:

toJavaInit in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

toJavaCheckTooBig

protected boolean toJavaCheckTooBig()

Overrides:

toJavaCheckTooBig in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

toJavaPredictBody

protected void toJavaPredictBody(water.util.SB bodySb,
                     water.util.SB classCtxSb,
                     water.util.SB fileCtxSb)

Overrides:

toJavaPredictBody in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>

checksum_impl

protected long checksum_impl()

Overrides:

checksum_impl in class hex.Model<DeepLearningModel,DeepLearningModel.DeepLearningParameters,DeepLearningModel.DeepLearningModelOutput>