hex

Class ScoringInfo

java.lang.Object

water.Iced

hex.ScoringInfo

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable

public class ScoringInfo
extends Iced

Lightweight scoring history snapshot, for things like displaying the scoring history.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ScoringInfo.HasEpochs
static interface	ScoringInfo.HasIterations
static interface	ScoringInfo.HasSamples

Field Summary

Fields

Modifier and Type	Field and Description
boolean	cross_validation
boolean	is_autoencoder
boolean	is_classification
ScoreKeeper	scored_train
ScoreKeeper	scored_valid
ScoreKeeper	scored_xval
long	this_scoring_time_ms
long	time_stamp_ms
long	total_scoring_time_ms
long	total_setup_time_ms
long	total_training_time_ms
AUC2	training_AUC
boolean	validation
AUC2	validation_AUC
VarImp	variable_importances

Constructor Summary

Constructors

Constructor and Description
ScoringInfo()

Method Summary

Methods

Modifier and Type	Method and Description
static java.util.Comparator<ScoringInfo>	comparator(ScoreKeeper.StoppingMetric criterion) Create a java.util.Comparator which allows us to sort an array of ScoringInfo based on a stopping criterion / metric.
static TwoDimTable	createScoringHistoryTable(ScoringInfo[] scoringInfos, boolean hasValidation, boolean hasCrossValidation, hex.ModelCategory modelCategory, boolean isAutoencoder) Create a TwoDimTable to display the scoring history from an array of scoringInfo.
ScoringInfo	deep_clone()
double	metric(ScoreKeeper.StoppingMetric criterion)
static ScoringInfo[]	prependScoringInfo(ScoringInfo scoringInfo, ScoringInfo[] scoringInfos) Add a new ScoringInfo to the given array and return the new array.
static ScoreKeeper[]	scoreKeepers(ScoringInfo[] scoring_history) For a given array of ScoringInfo return an array of the cross-validation, validation or training ScoreKeepers, as available.
static void	sort(ScoringInfo[] scoringInfos, ScoreKeeper.StoppingMetric criterion) Sort an array of ScoringInfo based on a stopping criterion / metric.

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

time_stamp_ms

public long time_stamp_ms

total_training_time_ms

public long total_training_time_ms

total_scoring_time_ms

public long total_scoring_time_ms

total_setup_time_ms

public long total_setup_time_ms

this_scoring_time_ms

public long this_scoring_time_ms

is_classification

public boolean is_classification

is_autoencoder

public boolean is_autoencoder

training_AUC

public AUC2 training_AUC

validation_AUC

public AUC2 validation_AUC

validation

public boolean validation

cross_validation

public boolean cross_validation

scored_train

public ScoreKeeper scored_train

scored_valid

public ScoreKeeper scored_valid

scored_xval

public ScoreKeeper scored_xval

variable_importances

public VarImp variable_importances

Constructor Detail

ScoringInfo

public ScoringInfo()

Method Detail

prependScoringInfo

public static ScoringInfo[] prependScoringInfo(ScoringInfo scoringInfo,
                               ScoringInfo[] scoringInfos)

Add a new ScoringInfo to the given array and return the new array. Note: this has no side effects.

Parameters:

scoringInfo -

scoreKeepers

public static ScoreKeeper[] scoreKeepers(ScoringInfo[] scoring_history)

For a given array of ScoringInfo return an array of the cross-validation, validation or training ScoreKeepers, as available.

metric

public double metric(ScoreKeeper.StoppingMetric criterion)

comparator

public static final java.util.Comparator<ScoringInfo> comparator(ScoreKeeper.StoppingMetric criterion)

Create a java.util.Comparator which allows us to sort an array of ScoringInfo based on a stopping criterion / metric. Uses cross-validation or validation metrics if available, otherwise falls back to training metrics. Understands whether more is better for the given criterion and will order the array so that the best models are last (to fit into the behavior of a model that improves over time)

Parameters:

criterion - scalar model metric / stopping criterion by which to sort

Returns:

a Comparator on a stopping criterion / metric

sort

public static void sort(ScoringInfo[] scoringInfos,
        ScoreKeeper.StoppingMetric criterion)

Sort an array of ScoringInfo based on a stopping criterion / metric. Uses cross-validation or validation metrics if available, otherwise falls back to training metrics. Understands whether more is better for the given criterion and will order the array so that the best models are last

Parameters:

scoringInfos - array of ScoringInfo to sort

criterion - scalar model metric / stopping criterion by which to sort

createScoringHistoryTable

public static TwoDimTable createScoringHistoryTable(ScoringInfo[] scoringInfos,
                                    boolean hasValidation,
                                    boolean hasCrossValidation,
                                    hex.ModelCategory modelCategory,
                                    boolean isAutoencoder)

Create a TwoDimTable to display the scoring history from an array of scoringInfo.

Parameters:

scoringInfos - array of ScoringInfo to render

hasValidation - do we have validation metrics?

hasCrossValidation - do we have cross-validation metrics?

modelCategory - the category for the model or models

isAutoencoder - is the model or are the models autoencoders?

Returns:

deep_clone

public ScoringInfo deep_clone()