hex

Class ModelMetrics

java.lang.Object

water.Iced<T>

water.Keyed<ModelMetrics>

hex.ModelMetrics

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable<ModelMetrics>

Direct Known Subclasses:

ModelMetricsSupervised, ModelMetricsUnsupervised

public class ModelMetrics
extends Keyed<ModelMetrics>

Container to hold the metric for a model as scored on a specific frame. The MetricBuilder class is used in a hot inner-loop of a Big Data pass, and when given a class-distribution, can be used to compute CM's, and AUC's "on the fly" during ModelBuilding - or after-the-fact with a Model and a new Frame to be scored.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ModelMetrics.MetricBuilder<T extends ModelMetrics.MetricBuilder<T>> Class used to compute AUCs, CMs & HRs "on the fly" during other passes over Big Data.

Field Summary

Fields

Modifier and Type	Field and Description
java.lang.String	_description
double	_MSE
long	_nobs
long	_scoring_time

Fields inherited from class water.Keyed

_key

Constructor Summary

Constructors

Constructor and Description
ModelMetrics(Model model, Frame frame, long nobs, double MSE, java.lang.String desc)

Method Summary

Methods

Modifier and Type	Method and Description
AUC2	auc_obj()
static Key<ModelMetrics>	buildKey(Model model, Frame frame)
static TwoDimTable	calcVarImp(double[] rel_imp, java.lang.String[] coef_names)
static TwoDimTable	calcVarImp(double[] rel_imp, java.lang.String[] coef_names, java.lang.String table_header, java.lang.String[] col_headers)
static TwoDimTable	calcVarImp(float[] rel_imp, java.lang.String[] coef_names)
static TwoDimTable	calcVarImp(VarImp vi)
protected long	checksum_impl() High-quality 64-bit checksum of the content of the object.
ConfusionMatrix	cm()
Frame	frame()
static java.util.Set<java.lang.String>	getAllowedMetrics(Key<Model> key)
static ModelMetrics	getFromDKV(Model model, Frame frame)
static double	getMetricFromModel(Key<Model> key, java.lang.String criterion)
float[]	hr()
boolean	isForFrame(Frame f)
boolean	isForModel(Model m)
Model	model()
double	mse()
long	residual_degrees_of_freedom()
double	rmse()
static java.util.List<Key<Model>>	sortModelsByMetric(java.lang.String sort_by, boolean decreasing, java.util.List<Key<Model>> modelKeys) Return a new list of models sorted by the named criterion, such as "auc", mse", "hr", "err", "err_count", "accuracy", "specificity", "recall", "precision", "mcc", "max_per_class_error", "f1", "f2", "f0point5".
java.lang.String	toString()

Methods inherited from class water.Keyed

checksum, makeSchema, readAll_impl, readAll, remove_impl, remove, remove, remove, remove, writeAll_impl, writeAll

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

_description

public java.lang.String _description

_scoring_time

public final long _scoring_time

_MSE

public final double _MSE

_nobs

public final long _nobs

Constructor Detail

ModelMetrics

public ModelMetrics(Model model,
            Frame frame,
            long nobs,
            double MSE,
            java.lang.String desc)

Method Detail

residual_degrees_of_freedom

public long residual_degrees_of_freedom()

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

model

public Model model()

frame

public Frame frame()

mse

public double mse()

rmse

public double rmse()

cm

public ConfusionMatrix cm()

hr

public float[] hr()

auc_obj

public AUC2 auc_obj()

getMetricFromModel

public static double getMetricFromModel(Key<Model> key,
                        java.lang.String criterion)

getAllowedMetrics

public static java.util.Set<java.lang.String> getAllowedMetrics(Key<Model> key)

sortModelsByMetric

public static java.util.List<Key<Model>> sortModelsByMetric(java.lang.String sort_by,
                                            boolean decreasing,
                                            java.util.List<Key<Model>> modelKeys)

Return a new list of models sorted by the named criterion, such as "auc", mse", "hr", "err", "err_count", "accuracy", "specificity", "recall", "precision", "mcc", "max_per_class_error", "f1", "f2", "f0point5". . .

Parameters:

sort_by - criterion by which we should sort

decreasing - sort by decreasing metrics or not

modelKeys - keys of models to sortm

Returns:

keys of the models, sorted by the criterion

calcVarImp

public static TwoDimTable calcVarImp(VarImp vi)

calcVarImp

public static TwoDimTable calcVarImp(float[] rel_imp,
                     java.lang.String[] coef_names)

calcVarImp

public static TwoDimTable calcVarImp(double[] rel_imp,
                     java.lang.String[] coef_names)

calcVarImp

public static TwoDimTable calcVarImp(double[] rel_imp,
                     java.lang.String[] coef_names,
                     java.lang.String table_header,
                     java.lang.String[] col_headers)

buildKey

public static Key<ModelMetrics> buildKey(Model model,
                         Frame frame)

isForModel

public boolean isForModel(Model m)

isForFrame

public boolean isForFrame(Frame f)

getFromDKV

public static ModelMetrics getFromDKV(Model model,
                      Frame frame)

checksum_impl

protected long checksum_impl()

Description copied from class: Keyed

High-quality 64-bit checksum of the content of the object. Similar to hashcode(), but a long to reduce the chance of hash clashes. For composite objects this should be defined using the subcomponents' checksums (or hashcodes if not available). If two Keyed objects have the same checksum() there should be a 1 - 1/2^64 chance that they are the same object by value.

Overrides:

checksum_impl in class Keyed<ModelMetrics>