Metrics in H2O

Metrics In H2O

class h2o.model.metrics_base.H2OAutoEncoderModelMetrics(metric_json, on=None, algo='')[source]

Bases: h2o.model.metrics_base.MetricsBase

class h2o.model.metrics_base.H2OBinomialModelMetrics(metric_json, on=None, algo='')[source]

Bases: h2o.model.metrics_base.MetricsBase

This class is essentially an API for the AUC object. This class contains methods for inspecting the AUC for different criteria. To input the different criteria, use the static variable criteria

F0point5(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The F0point5 for this set of metrics and thresholds.
F1(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The F1 for the given set of thresholds.
F2(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The F2 for this set of metrics and thresholds
accuracy(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The accuracy for this set of metrics and thresholds
confusion_matrix(metrics=None, thresholds=None)[source]

Get the confusion matrix for the specified metric

Parameters:
  • metrics – A string (or list of strings) in {“min_per_class_accuracy”, “absolute_MCC”, “tnr”, “fnr”, “fpr”, “tpr”, “precision”, “accuracy”, “f0point5”, “f2”, “f1”}
  • thresholds – A value (or list of values) between 0 and 1
Returns:

a list of ConfusionMatrix objects (if there are more than one to return), or a single ConfusionMatrix (if there is only one)
error(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The error for this set of metrics and thresholds.
fallout(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The fallout or False Positive Rate for this set of metrics and thresholds
find_idx_by_threshold(threshold)[source]

Retrieve the index in this metric’s threshold list at which the given threshold is located.

Parameters:threshold – Find the index of this input threshold.
Returns:Return the index or throw a ValueError if no such index can be found.
find_threshold_by_max_metric(metric)[source]
Parameters:metric – A string in {“min_per_class_accuracy”, “absolute_MCC”, “precision”, “accuracy”, “f0point5”, “f2”, “f1”}
Returns:the threshold at which the given metric is maximum.
fnr(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The False Negative Rate
fpr(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The False Positive Rate
fprs None[source]

Return all false positive rates for all threshold values.

Returns:a list of false positive rates.
max_per_class_error(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:Return 1 - min_per_class_accuracy
mcc(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The absolute MCC (a value between 0 and 1, 0 being totally dissimilar, 1 being identical)
metric(metric, thresholds=None)[source]
Parameters:
  • metric – The desired metric
  • thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:

The set of metrics for the list of thresholds
missrate(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:THe missrate or False Negative Rate.
plot(type='roc', **kwargs)[source]

Produce the desired metric plot :param type: the type of metric plot (currently, only ROC supported) :param show: if False, the plot is not shown. matplotlib show method is blocking. :return: None

precision(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The precision for this set of metrics and thresholds.
recall(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:Recall for this set of metrics and thresholds
sensitivity(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:Sensitivity or True Positive Rate for this set of metrics and thresholds
specificity(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The specificity or True Negative Rate.
tnr(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The True Negative Rate
tpr(thresholds=None)[source]
Parameters:thresholds – thresholds parameter must be a list (i.e. [0.01, 0.5, 0.99]). If None, then the thresholds in this set of metrics will be used.
Returns:The True Postive Rate
tprs None[source]

Return all true positive rates for all threshold values.

Returns:a list of true positive rates.
class h2o.model.metrics_base.H2OClusteringModelMetrics(metric_json, on=None, algo='')[source]

Bases: h2o.model.metrics_base.MetricsBase

betweenss()[source]
Returns:the Between Cluster Sum-of-Square Error, or None if not present.
tot_withinss()[source]
Returns:the Total Within Cluster Sum-of-Square Error, or None if not present.
totss()[source]
Returns:the Total Sum-of-Square Error to Grand Mean, or None if not present.
class h2o.model.metrics_base.H2ODimReductionModelMetrics(metric_json, on=None, algo='')[source]

Bases: h2o.model.metrics_base.MetricsBase

cat_err()[source]
Returns:the Number of Misclassified categories over non-missing categorical entries, or None if not present.
num_err()[source]
Returns:the Sum of Squared Error over non-missing numeric entries, or None if not present.
class h2o.model.metrics_base.H2OMultinomialModelMetrics(metric_json, on=None, algo='')[source]

Bases: h2o.model.metrics_base.MetricsBase

confusion_matrix()[source]

Returns a confusion matrix based of H2O’s default prediction threshold for a dataset

hit_ratio_table()[source]

Retrieve the Hit Ratios

class h2o.model.metrics_base.H2ORegressionModelMetrics(metric_json, on=None, algo='')[source]

Bases: h2o.model.metrics_base.MetricsBase

This class provides an API for inspecting the metrics returned by a regression model.

It is possible to retrieve the R^2 (1 - MSE/variance) and MSE

class h2o.model.metrics_base.MetricsBase(metric_json, on=None, algo='')[source]

Bases: object

A parent class to house common metrics available for the various Metrics types.

The methods here are available across different model categories, and so appear here.

aic()[source]
Returns:Retrieve the AIC for this set of metrics.
auc()[source]
Returns:Retrieve the AUC for this set of metrics.
giniCoef()[source]
Returns:Retrieve the Gini coefficeint for this set of metrics.
logloss()[source]
Returns:Retrieve the log loss for this set of metrics.
mean_residual_deviance()[source]
Returns:Retrieve the mean residual deviance for this set of metrics.
mse()[source]
Returns:Retrieve the MSE for this set of metrics
null_degrees_of_freedom()[source]
Returns:the null dof if the model has residual deviance, or None if no null dof.
null_deviance()[source]
Returns:the null deviance if the model has residual deviance, or None if no null deviance.
r2()[source]
Returns:Retrieve the R^2 coefficient for this set of metrics
residual_degrees_of_freedom()[source]
Returns:the residual dof if the model has residual deviance, or None if no residual dof.
residual_deviance()[source]
Returns:the residual deviance if the model has residual deviance, or None if no residual deviance.
show()[source]

Display a short summary of the metrics. :return: None