Isolation Forest

Introduction

Isolation Forest is similar in principle to Random Forest and is built on the basis of decision trees. Isolation Forest, however, identifies anomalies or outliers rather than profiling normal data points. Isolation Forest isolates observations by randomly selecting a feature and then randomly selecting a split value between the maximum and minimum values of that selected feature. This split depends on how long it takes to separate the points.

Random partitioning produces noticeably shorter paths for anomalies. When a forest of random trees collectively produces shorter path lengths for particular samples, they are highly likely to be anomalies.

Tutorials and Blogs

The following tutorials are available that describe how to use Isolation Forest to find anomalies in a dataset and how to interpret the results.

The Anomaly Detection with Isolation Forests using H2O blog provides a summary and examples of the Isolation Forest algorithm in H2O.

Defining an Isolation Forest Model

  • model_id: (Optional) Specify a custom name for the model to use as a reference. By default, H2O automatically generates a destination key.

  • training_frame: (Required) Specify the dataset used to build the model. NOTE: In Flow, if you click the Build a model button from the Parse cell, the training frame is entered automatically.

  • x: Specify a vector containing the names or indices of the predictor variables to use when building the model. If x is missing, then all columns except y are used.

  • score_each_iteration: (Optional) Enable this option to score during each iteration of the model training (disabled by default).

  • score_tree_interval: Score the model after every so many trees. This value is set to 0 (disabled) by default.

  • ignored_columns: (Optional, Python and Flow only) Specify the column or columns to be excluded from the model. In Flow, click the checkbox next to a column name to add it to the list of columns excluded from the model. To add all columns, click the All button. To remove a column from the list of ignored columns, click the X next to the column name. To remove all columns from the list of ignored columns, click the None button. To search for a specific column, type the column name in the Search field above the column list. To only show columns with a specific percentage of missing values, specify the percentage in the Only show columns with more than 0% missing values field. To change the selections for the hidden columns, use the Select Visible or Deselect Visible buttons.

  • ignore_const_cols: Specify whether to ignore constant training columns, since no information can be gained from them. This option is enabled by default.

  • ntrees: Specify the number of trees. This values defaults to 50.

  • max_depth: Specify the maximum tree depth. Higher values will make the model more complex and can lead to overfitting. Setting this value to 0 specifies no limit. This value defaults to 8.

  • min_rows: Specify the minimum number of observations for a leaf (nodesize in R). This value defaults to 1.

  • max_runtime_secs: Maximum allowed runtime in seconds for model training. This value is set to 0 (disabeld) by default.

  • seed: Specify the random number generator (RNG) seed for algorithm components dependent on randomization. The seed is consistent for each H2O instance so that you can create models with the same starting conditions in alternative configurations. This value defaults to -1 (time-based random number).

  • build_tree_one_node: Specify whether to run on a single node. This is suitable for small datasets as there is no network overhead but fewer CPUs are used. This value is disabled by default.

  • mtries: Specify the columns to randomly select at each level. If the default value of -1 is used, the number of variables is the square root of the number of columns for classification and p/3 for regression (where p is the number of predictors). If -2 is specified, all features of IF are used. Valid values for this option are -2, -1, and any value >= 1.

  • sample_size: The number of randomly sampled observations used to train each Isolation Forest tree. Only one of sample_size or sample_rate should be defined. If sample_rate is defined, sample_size will be ignored. This value defaults to 256.

  • sample_rate: Specify the row sampling rate (x-axis). (Note that this method is sample without replacement.) The range is 0.0 to 1.0. Higher values may improve training accuracy. Test accuracy improves when either columns or rows are sampled. For details, refer to “Stochastic Gradient Boosting” (Friedman, 1999). If set to -1 (default), then sample_size will be used instead.

  • col_sample_rate_change_per_level: This option specifies to change the column sampling rate as a function of the depth in the tree. This can be a value > 0.0 and <= 2.0 and defaults to 1. (Note that this method is sample without replacement.) For example:

    level 1: col_sample_rate

    level 2: col_sample_rate * factor

    level 3: col_sample_rate * factor^2

    level 4: col_sample_rate * factor^3

    etc.

  • col_sample_rate_per_tree: Specify the column sample rate per tree. This can be a value from 0.0 to 1.0 and defaults to 1. Note that this method is sample without replacement.

  • categorical_encoding: Specify one of the following encoding schemes for handling categorical features:

    • auto or AUTO: Allow the algorithm to decide (default). In Isolation Forest, the algorithm will automatically perform enum encoding.

    • enum or Enum: 1 column per categorical feature

    • enum_limited or EnumLimited: Automatically reduce categorical levels to the most prevalent ones during training and only keep the T (10) most frequent levels.

    • one_hot_explicit or OneHotExplicit: N+1 new columns for categorical features with N levels

    • binary or Binary: No more than 32 columns per categorical feature

    • eigen or Eigen: k columns per categorical feature, keeping projections of one-hot-encoded matrix onto k-dim eigen space only

    • label_encoder or LabelEncoder: Convert every enum into the integer of its index (for example, level 0 -> 0, level 1 -> 1, etc.)

  • stopping_rounds: Stops training when the option selected for stopping_metric doesn’t improve for the specified number of training rounds, based on a simple moving average. This value is set to 0 (disabled) by default. The metric is computed on the validation data (if provided); otherwise, training data is used.

    Note: If cross-validation is enabled:

    • All cross-validation models stop training when the validation metric doesn’t improve.

    • The main model runs for the mean number of epochs.

    • N+1 models may be off by the number specified for stopping_rounds from the best model, but the cross-validation metric estimates the performance of the main model for the resulting number of epochs (which may be fewer than the specified number of epochs).

  • stopping_metric: Specify the metric to use for early stopping. The available options are:

    • AUTO: This defaults to logloss for classification, deviance for regression, and anomaly_score for Isolation Forest. Note that custom and custom_increasing can only be used in GBM and DRF with the Python client. Must be one of: AUTO, anomaly_score. Defaults to AUTO.

    • anomaly_score (Isolation Forest only)

    • deviance

    • logloss

    • MSE

    • RMSE

    • MAE

    • RMSLE

    • AUC (area under the ROC curve)

    • AUCPR (area under the Precision-Recall curve)

    • lift_top_group

    • misclassification

    • mean_per_class_error

    • custom (Python client only)

    • custom_increasing (Python client only)

  • stopping_tolerance: Specify the relative tolerance for the metric-based stopping to stop training if the improvement is less than this value. This value defaults to 0.01.

  • export_checkpoints_dir: Specify a directory to which generated models will automatically be exported.

  • contamination: The contamination ratio is the proportion of anomalies in the input dataset. If undefined (-1), the predict function will not mark observations as anomalies and only anomaly score will be returned. Defaults to -1.

Anomaly Score

The output of Isolation Forest’s algorithm depends on the contamination parameter.

With contamination parameter:

Predict:

  • 1 = Anomaly

  • 0 = Normal point

A point is marked as an anomaly if the score is greater or equal to (1-contamination)% quantile of the score.

\[predict = score >= Q_{score}(1-contamination)\]

Score: the normalized mean_length.

\[score(mean\_length) = \frac{(max\_path\_length - mean\_length)}{(max\_path\_length - min\_path\_length)}\]

Where \(min\_path\_length\) and \(max\_path\_length\) are assigned in training. It can happen that an anomalous point has a value > 1. A higher value means a “more anomalous“ point. The score is not normalized by the average path of an unsuccessful search in a binary search tree (BST).

Mean_Length: mean path length of the point in a forest.

We are not using the formula (Equation (2)) from the Isolation Forest paper nor the estimation of the average path length of an unsuccessful search (Equation (2)).

\[mean\_length = \frac{path\_length}{ntrees}\]

Without contamination parameter:

The predict column contains values from the score column, and the mean_length column is not changed.

Examples

Below is a simple example showing how to build an Isolation Forest model.

R

Python

library(h2o)
h2o.init()

# Import the prostate dataset
prostate <- h2o.importFile(path = "https://raw.github.com/h2oai/h2o/master/smalldata/logreg/prostate.csv")

# Split dataset giving the training dataset 75% of the data
prostate_split <- h2o.splitFrame(data = prostate, ratios = 0.75)

# Create a training set from the 1st dataset in the split
train <- prostate_split[[1]]

# Create a testing set from the 2nd dataset in the split
test <- prostate_split[[2]]

# Build an Isolation forest model
model <- h2o.isolationForest(training_frame = train,
                             sample_rate = 0.1,
                             max_depth = 20,
                             ntrees = 50)

# Calculate score
score <- h2o.predict(model, test)
result_pred <- score$predict

# Predict the leaf node assignment
ln_pred <- h2o.predict_leaf_node_assignment(model, test)

import h2o
from h2o.estimators import H2OIsolationForestEstimator
h2o.init()

# Import the prostate dataset
h2o_df = h2o.import_file("https://raw.github.com/h2oai/h2o/master/smalldata/logreg/prostate.csv")

# Split the data giving the training dataset 75% of the data
train,test = h2o_df.split_frame(ratios=[0.75])

# Build an Isolation forest model
model = H2OIsolationForestEstimator(sample_rate = 0.1,
                                    max_depth = 20,
                                    ntrees = 50)
model.train(training_frame=train)

# Calculate score
score = model.predict(test)
result_pred = score["predict"]

# Predict the leaf node assignment
ln_pred = model.predict_leaf_node_assignment(test, "Path")