H2O Module¶

The H2O Python Module¶

This module provides access to the H2O JVM, as well as its extensions, objects, machine-learning algorithms, and modeling support capabilities, such as basic munging and feature generation.

The H2O JVM uses a web server so that all communication occurs on a socket (specified by an IP address and a port) via a series of REST calls (see connection.py for the REST layer implementation and details). There is a single active connection to the H2O JVM at any time, and this handle is stashed out of sight in a singleton instance of H2OConnection (this is the global __H2OConn__). In other words, this package does not rely on Jython, and there is no direct manipulation of the JVM.

The H2O python module is not intended as a replacement for other popular machine learning frameworks such as scikit-learn, pylearn2, and their ilk, but is intended to bring H2O to a wider audience of data and machine learning devotees who work exclusively with Python.

H2O from Python is a tool for rapidly turning over models, doing data munging, and building applications in a fast, scalable environment without any of the mental anguish about parallelism and distribution of work.

>>> df = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate data
>>>
>>> colmeans = [v.mean() for v in df]                            # compute column means
>>>
>>> colmeans                                                     # print the results
[5.843333333333335, 3.0540000000000007, 3.7586666666666693, 1.1986666666666672]

>>> df = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate data
>>>
>>> vol = df['VOL']                                              # select the VOL column
>>>
>>> vol[vol == 0] = None                                         # 0 VOL means 'missing'

>>> fr = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate data
>>>
>>> a = fr + 3.14159                                             # "a" is now an Expr
>>>
>>> type(a)                                                      # <class 'h2o.expr.Expr'>

>>> fr = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate data
>>>
>>> h2o.remove(fr)                                               # remove prostate data
>>> fr                                                           # attempting to use fr results in a ValueError

>>> fr = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate data
>>>
>>> fr[1] = fr[1].asfactor()                                     # make 2nd column a factor
>>>
>>> m = h2o.glm(x=fr[3:], y=fr[2])                               # build a glm with a method call
>>>
>>> m.__class__                                                  # <h2o.model.binomial.H2OBinomialModel object at 0x104659cd0>
>>>
>>> m.show()                                                     # print the model details
>>>
>>> m.summary()                                                  # print a model summary

>>> fr = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate data
>>>
>>> fr[1].isfactor()                                             # produces False
>>>
>>> m = h2o.gbm(x=fr[2:],y=fr[1])                                # build the gbm
>>>
>>> m.__class__                                                  # <h2o.model.regression.H2ORegressionModel object at 0x104d07590>
>>>
>>> fr[1] = fr[1].asfactor()                                     # cast the 2nd column to a factor column
>>>
>>> fr[1].isfactor()                                             # produces True
>>>
>>> m = h2o.gbm(x=fr[2:],y=fr[1])                                # build the gbm
>>>
>>> m.__class__                                                  # <h2o.model.binomial.H2OBinomialModel object at 0x104d18f50>

>>> fr = h2o.import_frame(path="smalldata/logreg/prostate.csv")  # import prostate
>>>
>>> fr[1] = fr[1].asfactor()                                     # cast to factor
>>>
>>> r = fr[0].runif()                                            # Random UNIform numbers, one per row
>>>
>>> train = fr[ r < 0.6 ]                                        # 60% for training data
>>>
>>> valid = fr[ (0.6 <= r) & (r < 0.9) ]                         # 30% for validation
>>>
>>> test  = fr[ 0.9 <= r ]                                       # 10% for testing
>>>
>>> m = h2o.deeplearning(x=train[2:],y=train[1],validation_x=valid[2:],validation_y=valid[1])  # build a deeplearning with a validation set (yes it's this simple)
>>>
>>> m                                                            # display the model summary by default (can also call m.show())
>>>
>>> m.show()                                                     # equivalent to the above
>>>
>>> m.model_performance()                                        # show the performance on the training data, (can also be m.performance(train=True)
>>>
>>> m.model_performance(valid=True)                              # show the performance on the validation data
>>>
>>> m.model_performance(test_data=test)                          # score and compute new metrics on the test data!

>>> m.mse()           # MSE on the training data
>>>
>>> m.mse(valid=True) # MSE on the validation data
>>>
>>> m.r2()            # R^2 on the training data
>>>
>>> m.r2(valid=True)  # R^2 on the validation data
>>>
>>> m.confusion_matrix()  # confusion matrix for max F1
>>>
>>> m.confusion_matrix("tpr") # confusion matrix for max true positive rate
>>>
>>> m.confusion_matrix("max_per_class_error")   # etc.

>>> fr = h2o.import_frame(path="smalldata/iris/iris_wheader.csv")   # import iris
>>>
>>> r = fr[0].runif()                       # generate a random vector for splitting
>>>
>>> train = fr[ r < 0.6 ]                   # split out 60% for training
>>>
>>> valid = fr[ 0.6 <= r & r < 0.9 ]        # split out 30% for validation
>>>
>>> test = fr[ 0.9 <= r ]                   # split out 10% for testing
>>>
>>> my_model = h2o.glm(x=train[1:], y=train[0], validation_x=valid[1:], validation_y=valid[0])  # build a GLM
>>>
>>> my_model.coef()                         # print the GLM coefficients, can also perform my_model.coef_norm() to get the normalized coefficients
>>>
>>> my_model.null_deviance()                # get the null deviance from the training set metrics
>>>
>>> my_model.residual_deviance()            # get the residual deviance from the training set metrics
>>>
>>> my_model.null_deviance(valid=True)      # get the null deviance from the validation set metrics (similar for residual deviance)
>>>
>>> # now generate a new metrics object for the test hold-out data:
>>>
>>> my_metrics = my_model.model_performance(test_data=test) # create the new test set metrics
>>>
>>> my_metrics.null_degrees_of_freedom()    # returns the test null dof
>>>
>>> my_metrics.residual_deviance()          # returns the test res. deviance
>>>
>>> my_metrics.aic()                        # returns the test aic

import h2o
h2o.init(ip="192.168.1.10", port=54321)
--------------------------  ------------------------------------
H2O cluster uptime:         2 minutes 1 seconds 966 milliseconds
H2O cluster version:        0.1.27.1064
H2O cluster name:           H2O_96762
H2O cluster total nodes:    4
H2O cluster total memory:   38.34 GB
H2O cluster total cores:    16
H2O cluster allowed cores:  80
H2O cluster healthy:        True
--------------------------  ------------------------------------
pathDataTrain = ["hdfs://192.168.1.10/user/data/data_train.csv"]
pathDataTest = ["hdfs://192.168.1.10/user/data/data_test.csv"]
trainFrame = h2o.import_frame(path=pathDataTrain)
testFrame = h2o.import_frame(path=pathDataTest)

#Parse Progress: [##################################################] 100%
#Imported [hdfs://192.168.1.10/user/data/data_train.csv'] into cluster with 60000 rows and 500 cols

#Parse Progress: [##################################################] 100%
#Imported ['hdfs://192.168.1.10/user/data/data_test.csv'] into cluster with 10000 rows and 500 cols

trainFrame[499]._name = "label"
testFrame[499]._name = "label"

model = h2o.gbm(x=trainFrame.drop("label"),
            y=trainFrame["label"],
            validation_x=testFrame.drop("label"),
            validation_y=testFrame["label"],
            ntrees=100,
            max_depth=10
            )

#gbm Model Build Progress: [##################################################] 100%

predictFrame = model.predict(testFrame)
model.model_performance(testFrame)

h2o¶

This module provides all of the top level calls for models and various data transform methods. By simply

class h2o.h2o.H2ODisplay(table=None, header=None, table_header=None, **kwargs)[source]¶

Pretty printing for H2O Objects; Handles both IPython and vanilla console display

h2o.h2o.as_list(data, use_pandas=True)[source]¶

Convert an H2O data object into a python-specific object.

WARNING: This will pull all data local!

If Pandas is available (and use_pandas is True), then pandas will be used to parse the data frame. Otherwise, a list-of-lists populated by character data will be returned (so the types of data will all be str).

Parameters:	data – An H2O data object. use_pandas – Try to use pandas for reading in the data.
Returns:	List of list (Rows x Columns).

h2o.h2o.autoencoder(x, **kwargs)[source]¶

Build an Autoencoder

Parameters:	x – Columns with which to build an autoencoder kwargs – Additional arguments to pass to the autoencoder.
Returns:	A new autoencoder model

h2o.h2o.check_dims_values(python_obj, h2o_frame, rows, cols)[source]¶

Check that the dimensions and values of the python object and H2OFrame are equivalent. Assumes that the python object conforms to the rules specified in the h2o frame documentation.

Parameters:	python_obj – a (nested) list, tuple, dictionary, numpy.ndarray, ,or pandas.DataFrame h2o_frame – an H2OFrame rows – number of rows cols – number of columns
Returns:	None

h2o.h2o.check_models(model1, model2, use_cross_validation=False, op='e')[source]¶

Check that the given models are equivalent :param model1: :param model2: :param use_cross_validation: boolean. if True, use validation metrics to determine model equality. Otherwise, use training metrics. :param op: comparison operator to use. ‘e’:==, ‘g’:>, ‘ge’:>= :return: None. Throw meaningful error messages if the check fails

h2o.h2o.cluster_info()[source]¶

Display the current H2O cluster information.

Returns:	None

h2o.h2o.cluster_status()[source]¶

TODO: This isn’t really a cluster status... it’s a node status check for the node we’re connected to. This is possibly confusing because this can come back without warning, but if a user tries to do any remoteSend, they will get a “cloud sick warning”

Retrieve information on the status of the cluster running H2O. :return: None

h2o.h2o.create_frame(id=None, rows=10000, cols=10, randomize=True, value=0, real_range=100, categorical_fraction=0.2, factors=100, integer_fraction=0.2, integer_range=100, binary_fraction=0.1, binary_ones_fraction=0.02, missing_fraction=0.01, response_factors=2, has_response=False, seed=None)[source]¶

Data Frame Creation in H2O. Creates a data frame in H2O with real-valued, categorical, integer, and binary columns specified by the user.

Parameters:	id – A string indicating the destination key. If empty, this will be auto-generated by H2O. rows – The number of rows of data to generate. cols – The number of columns of data to generate. Excludes the response column if has_response == True. randomize – A logical value indicating whether data values should be randomly generated. This must be TRUE if

either categorical_fraction or integer_fraction is non-zero. :param value: If randomize == FALSE, then all real-valued entries will be set to this value. :param real_range: The range of randomly generated real values. :param categorical_fraction: The fraction of total columns that are categorical. :param factors: The number of (unique) factor levels in each categorical column. :param integer_fraction: The fraction of total columns that are integer-valued. :param integer_range: The range of randomly generated integer values. :param binary_fraction: The fraction of total columns that are binary-valued. :param binary_ones_fraction: The fraction of values in a binary column that are set to 1. :param missing_fraction: The fraction of total entries in the data frame that are set to NA. :param response_factors: If has_response == TRUE, then this is the number of factor levels in the response column. :param has_response: A logical value indicating whether an additional response column should be pre-pended to the final H2O data frame. If set to TRUE, the total number of columns will be cols+1. :param seed: A seed used to generate random values when randomize = TRUE. :return: the H2OFrame that was created

h2o.h2o.deeplearning(x, y=None, validation_x=None, validation_y=None, **kwargs)[source]¶

Build a supervised Deep Learning model Performs Deep Learning neural networks on an H2OFrame

Parameters:	x – An H2OFrame containing the predictors in the model. y – An H2OFrame of the response variable in the model. training_frame – (Optional) An H2OFrame. Only used to retrieve weights, offset, or nfolds columns, if they

aren’t already provided in x. :param model_id: (Optional) The unique id assigned to the resulting model. If none is given, an id will automatically be generated. :param overwrite_with_best_model: Logical. If True, overwrite the final model with the best model found during training. Defaults to True. :param validation_frame: (Optional) An H2OFrame object indicating the validation dataset used to construct the confusion matrix. If left blank, this defaults to the training data when nfolds = 0 :param checkpoint: “Model checkpoint (either key or H2ODeepLearningModel) to resume training with.” :param autoencoder: Enable auto-encoder for model building. :param use_all_factor_levels: Logical. Use all factor levels of categorical variance. Otherwise the first factor level is omitted (without loss of accuracy). Useful for variable importances and auto-enabled for autoencoder. :param activation: A string indicating the activation function to use. Must be either “Tanh”, “TanhWithDropout”, “Rectifier”, “RectifierWithDropout”, “Maxout”, or “MaxoutWithDropout” :param hidden: Hidden layer sizes (e.g. c(100,100)) :param epochs: How many times the dataset should be iterated (streamed), can be fractional :param train_samples_per_iteration: Number of training samples (globally) per MapReduce iteration. Special values are: 0 one epoch; -1 all available data (e.g., replicated training data); or -2 auto-tuning (default) :param seed: Seed for random numbers (affects sampling) - Note: only reproducible when running single threaded :param adaptive_rate: Logical. Adaptive learning rate (ADAELTA) :param rho: Adaptive learning rate time decay factor (similarity to prior updates) :param epsilon: Adaptive learning rate parameter, similar to learn rate annealing during initial training phase. Typical values are between 1.0e-10 and 1.0e-4 :param rate: Learning rate (higher => less stable, lower => slower convergence) :param rate_annealing: Learning rate annealing: eqn{(rate)/(1 + rate_annealing*samples) :param rate_decay: Learning rate decay factor between layers (N-th layer: eqn{rate*lpha^(N-1)) :param momentum_start: Initial momentum at the beginning of training (try 0.5) :param momentum_ramp: Number of training samples for which momentum increases :param momentum_stable: Final momentum after the amp is over (try 0.99) :param nesterov_accelerated_gradient: Logical. Use Nesterov accelerated gradient (recommended) :param input_dropout_ratio: A fraction of the features for each training row to be omitted from training in order to improve generalization (dimension sampling). :param hidden_dropout_ratios: Input layer dropout ratio (can improve generalization) specify one value per hidden layer, defaults to 0.5 :param l1: L1 regularization (can add stability and improve generalization, causes many weights to become 0) :param l2: L2 regularization (can add stability and improve generalization, causes many weights to be small) :param max_w2: Constraint for squared sum of incoming weights per unit (e.g. Rectifier) :param initial_weight_distribution: Can be “Uniform”, “UniformAdaptive”, or “Normal” :param initial_weight_scale: Uniform: -value ... value, Normal: stddev :param loss: Loss function: “Automatic”, “CrossEntropy” (for classification only), “MeanSquare”, “Absolute” (experimental) or “Huber” (experimental) :param distribution: A character string. The distribution function of the response. Must be “AUTO”, “bernoulli”, “multinomial”, “poisson”, “gamma”, “tweedie”, “laplace”, “huber” or “gaussian” :param tweedie_power: Tweedie power (only for Tweedie distribution, must be between 1 and 2) :param score_interval: Shortest time interval (in secs) between model scoring :param score_training_samples: Number of training set samples for scoring (0 for all) :param score_validation_samples: Number of validation set samples for scoring (0 for all) :param score_duty_cycle: Maximum duty cycle fraction for scoring (lower: more training, higher: more scoring) :param classification_stop: Stopping criterion for classification error fraction on training data (-1 to disable) :param regression_stop: Stopping criterion for regression error (MSE) on training data (-1 to disable) :param quiet_mode: Enable quiet mode for less output to standard output :param max_confusion_matrix_size: Max. size (number of classes) for confusion matrices to be shown :param max_hit_ratio_k: Max number (top K) of predictions to use for hit ratio computation(for multi-class only, 0 to disable) :param balance_classes: Balance training data class counts via over/under-sampling (for imbalanced data) :param class_sampling_factors: Desired over/under-sampling ratios per class (in lexicographic order). If not specified, sampling factors will be automatically computed to obtain class balance during training. Requires balance_classes. :param max_after_balance_size: Maximum relative size of the training data after balancing class counts (can be less than 1.0) :param score_validation_sampling: Method used to sample validation dataset for scoring :param diagnostics: Enable diagnostics for hidden layers :param variable_importances: Compute variable importances for input features (Gedeon method) - can be slow for large networks) :param fast_mode: Enable fast mode (minor approximations in back-propagation) :param ignore_const_cols: Ignore constant columns (no information can be gained anyway) :param force_load_balance: Force extra load balancing to increase training speed for small datasets (to keep all cores busy) :param replicate_training_data: Replicate the entire training dataset onto every node for faster training :param single_node_mode: Run on a single node for fine-tuning of model parameters :param shuffle_training_data: Enable shuffling of training data (recommended if training data is replicated and train_samples_per_iteration is close to eqn{numRows*numNodes :param sparse: Sparse data handling (Experimental) :param col_major: Use a column major weight matrix for input layer. Can speed up forward propagation, but might slow down backpropagation (Experimental) :param average_activation: Average activation for sparse auto-encoder (Experimental) :param sparsity_beta: Sparsity regularization (Experimental) :param max_categorical_features: Max. number of categorical features, enforced via hashing Experimental) :param reproducible: Force reproducibility on small data (will be slow - only uses 1 thread) :param export_weights_and_biases: Whether to export Neural Network weights and biases to H2O Frames” :param offset_column: Specify the offset column. :param weights_column: Specify the weights column. :param nfolds: (Optional) Number of folds for cross-validation. If nfolds >= 2, then validation must remain empty. :param fold_column: (Optional) Column with cross-validation fold index assignment per observation :param fold_assignment: Cross-validation fold assignment scheme, if fold_column is not specified Must be “AUTO”, “Random” or “Modulo” :param keep_cross_validation_predictions: Whether to keep the predictions of the cross-validation models :return: Return a new classifier or regression model.

h2o.h2o.download_all_logs(dirname='.', filename=None)[source]¶

Download H2O Log Files to Disk :param dirname: (Optional) A character string indicating the directory that the log file should be saved in. :param filename: (Optional) A string indicating the name that the CSV file should be :return: path of logs written (as a string)

h2o.h2o.download_csv(data, filename)[source]¶

Download an H2O data set to a CSV file on the local disk. Warning: Files located on the H2O server may be very large! Make sure you have enough hard drive space to accommodate the entire file.

Parameters:	data – an H2OFrame object to be downloaded.

:param filename:A string indicating the name that the CSV file should be should be saved to. :return: None

h2o.h2o.download_pojo(model, path='')[source]¶

Download the POJO for this model to the directory specified by path (no trailing slash!). If path is “”, then dump to screen.

Parameters:	model – Retrieve this model’s scoring POJO. path – An absolute path to the directory where POJO should be saved.
Returns:	None

h2o.h2o.export_file(frame, path, force=False)[source]¶

Export a given H2OFrame to a path on the machine this python session is currently connected to. To view the current session, call h2o.cluster_info().

Parameters:	frame – The Frame to save to disk. path – The path to the save point on disk. force – Overwrite any preexisting file with the same path
Returns:	None

h2o.h2o.frame(frame_id, exclude='')[source]¶

Retrieve metadata for a id that points to a Frame.

Parameters:	frame_id – A pointer to a Frame in H2O.
Returns:	Meta information on the frame

h2o.h2o.frames()[source]¶

Retrieve all the Frames.

Returns:	Meta information on the frames

h2o.h2o.gbm(x, y, validation_x=None, validation_y=None, **kwargs)[source]¶

Builds gradient boosted classification trees, and gradient boosted regression trees on a parsed data set. The default distribution function will guess the model type based on the response column typerun properly the response column must be an numeric for “gaussian” or an enum for “bernoulli” or “multinomial”.

Parameters:	x – An H2OFrame containing the predictors in the model. y – An H2OFrame of the response variable in the model. training_frame – (Optional) An H2OFrame. Only used to retrieve weights, offset, or nfolds columns, if they

aren’t already provided in x. :param model_id: (Optional) The unique id assigned to the resulting model. If none is given, an id will automatically be generated. :param distribution: A character string. The distribution function of the response. Must be “AUTO”, “bernoulli”, “multinomial”, “poisson”, “gamma”, “tweedie” or “gaussian” :param tweedie_power: Tweedie power (only for Tweedie distribution, must be between 1 and 2) :param ntrees: A nonnegative integer that determines the number of trees to grow. :param max_depth: Maximum depth to grow the tree. :param min_rows: Minimum number of rows to assign to teminal nodes. :param learn_rate: An integer from 0.0 to 1.0 :param nbins: For numerical columns (real/int), build a histogram of this many bins, then split at the best point :param nbins_cats: For categorical columns (enum), build a histogram of this many bins, then split at the best point. Higher values can lead to more overfitting. :param validation_frame: An H2OFrame object indicating the validation dataset used to contruct the confusion matrix. If left blank, this defaults to the training data when nfolds = 0 :param balance_classes: logical, indicates whether or not to balance training data class counts via over/under-sampling (for imbalanced data) :param max_after_balance_size: Maximum relative size of the training data after balancing class counts (can be less than 1.0) :param seed: Seed for random numbers (affects sampling when balance_classes=T) :param build_tree_one_node: Run on one node only; no network overhead but fewer cpus used. Suitable for small datasets. :param nfolds: (Optional) Number of folds for cross-validation. If nfolds >= 2, then validation must remain empty. :param fold_column: (Optional) Column with cross-validation fold index assignment per observation :param fold_assignment: Cross-validation fold assignment scheme, if fold_column is not specified Must be “AUTO”, “Random” or “Modulo” :param keep_cross_validation_predictions: Whether to keep the predictions of the cross-validation models :param score_each_iteration: Attempts to score each tree. :param offset_column: Specify the offset column. :param weights_column: Specify the weights column. :return: A new classifier or regression model.

h2o.h2o.get_frame(frame_id)[source]¶

Obtain a handle to the frame in H2O with the frame_id key.

Returns:	An H2OFrame

h2o.h2o.get_future_model(future_model)[source]¶

Waits for the future model to finish building, and then returns the model.

Parameters:	future_model – an H2OModelFuture object
Returns:	a resolved model (i.e. an H2OBinomialModel, H2ORegressionModel, H2OMultinomialModel, ...)

h2o.h2o.get_model(model_id)[source]¶

Return the specified model

Parameters:	model_id – The model identification in h2o

h2o.h2o.get_timezone()[source]¶

Get the Time Zone on the H2O Cloud

Returns:	the time zone (string)

h2o.h2o.glm(x, y, validation_x=None, validation_y=None, **kwargs)[source]¶

Build a Generalized Linear Model Fit a generalized linear model, specified by a response variable, a set of predictors, and a description of the error distribution.

Parameters:	x – An H2OFrame containing the predictors in the model. y – An H2OFrame of the response variable in the model. training_frame – (Optional) An H2OFrame. Only used to retrieve weights, offset, or nfolds columns, if they

aren’t already provided in x. :param model_id: (Optional) The unique id assigned to the resulting model. If none is given, an id will automatically be generated. :param validation_frame: An H2OFrame object containing the variables in the model. :param max_iterations: A non-negative integer specifying the maximum number of iterations. :param beta_epsilon: A non-negative number specifying the magnitude of the maximum difference between the coefficient estimates from successive iterations. Defines the convergence criterion for h2o.glm. :param solver: A character string specifying the solver used: IRLSM (supports more features), L_BFGS (scales better for datasets with many columns) :param standardize: A logical value indicating whether the numeric predictors should be standardized to have a mean of 0 and a variance of 1 prior to training the models. :param family: A character string specifying the distribution of the model: gaussian, binomial, poisson, gamma, tweedie. :param link: A character string specifying the link function. The default is the canonical link for the family. The supported links for each of the family specifications are:

“gaussian”: “identity”, “log”, “inverse” “binomial”: “logit”, “log” “poisson”: “log”, “identity” “gamma”: “inverse”, “log”, “identity” “tweedie”: “tweedie”

Parameters:	tweedie_variance_power – A numeric specifying the power for the variance function when family = “tweedie”. tweedie_link_power – A numeric specifying the power for the link function when family = “tweedie”. alpha – A numeric in [0, 1] specifying the elastic-net mixing parameter.

The elastic-net penalty is defined to be: eqn{P(lpha,eta) = (1-lpha)/2||eta||_2^2 + lpha||eta||_1 = sum_j [(1-lpha)/2 eta_j^2 + lpha|eta_j|], making alpha = 1 the lasso penalty and alpha = 0 the ridge penalty. :param lambda: A non-negative shrinkage parameter for the elastic-net, which multiplies eqn{P(lpha,eta) in the objective function. When lambda = 0, no elastic-net penalty is applied and ordinary generalized linear models are fit. :param prior: (Optional) A numeric specifying the prior probability of class 1 in the response when family = “binomial”. The default prior is the observational frequency of class 1. :param lambda_search: A logical value indicating whether to conduct a search over the space of lambda values starting from the lambda max, given lambda is interpreted as lambda min. :param nlambdas: The number of lambda values to use when lambda_search = TRUE. :param lambda_min_ratio: Smallest value for lambda as a fraction of lambda.max. By default if the number of observations is greater than the the number of variables then lambda_min_ratio = 0.0001; if the number of observations is less than the number of variables then lambda_min_ratio = 0.01. :param beta_constraints: A data.frame or H2OParsedData object with the columns [“names”, “lower_bounds”, “upper_bounds”, “beta_given”], where each row corresponds to a predictor in the GLM. “names” contains the predictor names, “lower”/”upper_bounds”, are the lower and upper bounds of beta, and “beta_given” is some supplied starting values for the :param offset_column: Specify the offset column. :param weights_column: Specify the weights column. :param nfolds: (Optional) Number of folds for cross-validation. If nfolds >= 2, then validation must remain empty. :param fold_column: (Optional) Column with cross-validation fold index assignment per observation :param fold_assignment: Cross-validation fold assignment scheme, if fold_column is not specified Must be “AUTO”, “Random” or “Modulo” :param keep_cross_validation_predictions: Whether to keep the predictions of the cross-validation models :param intercept: Logical, include constant term (intercept) in the model :return: A subclass of ModelBase is returned. The specific subclass depends on the machine learning task at hand (if it’s binomial classification, then an H2OBinomialModel is returned, if it’s regression then a H2ORegressionModel is returned). The default print-out of the models is shown, but further GLM-specifc information can be queried out of the object. Upon completion of the GLM, the resulting object has coefficients, normalized coefficients, residual/null deviance, aic, and a host of model metrics including MSE, AUC (for logistic regression), degrees of freedom, and confusion matrices.

h2o.h2o.ifelse(test, yes, no)[source]¶

Semantically equivalent to R’s ifelse. Based on the booleans in the test vector, the output has the values of the yes and no vectors interleaved (or merged together).

Parameters:	test – A “test” H2OFrame yes – A “yes” H2OFrame no – A “no” H2OFrame
Returns:	An H2OFrame

h2o.h2o.import_file(path)[source]¶

Import a single file or collection of files.

Parameters:	path – A path to a data file (remote or local).
Returns:	A new H2OFrame

h2o.h2o.import_frame(path=None)[source]¶

Import a frame from a file (remote or local machine). If you run H2O on Hadoop, you can access to HDFS

Parameters:	path – A path specifying the location of the data to import.
Returns:	A new H2OFrame

h2o.h2o.init(ip='localhost', port=54321, size=1, start_h2o=False, enable_assertions=False, license=None, max_mem_size_GB=None, min_mem_size_GB=None, ice_root=None, strict_version_check=False)[source]¶

Initiate an H2O connection to the specified ip and port.

Parameters:

ip – A string representing the hostname or IP address of the server where H2O is running. port – A port, default is 54321 size – THe expected number of h2o instances (ignored if start_h2o is True) start_h2o – A boolean dictating whether this module should start the H2O jvm. An attempt is made anyways if _connect fails. enable_assertions – If start_h2o, pass -ea as a VM option.s license – If not None, is a path to a license file. max_mem_size_GB – Maximum heap size (jvm option Xmx) in gigabytes. min_mem_size_GB – Minimum heap size (jvm option Xms) in gigabytes. ice_root – A temporary directory (default location is determined by tempfile.mkdtemp()) to hold H2O log files.

Returns:

None

h2o.h2o.interaction(data, factors, pairwise, max_factors, min_occurrence, destination_frame=None)[source]¶

Categorical Interaction Feature Creation in H2O. Creates a frame in H2O with n-th order interaction features between categorical columns, as specified by the user.

Parameters:	data – the H2OFrame that holds the target categorical columns. factors – factors Factor columns (either indices or column names). pairwise – Whether to create pairwise interactions between factors (otherwise create one

higher-order interaction). Only applicable if there are 3 or more factors. :param max_factors: Max. number of factor levels in pair-wise interaction terms (if enforced, one extra catch-all factor will be made) :param min_occurrence: Min. occurrence threshold for factor levels in pair-wise interaction terms :param destination_frame: A string indicating the destination key. If empty, this will be auto-generated by H2O. :return: H2OFrame

h2o.h2o.keys_leaked(num_keys)[source]¶

Ask H2O if any keys leaked. @param num_keys: The number of keys that should be there. :return: A boolean True/False if keys leaked. If keys leaked, check H2O logs for further detail.

h2o.h2o.kmeans(x, validation_x=None, **kwargs)[source]¶

Performs k-means clustering on an H2O dataset.

Parameters:	x – (Optional) A vector containing the data columns on which k-means operates. k – The number of clusters. Must be between 1 and 1e7 inclusive. k may be omitted if the user specifies the

initial centers in the init parameter. If k is not omitted, in this case, then it should be equal to the number of user-specified centers. :param model_id: (Optional) The unique id assigned to the resulting model. If none is given, an id will automatically be generated. :param max_iterations: The maximum number of iterations allowed. Must be between 0 and 1e6 inclusive. :param standardize: Logical, indicates whether the data should be standardized before running k-means. :param init: A character string that selects the initial set of k cluster centers. Possible values are “Random”: for random initialization, “PlusPlus”: for k-means plus initialization, or “Furthest”: for initialization at the furthest point from each successive center. Additionally, the user may specify a the initial centers as a matrix, data.frame, H2OFrame, or list of vectors. For matrices, data.frames, and H2OFrames, each row of the respective structure is an initial center. For lists of vectors, each vector is an initial center. :param seed: (Optional) Random seed used to initialize the cluster centroids. :param nfolds: (Optional) Number of folds for cross-validation. If nfolds >= 2, then validation must remain empty. :param fold_column: (Optional) Column with cross-validation fold index assignment per observation :param fold_assignment: Cross-validation fold assignment scheme, if fold_column is not specified Must be “AUTO”, “Random” or “Modulo” :return: Returns an object of class H2OClusteringModel.

h2o.h2o.list_timezones()[source]¶

Get a list of all the timezones

Returns:	the time zones (as an H2OFrame)

h2o.h2o.load_model(path)[source]¶

Load a saved H2O model from disk. :param path: The full path of the H2O Model to be imported. For example, if the dir argument in h2o.saveModel was set to “/Users/UserName/Desktop” then the path argument in h2o.loadModel should be set to something like “/Users/UserName/Desktop/K-meansModel__a7cebf318ca5827185e209edf47c4052” :return: the model

h2o.h2o.locate(path)[source]¶

Search for a relative path and turn it into an absolute path. This is handy when hunting for data files to be passed into h2o and used by import file. Note: This function is for unit testing purposes only.

Parameters:	path – Path to search for
Returns:	Absolute path if it is found. None otherwise.

h2o.h2o.log_and_echo(message)[source]¶

Log a message on the server-side logs This is helpful when running several pieces of work one after the other on a single H2O cluster and you want to make a notation in the H2O server side log where one piece of work ends and the next piece of work begins.

Sends a message to H2O for logging. Generally used for debugging purposes.

Parameters:	message – A character string with the message to write to the log.
Returns:	None

h2o.h2o.ls()[source]¶

List Keys on an H2O Cluster

Returns:	Returns a list of keys in the current H2O instance

h2o.h2o.naive_bayes(x, y, validation_x=None, validation_y=None, **kwargs)[source]¶

The naive Bayes classifier assumes independence between predictor variables conditional on the response, and a Gaussian distribution of numeric predictors with mean and standard deviation computed from the training dataset. When building a naive Bayes classifier, every row in the training dataset that contains at least one NA will be skipped completely. If the test dataset has missing values, then those predictors are omitted in the probability calculation during prediction.

Parameters:	laplace – A positive number controlling Laplace smoothing. The default zero disables smoothing. threshold – The minimum standard deviation to use for observations without enough data. Must be at least 1e-10. eps – A threshold cutoff to deal with numeric instability, must be positive. compute_metrics – A logical value indicating whether model metrics should be computed. Set to FALSE to reduce

the runtime of the algorithm. :return: Returns an H2OBinomialModel if the response has two categorical levels, H2OMultinomialModel otherwise.

h2o.h2o.np_comparison_check(h2o_data, np_data, num_elements)[source]¶

Check values achieved by h2o against values achieved by numpy

Parameters:	h2o_data – an H2OFrame or H2OVec np_data – a numpy array num_elements – number of elements to compare
Returns:	None

h2o.h2o.ou()[source]¶

Where is my baguette!?

Returns:	the name of the baguette. oh uhr uhr huhr

h2o.h2o.parse(setup, h2o_name, first_line_is_header=(-1, 0, 1))[source]¶

Trigger a parse; blocking; removeFrame just keep the Vecs.

Parameters:	setup – The result of calling parse_setup. h2o_name – The name of the H2O Frame on the back end. first_line_is_header – -1 means data, 0 means guess, 1 means header.
Returns:	A new parsed object

h2o.h2o.parse_raw(setup, id=None, first_line_is_header=(-1, 0, 1))[source]¶

Used in conjunction with import_file and parse_setup in order to make alterations before parsing.

Parameters:	setup – Result of h2o.parse_setup id – An optional id for the frame. first_line_is_header – -1,0,1 if the first line is to be used as the header
Returns:	An H2OFrame object

h2o.h2o.parse_setup(raw_frames)[source]¶

Parameters:	raw_frames – A collection of imported file frames
Returns:	A ParseSetup “object”

h2o.h2o.prcomp(x, validation_x=None, **kwargs)[source]¶

Principal components analysis of a H2O dataset using the power method to calculate the singular value decomposition of the Gram matrix.

Parameters:	k – The number of principal components to be computed. This must be between 1 and min(ncol(training_frame),

nrow(training_frame)) inclusive. :param model_id: (Optional) The unique hex key assigned to the resulting model. Automatically generated if none is provided. :param max_iterations: The maximum number of iterations to run each power iteration loop. Must be between 1 and 1e6 inclusive. :param transform: A character string that indicates how the training data should be transformed before running PCA. Possible values are “NONE”: for no transformation, “DEMEAN”: for subtracting the mean of each column, “DESCALE”: for dividing by the standard deviation of each column, “STANDARDIZE”: for demeaning and descaling, and “NORMALIZE”: for demeaning and dividing each column by its range (max - min). :param seed: (Optional) Random seed used to initialize the right singular vectors at the beginning of each power method iteration. :param use_all_factor_levels: (Optional) A logical value indicating whether all factor levels should be included in each categorical column expansion. If FALSE, the indicator column corresponding to the first factor level of every categorical variable will be dropped. Defaults to FALSE. :return: a new dim reduction model

h2o.h2o.random_forest(x, y, validation_x=None, validation_y=None, **kwargs)[source]¶

Build a Big Data Random Forest Model Builds a Random Forest Model on an H2OFrame

Parameters:	x – An H2OFrame containing the predictors in the model. y – An H2OFrame of the response variable in the model. training_frame – (Optional) An H2OFrame. Only used to retrieve weights, offset, or nfolds columns, if they

aren’t already provided in x. :param model_id: (Optional) The unique id assigned to the resulting model. If none is given, an id will automatically be generated. :param mtries: Number of variables randomly sampled as candidates at each split. If set to -1, defaults to sqrt{p for classification, and p/3 for regression, where p is the number of predictors. :param sample_rate: Sample rate, from 0 to 1.0. :param build_tree_one_node: Run on one node only; no network overhead but fewer cpus used. Suitable for small datasets. :param ntrees: A nonnegative integer that determines the number of trees to grow. :param max_depth: Maximum depth to grow the tree. :param min_rows: Minimum number of rows to assign to teminal nodes. :param nbins: For numerical columns (real/int), build a histogram of this many bins, then split at the best point. :param nbins_cats: For categorical columns (enum), build a histogram of this many bins, then split at the best point. Higher values can lead to more overfitting. :param binomial_double_trees: For binary classification: Build 2x as many trees (one per class) - can lead to higher accuracy. :param validation_frame: An H2OFrame object containing the variables in the model. :param balance_classes: logical, indicates whether or not to balance training data class counts via over/under-sampling (for imbalanced data) :param max_after_balance_size: Maximum relative size of the training data after balancing class counts (can be less than 1.0) :param seed: Seed for random numbers (affects sampling) - Note: only reproducible when running single threaded :param offset_column: Specify the offset column. :param weights_column: Specify the weights column. :param nfolds: (Optional) Number of folds for cross-validation. If nfolds >= 2, then validation must remain empty. :param fold_column: (Optional) Column with cross-validation fold index assignment per observation :param fold_assignment: Cross-validation fold assignment scheme, if fold_column is not specified Must be “AUTO”, “Random” or “Modulo” :param keep_cross_validation_predictions: Whether to keep the predictions of the cross-validation models :return: A new classifier or regression model.

h2o.h2o.rapids(expr, id=None)[source]¶

Fire off a Rapids expression.

Parameters:	expr – The rapids expression (ascii string).
Returns:	The JSON response of the Rapids execution

h2o.h2o.remove(object)[source]¶

Remove object from H2O. This is a “hard” delete of the object. It removes all subparts.

Parameters:	object – The object pointing to the object to be removed.
Returns:	None

h2o.h2o.removeFrameShallow(key)[source]¶

Do a shallow DKV remove of the frame (does not remove any internal Vecs). This is a “soft” delete. Just removes the top level pointer, but all big data remains!

Parameters:	key – A Frame Key to be removed
Returns:	None

h2o.h2o.remove_all()[source]¶

Remove all objects from H2O.

:return None

h2o.h2o.save_model(model, dir='', name='', filename='', force=False)[source]¶

Save an H2O Model Object to Disk. In the case of existing files force = TRUE will overwrite the file. Otherwise, the operation will fail. :param dir: string indicating the directory the model will be written to. :param name: string name of the file. :param filename: full path to the file. :param force: logical, indicates how to deal with files that already exist :return: the path of the model (string)

h2o.h2o.set_timezone(tz)[source]¶

Set the Time Zone on the H2O Cloud

Parameters:	tz – The desired timezone.
Returns:	None

h2o.h2o.shutdown(conn=None, prompt=True)[source]¶

Shut down the specified instance. All data will be lost. This method checks if H2O is running at the specified IP address and port, and if it is, shuts down that H2O instance.

Parameters:	conn – An H2OConnection object containing the IP address and port of the server running H2O. prompt – A logical value indicating whether to prompt the user before shutting down the H2O server.
Returns:	None

h2o.h2o.start_glm_job(x, y, validation_x=None, validation_y=None, **kwargs)[source]¶

Build a Generalized Linear Model Note: this function is the same as glm(), but it doesn’t block on model-build. Instead, it returns and H2OModelFuture object immediately. The model can be retrieved from the H2OModelFuture object with get_future_model().

Returns:	H2OModelFuture

h2o.h2o.store_size()[source]¶

Get the H2O store size (current count of keys). :return: number of keys in H2O cloud

h2o.h2o.svd(x, validation_x=None, **kwargs)[source]¶

Singular value decomposition of a H2O dataset using the power method.

Parameters:	nv – The number of right singular vectors to be computed. This must be between 1 and min(ncol(training_frame),

nrow(training_frame)) inclusive. :param max_iterations: The maximum number of iterations to run each power iteration loop. Must be between 1 and 1e6 inclusive.max_iterations The maximum number of iterations to run each power iteration loop. Must be between 1 and 1e6 inclusive. :param transform: A character string that indicates how the training data should be transformed before running PCA. Possible values are “NONE”: for no transformation, “DEMEAN”: for subtracting the mean of each column, “DESCALE”: for dividing by the standard deviation of each column, “STANDARDIZE”: for demeaning and descaling, and “NORMALIZE”: for demeaning and dividing each column by its range (max - min). :param seed: (Optional) Random seed used to initialize the right singular vectors at the beginning of each power method iteration. :param use_all_factor_levels: (Optional) A logical value indicating whether all factor levels should be included in each categorical column expansion. If FALSE, the indicator column corresponding to the first factor level of every categorical variable will be dropped. Defaults to TRUE. :return: a new dim reduction model

h2o.h2o.upload_file(path, destination_frame='')[source]¶

Upload a dataset at the path given from the local machine to the H2O cluster.

Parameters:	path – A path specifying the location of the data to upload. destination_frame – The name of the H2O Frame in the H2O Cluster.
Returns:	A new H2OFrame

h2o.h2o.which(condition)[source]¶

Parameters:	condition – A conditional statement.
Returns:	A H2OFrame of 1 column filled with 0-based indices for which the condition is True