ModelBuilder (h2o-core version 3.12.0.1 API)

java.lang.Object
- water.Iced
- - hex.ModelBuilder<M,P,O>

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable

Direct Known Subclasses:

ClusteringModelBuilder, Quantile
```
public abstract class ModelBuilder<M extends Model<M,P,O>,P extends Model.Parameters,O extends Model.Output>
extends Iced
```
Model builder parent class. Contains the common interfaces and fields across all model builders.

See Also:
Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`ModelBuilder.BuilderVisibility` Visibility for this algo: is it always visible, is it beta (always visible but with a note in the UI) or is it experimental (hidden by default, visible in the UI if the user gives an "experimental" flag at startup); test-only builders are "experimental"
`protected class`	`ModelBuilder.Driver`
`class`	`ModelBuilder.FilterCols`
`static class`	`ModelBuilder.ValidationMessage` Can be an ERROR, meaning the parameters can't be used as-is, a TRACE, which means the specified field should be hidden given the values of other fields, or a WARN or INFO for informative messages to the user.

Field Summary

Fields
Modifier and Type	Field and Description
`protected Vec`	`_fold`
`Job<M>`	`_job`
`ModelBuilder.ValidationMessage[]`	`_messages` A list of field validation issues.
`protected int`	`_nclass`
`protected Vec`	`_offset`
`protected java.lang.String[][]`	`_origDomains`
`protected java.lang.String[]`	`_origNames`
`P`	`_parms` All the parameters required to build the model.
`protected double[]`	`_priorClassDist`
`java.util.HashSet<java.lang.String>`	`_removedCols`
`protected Vec`	`_response`
`protected Key<M>`	`_result`
`protected Frame`	`_train`
`protected Frame`	`_valid`
`protected Vec`	`_vresponse`
`protected Vec`	`_weights`

Constructor Summary

Constructors
Modifier	Constructor and Description
`protected`	`ModelBuilder(P parms)` Default easy constructor: Unique new job and unique new result key
`protected`	`ModelBuilder(P parms, boolean startup_once)` One-time start-up only ModelBuilder, endlessly cloned by the GUI for the default settings.
`protected`	`ModelBuilder(P parms, boolean startup_once, java.lang.String externalSchemaDirectory)`
`protected`	`ModelBuilder(P parms, Job<M> job)` Shared pre-existing Job and unique new result key
`protected`	`ModelBuilder(P parms, Key<M> key)` Unique new job and named result key

Method Summary

Methods
Modifier and Type	Method and Description
`static java.lang.String`	`algoName(java.lang.String urlName)` gbm -> GBM, deeplearning -> DeepLearning
`static java.lang.String[]`	`algos()`
`ModelBuilder.BuilderVisibility`	`builderVisibility()`
`abstract hex.ModelCategory[]`	`can_build()` List containing the categories of models that this builder can build.
`void`	`checkDistributions()`
`protected void`	`checkMemoryFootPrint()` Override this method to call error() if the model is expected to not fit in memory, and say why
`void`	`clearInitState()` Clear whatever was done by init() so it can be run again.
`void`	`clearValidationErrors()`
`void`	`computeCrossValidation()` Default naive (serial) implementation of N-fold cross-validation (builds N+1 models, all have train+validation metrics, the main model has N-fold cross-validated validation metrics)
`protected boolean`	`computePriorClassDistribution()`
`Vec`	`cv_AssignFold(int N)`
`void`	`cv_buildModels(int N, ModelBuilder<M,P,O>[] cvModelBuilders)`
`void`	`cv_computeAndSetOptimalParameters(ModelBuilder<M,P,O>[] cvModelBuilders)` Override for model-specific checks / modifications to _parms for the main model during N-fold cross-validation.
`void`	`cv_mainModelScores(int N, ModelMetrics.MetricBuilder[] mbs, ModelBuilder<M,P,O>[] cvModelBuilders)`
`ModelBuilder<M,P,O>[]`	`cv_makeFramesAndBuilders(int N, Vec[] weights)`
`Vec[]`	`cv_makeWeights(int N, Vec foldAssignment)`
`ModelMetrics.MetricBuilder[]`	`cv_scoreCVModels(int N, Vec[] weights, ModelBuilder<M,P,O>[] cvModelBuilders)`
`static <S extends Model> Key<S>`	`defaultKey(java.lang.String algoName)` Default model-builder key
`protected int`	`desiredChunks(Frame original_fr, boolean local)` Find desired number of chunks.
`Key<M>`	`dest()`
`int`	`error_count()`
`void`	`error(java.lang.String field_name, java.lang.String message)`
`M`	`get()` Block till completion, and return the built model from the DKV.
`ToEigenVec`	`getToEigenVec()`
`boolean`	`hasFoldCol()`
`boolean`	`hasOffsetCol()`
`boolean`	`hasWeightCol()`
`boolean`	`haveMojo()`
`boolean`	`havePojo()`
`void`	`hide(java.lang.String field_name, java.lang.String message)`
`protected void`	`ignoreBadColumns(int npredictors, boolean expensive)` Ignore constant columns, columns with all NAs and strings.
`protected boolean`	`ignoreConstColumns()`
`protected void`	`ignoreInvalidColumns(int npredictors, boolean expensive)` Ignore invalid columns (columns that have a very high max value, which can cause issues in DHistogram)
`protected boolean`	`ignoreStringColumns()`
`void`	`info(java.lang.String field_name, java.lang.String message)`
`protected Frame`	`init_adaptFrameToTrain(Frame fr, java.lang.String frDesc, java.lang.String field, boolean expensive)` Adapts a given frame to the same schema as the training frame.
`void`	`init(boolean expensive)` Initialize the ModelBuilder, validating all arguments and preparing the training frame.
`boolean`	`isClassifier()`
`boolean`	`isStopped()`
`abstract boolean`	`isSupervised()`
`static java.lang.String`	`javaName(java.lang.String urlName)` gbm -> hex.tree.gbm.GBM, deeplearning -> hex.deeplearning.DeepLearning
`protected boolean`	`logMe()`
`static <B extends ModelBuilder> B`	`make(java.lang.String algo, Job job, Key<Model> result)` Factory method to create a ModelBuilder instance for given the algo name.
`void`	`message(byte log_level, java.lang.String field_name, java.lang.String message)`
`int`	`nclasses()`
`boolean`	`nFoldCV()`
`protected int`	`nFoldWork()`
`protected int`	`nModelsInParallel()` How many should be trained in parallel during N-fold cross-validation? Train all CV models in parallel when parallelism is enabled, otherwise train one at a time Each model can override this logic, based on parameters, dataset size, etc.
`int`	`numSpecialCols()`
`static java.lang.String`	`paramName(java.lang.String urlName)` gbm -> GBMParameters
`protected Frame`	`rebalance(Frame original_fr, boolean local, java.lang.String name)` Rebalance a frame for load balancing
`Vec`	`response()` Train response vector.
`static java.lang.String`	`schemaDirectory(java.lang.String urlName)` gbm -> "hex.schemas." ; custAlgo -> "org.myOrg.schemas."
`int`	`separateFeatureVecs()` Find and set response/weights/offset/fold and put them all in the end,
`void`	`setTrain(Frame train)`
`boolean`	`shouldReorder(Vec v)`
`java.lang.String[]`	`specialColNames()`
`protected boolean`	`stop_requested()`
`protected boolean`	`timeout()`
`Frame`	`train()` Training frame: derived from the parameter's training frame, excluding all ignored columns, all constant and bad columns, perhaps flipping the response column to an Categorical, etc.
`Job<M>`	`trainModel()` Method to launch training of a Model, based on its parameters.
`protected abstract ModelBuilder.Driver`	`trainModelImpl()` Model-specific implementation of model training
`M`	`trainModelNested(Frame fr)` Train a model as part of a larger Job;
`protected Frame`	`valid()` Validation frame: derived from the parameter's validation frame, excluding all ignored columns, all constant and bad columns, perhaps flipping the response column to a Categorical, etc.
`java.lang.String`	`validationErrors()` Get a string representation of only the ERROR ValidationMessages (e.g., to use in an exception throw).
`Vec`	`vresponse()` Validation response vector.
`void`	`warn(java.lang.String field_name, java.lang.String message)`

Methods inherited from class water.Iced
asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - _job
```
public Job<M extends Model<M,P,O>> _job
```
  - _result
```
protected Key<M extends Model<M,P,O>> _result
```
  - _parms
```
public P extends Model.Parameters _parms
```
    All the parameters required to build the model.
  - _train
```
protected transient Frame _train
```
  - _valid
```
protected transient Frame _valid
```
  - _response
```
protected transient Vec _response
```
  - _vresponse
```
protected transient Vec _vresponse
```
  - _offset
```
protected transient Vec _offset
```
  - _weights
```
protected transient Vec _weights
```
  - _fold
```
protected transient Vec _fold
```
  - _origNames
```
protected transient java.lang.String[] _origNames
```
  - _origDomains
```
protected transient java.lang.String[][] _origDomains
```
  - _nclass
```
protected int _nclass
```
  - _priorClassDist
```
protected transient double[] _priorClassDist
```
  - _messages
```
public ModelBuilder.ValidationMessage[] _messages
```
    A list of field validation issues.
  - _removedCols
```
public transient java.util.HashSet<java.lang.String> _removedCols
```
- Constructor Detail
  - ModelBuilder
```
protected ModelBuilder(P parms)
```
    Default easy constructor: Unique new job and unique new result key
  - ModelBuilder
```
protected ModelBuilder(P parms,
            Key<M> key)
```
    Unique new job and named result key
  - ModelBuilder
```
protected ModelBuilder(P parms,
            Job<M> job)
```
    Shared pre-existing Job and unique new result key
  - ModelBuilder
```
protected ModelBuilder(P parms,
            boolean startup_once)
```
    One-time start-up only ModelBuilder, endlessly cloned by the GUI for the default settings.
  - ModelBuilder
```
protected ModelBuilder(P parms,
            boolean startup_once,
            java.lang.String externalSchemaDirectory)
```
- Method Detail
  - getToEigenVec
```
public ToEigenVec getToEigenVec()
```
  - shouldReorder
```
public boolean shouldReorder(Vec v)
```
  - get
```
public final M get()
```
    Block till completion, and return the built model from the DKV. Note the funny assert: the Job does NOT have to be controlling this model build, but might, e.g. be controlling a Grid search for which this is just one of many results. Calling 'get' means that we are blocking on the Job which is controlling ONLY this ModelBuilder, and when the Job completes we can return built Model.
  - isStopped
```
public final boolean isStopped()
```
  - dest
```
public final Key<M> dest()
```
  - timeout
```
protected boolean timeout()
```
  - stop_requested
```
protected boolean stop_requested()
```
  - defaultKey
```
public static <S extends Model> Key<S> defaultKey(java.lang.String algoName)
```
    Default model-builder key
  - algos
```
public static java.lang.String[] algos()
```
  - algoName
```
public static java.lang.String algoName(java.lang.String urlName)
```
    gbm -> GBM, deeplearning -> DeepLearning
  - javaName
```
public static java.lang.String javaName(java.lang.String urlName)
```
    gbm -> hex.tree.gbm.GBM, deeplearning -> hex.deeplearning.DeepLearning
  - paramName
```
public static java.lang.String paramName(java.lang.String urlName)
```
    gbm -> GBMParameters
  - schemaDirectory
```
public static java.lang.String schemaDirectory(java.lang.String urlName)
```
    gbm -> "hex.schemas." ; custAlgo -> "org.myOrg.schemas."
  - make
```
public static <B extends ModelBuilder> B make(java.lang.String algo,
                              Job job,
                              Key<Model> result)
```
    Factory method to create a ModelBuilder instance for given the algo name. Shallow clone of both the default ModelBuilder instance and a Parameter.
  - train
```
public final Frame train()
```
    Training frame: derived from the parameter's training frame, excluding all ignored columns, all constant and bad columns, perhaps flipping the response column to an Categorical, etc.
  - setTrain
```
public void setTrain(Frame train)
```
  - valid
```
protected final Frame valid()
```
    Validation frame: derived from the parameter's validation frame, excluding all ignored columns, all constant and bad columns, perhaps flipping the response column to a Categorical, etc. Is null if no validation key is set.
  - response
```
public Vec response()
```
    Train response vector.
  - vresponse
```
public Vec vresponse()
```
    Validation response vector.
  - trainModel
```
public final Job<M> trainModel()
```
    Method to launch training of a Model, based on its parameters.
  - trainModelNested
```
public final M trainModelNested(Frame fr)
```
    Train a model as part of a larger Job;
    
    Parameters:
    fr: - Input frame override, ignored if null. In some cases, algos do not work directly with the original frame in the K/V store. Instead they run on a private anonymous copy (eg: reblanced dataset). Use this argument if you want nested job to work on the actual working copy rather than the original Frame in the K/V. Example: Outer job rebalances dataset and then calls nested job. To avoid needless second reblance, pass in the (already rebalanced) working copy.
  - trainModelImpl
```
protected abstract ModelBuilder.Driver trainModelImpl()
```
    Model-specific implementation of model training
    
    Returns:
    A F/J Job, which, when executed, does the build. F/J is NOT started.
  - nModelsInParallel
```
protected int nModelsInParallel()
```
    How many should be trained in parallel during N-fold cross-validation? Train all CV models in parallel when parallelism is enabled, otherwise train one at a time Each model can override this logic, based on parameters, dataset size, etc.
    
    Returns:
    How many models to train in parallel during cross-validation
  - nFoldWork
```
protected int nFoldWork()
```
  - computeCrossValidation
```
public void computeCrossValidation()
```
    Default naive (serial) implementation of N-fold cross-validation (builds N+1 models, all have train+validation metrics, the main model has N-fold cross-validated validation metrics)
  - cv_AssignFold
```
public Vec cv_AssignFold(int N)
```
  - cv_makeWeights
```
public Vec[] cv_makeWeights(int N,
                   Vec foldAssignment)
```
  - cv_makeFramesAndBuilders
```
public ModelBuilder<M,P,O>[] cv_makeFramesAndBuilders(int N,
                                             Vec[] weights)
```
  - cv_buildModels
```
public void cv_buildModels(int N,
                  ModelBuilder<M,P,O>[] cvModelBuilders)
```
  - cv_scoreCVModels
```
public ModelMetrics.MetricBuilder[] cv_scoreCVModels(int N,
                                            Vec[] weights,
                                            ModelBuilder<M,P,O>[] cvModelBuilders)
```
  - cv_mainModelScores
```
public void cv_mainModelScores(int N,
                      ModelMetrics.MetricBuilder[] mbs,
                      ModelBuilder<M,P,O>[] cvModelBuilders)
```
  - cv_computeAndSetOptimalParameters
```
public void cv_computeAndSetOptimalParameters(ModelBuilder<M,P,O>[] cvModelBuilders)
```
    Override for model-specific checks / modifications to _parms for the main model during N-fold cross-validation. Also allow the cv models to be modified after all of them have been built. For example, the model might need to be told to not do early stopping. CV models might have their lambda value modified, etc.
  - nFoldCV
```
public boolean nFoldCV()
```
    Returns:
    Whether n-fold cross-validation is done
  - can_build
```
public abstract hex.ModelCategory[] can_build()
```
    List containing the categories of models that this builder can build. Each ModelBuilder must have one of these.
  - builderVisibility
```
public ModelBuilder.BuilderVisibility builderVisibility()
```
  - clearInitState
```
public void clearInitState()
```
    Clear whatever was done by init() so it can be run again.
  - logMe
```
protected boolean logMe()
```
  - isSupervised
```
public abstract boolean isSupervised()
```
  - hasOffsetCol
```
public boolean hasOffsetCol()
```
  - hasWeightCol
```
public boolean hasWeightCol()
```
  - hasFoldCol
```
public boolean hasFoldCol()
```
  - numSpecialCols
```
public int numSpecialCols()
```
  - specialColNames
```
public java.lang.String[] specialColNames()
```
  - havePojo
```
public boolean havePojo()
```
  - haveMojo
```
public boolean haveMojo()
```
  - nclasses
```
public int nclasses()
```
  - isClassifier
```
public final boolean isClassifier()
```
  - separateFeatureVecs
```
public int separateFeatureVecs()
```
    Find and set response/weights/offset/fold and put them all in the end,
    
    Returns:
    number of non-feature vecs
  - ignoreStringColumns
```
protected boolean ignoreStringColumns()
```
  - ignoreConstColumns
```
protected boolean ignoreConstColumns()
```
  - ignoreBadColumns
```
protected void ignoreBadColumns(int npredictors,
                    boolean expensive)
```
    Ignore constant columns, columns with all NAs and strings.
    
    Parameters:
    npredictors -
    expensive -
  - ignoreInvalidColumns
```
protected void ignoreInvalidColumns(int npredictors,
                        boolean expensive)
```
    Ignore invalid columns (columns that have a very high max value, which can cause issues in DHistogram)
    
    Parameters:
    npredictors -
    expensive -
  - checkMemoryFootPrint
```
protected void checkMemoryFootPrint()
```
    Override this method to call error() if the model is expected to not fit in memory, and say why
  - computePriorClassDistribution
```
protected boolean computePriorClassDistribution()
```
  - error_count
```
public int error_count()
```
  - hide
```
public void hide(java.lang.String field_name,
        java.lang.String message)
```
  - info
```
public void info(java.lang.String field_name,
        java.lang.String message)
```
  - warn
```
public void warn(java.lang.String field_name,
        java.lang.String message)
```
  - error
```
public void error(java.lang.String field_name,
         java.lang.String message)
```
  - clearValidationErrors
```
public void clearValidationErrors()
```
  - message
```
public void message(byte log_level,
           java.lang.String field_name,
           java.lang.String message)
```
  - validationErrors
```
public java.lang.String validationErrors()
```
    Get a string representation of only the ERROR ValidationMessages (e.g., to use in an exception throw).
  - init
```
public void init(boolean expensive)
```
    Initialize the ModelBuilder, validating all arguments and preparing the training frame. This call is expected to be overridden in the subclasses and each subclass will start with "super.init();". This call is made by the front-end whenever the GUI is clicked, and needs to be fast whenever expensive is false; it will be called once again at the start of model building trainModel() with expensive set to true.
    The incoming training frame (and validation frame) will have ignored columns dropped out, plus whatever work the parent init did.
    NOTE: The front end initially calls this through the parameters validation endpoint with no training_frame, so each subclass's init() method has to work correctly with the training_frame missing.
  - init_adaptFrameToTrain
```
protected Frame init_adaptFrameToTrain(Frame fr,
                           java.lang.String frDesc,
                           java.lang.String field,
                           boolean expensive)
```
    Adapts a given frame to the same schema as the training frame. This includes encoding of categorical variables (if expensive is enabled). Note: This method should only be used during ModelBuilder initialization - it should be called in init(..) method.
    
    Parameters:
    fr - input frame
    frDesc - frame description, eg. "Validation Frame" - will be shown in validation error messages
    field - name of a field for validation errors
    expensive - indicates full ("expensive") processing
    
    Returns:
    adapted frame
  - rebalance
```
protected Frame rebalance(Frame original_fr,
              boolean local,
              java.lang.String name)
```
    Rebalance a frame for load balancing
    
    Parameters:
    original_fr - Input frame
    local - Whether to only create enough chunks to max out all cores on one node only
    name - Name of rebalanced frame
    
    Returns:
    Frame that has potentially more chunks
  - desiredChunks
```
protected int desiredChunks(Frame original_fr,
                boolean local)
```
    Find desired number of chunks. If fewer, dataset will be rebalanced.
    
    Returns:
    Lower bound on number of chunks after rebalancing.
  - checkDistributions
```
public void checkDistributions()
```

Class ModelBuilder<M extends Model<M,P,O>,P extends Model.Parameters,O extends Model.Output>

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class water.Iced

Methods inherited from class java.lang.Object

Field Detail

_job

_result

_parms

_train

_valid

_response

_vresponse

_offset

_weights

_fold

_origNames

_origDomains

_nclass

_priorClassDist

_messages

_removedCols

Constructor Detail

ModelBuilder

ModelBuilder

ModelBuilder

ModelBuilder

ModelBuilder

Method Detail

getToEigenVec

shouldReorder

get

isStopped

dest

timeout

stop_requested

defaultKey

algos

algoName

javaName

paramName

schemaDirectory

make

train

setTrain

valid

response

vresponse

trainModel

trainModelNested

trainModelImpl

nModelsInParallel

nFoldWork

computeCrossValidation

cv_AssignFold

cv_makeWeights

cv_makeFramesAndBuilders

cv_buildModels

cv_scoreCVModels

cv_mainModelScores

cv_computeAndSetOptimalParameters

nFoldCV

can_build

builderVisibility

clearInitState

logMe

isSupervised

hasOffsetCol

hasWeightCol

hasFoldCol

numSpecialCols

specialColNames

havePojo

haveMojo

nclasses

isClassifier

separateFeatureVecs

ignoreStringColumns