hex.modelselection

Class ModelSelection

java.lang.Object

water.Iced

hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

hex.modelselection.ModelSelection

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable

public class ModelSelection
extends hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	ModelSelection.ModelSelectionDriver

Nested classes/interfaces inherited from class hex.ModelBuilder

hex.ModelBuilder.BuilderVisibility, hex.ModelBuilder.Driver, hex.ModelBuilder.FilterCols, hex.ModelBuilder.ModelTrainingCoordinator, hex.ModelBuilder.ValidationMessage

Field Summary

Fields

Modifier and Type	Field and Description
java.lang.String[][]	_bestModelPredictors
double[]	_bestR2Values
int	_glmNFolds
int	_numPredictors
java.lang.String[]	_predictorNames

Fields inherited from class hex.ModelBuilder

_coordinator, _desc, _eventPublisher, _fold, _input_parms, _job, _messages, _nclass, _offset, _orig_projection_array, _origDomains, _origNames, _origTrain, _parms, _priorClassDist, _removedCols, _response, _result, _startUpOnceModelBuilder, _train, _treatment, _valid, _vresponse, _weights

Constructor Summary

Constructors

Constructor and Description
ModelSelection(boolean startup_once)
ModelSelection(ModelSelectionModel.ModelSelectionParameters parms)
ModelSelection(ModelSelectionModel.ModelSelectionParameters parms, water.Key<ModelSelectionModel> key)

Method Summary

Modifier and Type	Method and Description
static GLMModel	buildExtractBestR2Model(water.fvec.Frame[] trainingFrames, ModelSelectionModel.ModelSelectionParameters parms, int glmNFolds, java.lang.String foldColumn, hex.Model.Parameters.FoldAssignmentScheme foldAssignment) Given the training Frame array, build models for each training frame and return the GLMModel with the best R2 values.
hex.ModelCategory[]	can_build()
static GLMModel	forwardStep(java.util.List<java.lang.Integer> currSubsetIndices, java.util.List<java.lang.String> coefNames, int predPos, java.util.List<java.lang.Integer> validSubsets, ModelSelectionModel.ModelSelectionParameters parms, java.lang.String foldColumn, int glmNFolds, hex.Model.Parameters.FoldAssignmentScheme foldAssignment) Given a predictor subset with indices stored in currSubsetIndices, one more predictor from the coefNames that was not found in currSubsetIndices was added to the subset to form a new Training frame.
boolean	haveMojo()
boolean	havePojo()
void	init(boolean expensive)
boolean	isSupervised()
protected int	nModelsInParallel(int folds)
static GLMModel	replacement(java.util.List<java.lang.Integer> currSubsetIndices, java.util.List<java.lang.String> coefNames, double bestR2, ModelSelectionModel.ModelSelectionParameters parms, int glmNFolds, java.lang.String foldColumn, java.util.List<java.lang.Integer> validSubset, hex.Model.Parameters.FoldAssignmentScheme foldAssignment) consider the predictors in subset as pred0, pred1, pred2 (using subset size 3 as example): a.
protected ModelSelection.ModelSelectionDriver	trainModelImpl()

Methods inherited from class hex.ModelBuilder

algoName, algos, builderVisibility, canLearnFromNAs, checkDistributions, checkEarlyStoppingReproducibility, checkMemoryFootPrint_impl, checkMemoryFootPrint, checkResponseVariable, clearInitState, clearValidationErrors, computeCrossValidation, computePriorClassDistribution, cv_buildModels, cv_canBuildMainModelInParallel, cv_computeAndSetOptimalParameters, cv_initStoppingParameters, cv_mainModelScores, cv_makeAggregateModelMetircs, cv_scoreCVModels, cv_updateOptimalParameters, defaultKey, desiredChunks, dest, error_count, error, get, getName, getSysProperty, getToEigenVec, hasFoldCol, hasOffsetCol, hasTreatmentCol, hasWeightCol, hide, ignoreBadColumns, ignoreConstColumns, ignoreInvalidColumns, ignoreStringColumns, ignoreUuidColumns, info, init_adaptFrameToTrain, init_getNClass, initWorkspace, isClassifier, isResponseOptional, isStopped, javaName, logMe, make, make, make, makeCVModelBuilder, makePojoWriter, message, nclasses, nFoldCV, nFoldWork, nModelsInParallel, nModelsInParallel, numSpecialCols, paramName, rebalance, remainingTimeSecs, response, schemaDirectory, separateFeatureVecs, setTrain, setValid, shouldReorder, stop_requested, timeout, train, trainModel, trainModel, trainModelNested, trainModelNested, trainModelOnH2ONode, valid, validateStoppingMetric, validationErrors, validationWarnings, vresponse, warn

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_bestModelPredictors

public java.lang.String[][] _bestModelPredictors

_bestR2Values

public double[] _bestR2Values

_numPredictors

public int _numPredictors

_predictorNames

public java.lang.String[] _predictorNames

_glmNFolds

public int _glmNFolds

Constructor Detail

ModelSelection

public ModelSelection(boolean startup_once)

ModelSelection

public ModelSelection(ModelSelectionModel.ModelSelectionParameters parms)

ModelSelection

public ModelSelection(ModelSelectionModel.ModelSelectionParameters parms,
                      water.Key<ModelSelectionModel> key)

Method Detail

nModelsInParallel

protected int nModelsInParallel(int folds)

Overrides:

nModelsInParallel in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

trainModelImpl

protected ModelSelection.ModelSelectionDriver trainModelImpl()

Specified by:

trainModelImpl in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

can_build

public hex.ModelCategory[] can_build()

Specified by:

can_build in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

isSupervised

public boolean isSupervised()

Specified by:

isSupervised in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

haveMojo

public boolean haveMojo()

Overrides:

haveMojo in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

havePojo

public boolean havePojo()

Overrides:

havePojo in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

init

public void init(boolean expensive)

Overrides:

init in class hex.ModelBuilder<ModelSelectionModel,ModelSelectionModel.ModelSelectionParameters,ModelSelectionModel.ModelSelectionModelOutput>

buildExtractBestR2Model

public static GLMModel buildExtractBestR2Model(water.fvec.Frame[] trainingFrames,
                                               ModelSelectionModel.ModelSelectionParameters parms,
                                               int glmNFolds,
                                               java.lang.String foldColumn,
                                               hex.Model.Parameters.FoldAssignmentScheme foldAssignment)

Given the training Frame array, build models for each training frame and return the GLMModel with the best R2 values.

Parameters:

trainingFrames -

Returns:

forwardStep

public static GLMModel forwardStep(java.util.List<java.lang.Integer> currSubsetIndices,
                                   java.util.List<java.lang.String> coefNames,
                                   int predPos,
                                   java.util.List<java.lang.Integer> validSubsets,
                                   ModelSelectionModel.ModelSelectionParameters parms,
                                   java.lang.String foldColumn,
                                   int glmNFolds,
                                   hex.Model.Parameters.FoldAssignmentScheme foldAssignment)

Given a predictor subset with indices stored in currSubsetIndices, one more predictor from the coefNames that was not found in currSubsetIndices was added to the subset to form a new Training frame. An array of training frame are built training frames from all elligible predictors from coefNames. GLMParameters are built for all TrainingFrames, GLM models are built from those parameters. The GLM model with the highest R2 value is returned. The added predictor which resulted in the highest R2 value will be added to currSubsetIndices. see doc at https://h2oai.atlassian.net/browse/PUBDEV-8444 for details.

Parameters:

currSubsetIndices: - stored predictors that are chosen in the subset

coefNames: - predictor names of full training frame

predPos: - index/location of predictor to be added into currSubsetIndices

Returns:

GLMModel with highest R2 value

replacement

public static GLMModel replacement(java.util.List<java.lang.Integer> currSubsetIndices,
                                   java.util.List<java.lang.String> coefNames,
                                   double bestR2,
                                   ModelSelectionModel.ModelSelectionParameters parms,
                                   int glmNFolds,
                                   java.lang.String foldColumn,
                                   java.util.List<java.lang.Integer> validSubset,
                                   hex.Model.Parameters.FoldAssignmentScheme foldAssignment)

consider the predictors in subset as pred0, pred1, pred2 (using subset size 3 as example): a. Keep pred1, pred2 and replace pred0 with remaining predictors, find replacement with highest R2 b. keep pred0, pred2 and replace pred1 with remaining predictors, find replacement with highest R2 c. keeep pred0, pred1 and replace pred2 with remaining predictors, find replacement with highest R2 d. from step 2a, 2b, 2c, choose the predictor subset with highest R2, say the subset is pred0, pred4, pred2 e. Take subset pred0, pred4, pred2 and go through steps a,b,c,d again and only stop when the best R2 does not improve anymore. see doc at https://h2oai.atlassian.net/browse/PUBDEV-8444 for details.