water

Class Job.ValidatedJob

java.lang.Object

water.Iced

water.api.Constants

water.api.RequestStatics

water.api.RequestArguments

water.api.RequestQueries

water.api.RequestBuilders

water.api.Request

water.Request2

water.Func

water.Job

water.Job.FrameJob

water.Job.ColumnsJob

water.Job.ColumnsResJob

water.Job.ModelJobWithoutClassificationField

water.Job.ModelJob

water.Job.ValidatedJob

All Implemented Interfaces:

java.lang.Cloneable, Freezable

Direct Known Subclasses:

DeepLearning, NeuralNet, SharedTreeModelBuilder, SpeeDRF

Enclosing class:

Job

public abstract static class Job.ValidatedJob
extends Job.ModelJob

Job which produces model and validate it on a given dataset. INPUT validation frame

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	Job.ValidatedJob.Response2CMAdaptor A micro helper for transforming model/validation responses to confusion matrix domain.

Nested classes/interfaces inherited from class water.Job

Job.ChunkProgress, Job.ChunkProgressJob, Job.ColumnsJob, Job.ColumnsResJob, Job.Fail, Job.FrameJob, Job.JobCancelledException, Job.JobHandle, Job.JobState, Job.List, Job.ModelJob, Job.ModelJobWithoutClassificationField, Job.Progress, Job.ProgressMonitor, Job.ValidatedJob

Nested classes/interfaces inherited from class water.Request2

Request2.ColumnSelect, Request2.Dependent, Request2.DoClassBoolean, Request2.DRFCopyDataBoolean, Request2.MultiVecSelect, Request2.MultiVecSelectType, Request2.SpecialVecSelect, Request2.TypeaheadKey, Request2.VecClassSelect, Request2.VecSelect

Nested classes/interfaces inherited from class water.api.Request

Request.API, Request.Default, Request.Filter, Request.Validator<V>

Nested classes/interfaces inherited from class water.api.RequestBuilders

RequestBuilders.ArrayBuilder, RequestBuilders.ArrayHeaderRowBuilder, RequestBuilders.ArrayRowBuilder, RequestBuilders.ArrayRowElementBuilder, RequestBuilders.ArrayRowSingleColBuilder, RequestBuilders.BooleanStringBuilder, RequestBuilders.Builder, RequestBuilders.ElementBuilder, RequestBuilders.HideBuilder, RequestBuilders.KeyCellBuilder, RequestBuilders.KeyElementBuilder, RequestBuilders.KeyLinkElementBuilder, RequestBuilders.KeyMinAvgMaxBuilder, RequestBuilders.NoCaptionObjectBuilder, RequestBuilders.ObjectBuilder, RequestBuilders.PaginatedTable, RequestBuilders.PreFormattedBuilder, RequestBuilders.Response, RequestBuilders.ResponseInfo, RequestBuilders.WarningCellBuilder

Nested classes/interfaces inherited from class water.api.RequestArguments

RequestArguments.Argument<T>, RequestArguments.Bool, RequestArguments.ClassifyBool, RequestArguments.DRFCopyDataBool, RequestArguments.EnumArgument<T extends java.lang.Enum<T>>, RequestArguments.ExistingFile, RequestArguments.FrameClassVec, RequestArguments.FrameKeyMultiVec, RequestArguments.FrameKeyVec, RequestArguments.GeneralFile, RequestArguments.H2OExistingKey, RequestArguments.H2OIllegalArgumentException, RequestArguments.H2OKey, RequestArguments.H2OKey2, RequestArguments.InputCheckBox, RequestArguments.InputSelect<T>, RequestArguments.InputText<T>, RequestArguments.Int, RequestArguments.LongInt, RequestArguments.MultipleSelect<T>, RequestArguments.MultipleText<T>, RequestArguments.NumberSequence, RequestArguments.NumberSequenceFloat, RequestArguments.Real, RequestArguments.Record<T>, RequestArguments.RSeq, RequestArguments.RSeqFloat, RequestArguments.Str, RequestArguments.StringList, RequestArguments.TypeaheadInputText<T>

Nested classes/interfaces inherited from class water.api.RequestStatics

RequestStatics.RequestType

Nested classes/interfaces inherited from class water.api.Constants

Constants.Extensions, Constants.Schemes, Constants.Suffixes

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String[]	_cmDomain CM domain derived from _validResponseDomain and _sourceResponseDomain.
int	_cv_count
protected java.lang.String[]	_names Names of columns
java.lang.String	_responseName Name of validation response.
protected java.lang.String[]	_sourceResponseDomain Source response domain or null if response is float.
protected Vec[]	_train
protected Vec[]	_valid
protected Vec	_validResponse Validation vector extracted from validation frame.
protected java.lang.String[]	_validResponseDomain Validation response domain or null if validation is not specified or null if response is float.
static DocGen.FieldDoc[]	DOC_FIELDS
float	holdout_fraction
boolean	keep_cross_validation_splits
int	n_folds
Frame	validation
Key[]	xval_models

Fields inherited from class water.Job.ModelJob

classification

Fields inherited from class water.Job.ColumnsResJob

response

Fields inherited from class water.Job.ColumnsJob

cols, ignored_cols, ignored_cols_by_name

Fields inherited from class water.Job.FrameJob

source

Fields inherited from class water.Job

_cv, _fjtask, description, destination_key, end_time, exception, job_key, LIST, start_time, state

Fields inherited from class water.Request2

_parms, response_info

Fields inherited from class water.api.Request

_requestHelp, SUPPORTS_ONLY_V1, SUPPORTS_ONLY_V2, SUPPORTS_V1_V2

Fields inherited from class water.api.RequestBuilders

ARRAY_BUILDER, ARRAY_HEADER_ROW_BUILDER, ARRAY_ROW_BUILDER, ARRAY_ROW_ELEMENT_BUILDER, ARRAY_ROW_SINGLECOL_BUILDER, ELEMENT_BUILDER, GSON_BUILDER, OBJECT_BUILDER, ROOT_OBJECT

Fields inherited from class water.api.RequestQueries

_queryHtml

Fields inherited from class water.api.RequestArguments

_arguments

Fields inherited from class water.api.Constants

ALPHA, ARGUMENTS, AUC, BASE, BEST_THRESHOLD, BETA_EPS, BIN_LIMIT, BROWSE, BUCKET, BUILT_IN_KEY_JOBS, CANCELLED, CARDINALITY, CASE, CASE_MODE, CHUNK, CLASS, CLOUD_HEALTH, CLOUD_NAME, CLOUD_SIZE, CLOUD_UPTIME_MILLIS, CLUSTERS, COEFFICIENTS, COL_INDEX, COLS, COLUMN_NAME, COLUMNS_DISPLAY, CONSENSUS, CONTENTS, COUNT, DATA_KEY, DEPTH, DESCRIPTION, DEST_KEY, DTHRESHOLDS, ELAPSED, END_TIME, ENUM_DOMAIN_SIZE, ERROR, ESCAPE_NAN, EXCLUSIVE_SPLIT_LIMIT, EXPRESSION, FAILED, FAMILY, FEATURES, FILE, FILES, FILTER, FIRST_CHUNK, FJ_QUEUE_HI, FJ_QUEUE_LO, FJ_THREADS_HI, FJ_THREADS_LO, FREE_DISK, FREE_MEM, GFLOPS, HEADER, HEIGHT, HELP, IGNORE, ITEMS, ITERATIVE_CM, JOB, JOB_KEY, JOBS, JSON_H2O, KEY, KEYS, LAMBDA, LAST_CONTACT, LIMIT, LINK, LOCKED, MAX, MAX_DISK, MAX_ITER, MAX_MEM, MAX_ROWS, MEAN, MEM_BW, MIN, MODEL_KEY, MODELS, MORE, MTRY, MTRY_NODES, NAME, NEG_X, NO_CM, NODE, NODE_HEALTH, NODE_NAME, NODES, NORMALIZE, NUM_COLS, NUM_CPUS, NUM_FAILED, NUM_KEYS, NUM_MISSING_VALUES, NUM_ROWS, NUM_SUCCEEDED, NUM_TREES, OBJECT, OFFSET, OOBEE, PARALLEL, PARSER_TYPE, PATH, PREVIEW, PREVIOUS_MODEL_KEY, PRIOR, PROGRESS, PROGRESS_KEY, PROGRESS_TOTAL, REDIRECT, REDIRECT_ARGS, REPLICATION_FACTOR, REQUEST_TIME, RESPONSE, RHO, ROW, ROW_SIZE, ROWS, RPCS, SAMPLE, SAMPLING_STRATEGY, SCALE, SEED, SENT_ROWS, SEPARATOR, SIZE, SOURCE_KEY, STACK_TRACES, START_TIME, STAT_TYPE, STATUS, STEP, STRATA_SAMPLES, SUCCEEDED, SYSTEM_LOAD, TASK_KEY, TCPS_ACTIVE, TCPS_DUTY, TIME, TO_ENUM, TOT_MEM, TREE_COUNT, TREE_DEPTH, TREE_LEAVES, TREE_NUM, TREES, TWEEDIE_POWER, TYPE, URL, USE_NON_LOCAL_DATA, VALUE, VALUE_SIZE, VALUE_TYPE, VARIANCE, VERSION, VIEW, WARNINGS, WEIGHT, WEIGHTS, WIDTH, X, XVAL, Y

Constructor Summary

Constructors

Constructor and Description
Job.ValidatedJob()

Method Summary

Methods

Modifier and Type	Method and Description
void	crossValidate(Frame[] splits, Frame[] cv_preds, long[] offsets, int i) Cross-Validate this Job (to be overridden for each instance, which also calls genericCrossValidation)
float	cv_progress(float p) Helper to compute the actual progress if we're doing cross-validation.
protected void	genericCrossValidation(Frame[] splits, long[] offsets, int i) Helper to perform the generic part of cross validation Expected to be called from each specific instance's crossValidate method
protected java.lang.String[]	getCMDomain() Returns a domain for confusion matrix.
protected Frame	getOrigValidation() Returns original validation dataset.
Job.ValidatedJob.Response2CMAdaptor	getValidAdaptor()
protected Frame	getValidation() Return validation dataset which can be adapted to a model if it is necessary.
protected java.lang.String[]	getVectorDomain(Vec v)
protected boolean	hasValidation() Returns true if the job has specified validation dataset.
protected void	init() Invoked before job runs.
protected void	prepareValidationWithModel(Model model)
protected void	queryArgumentValueSet(RequestArguments.Argument arg, java.util.Properties inputArgs) Helper to handle arguments based on existing input values
protected void	registered(RequestServer.API_VERSION ver) Helper to specify which arguments trigger a refresh on change
dontweave.gson.JsonObject	toJSON() Annotate the number of columns and rows of the validation data set in the job parameter JSON

Methods inherited from class water.Job.ColumnsJob

selectFrame, selectVecs

Methods inherited from class water.Job

all, cancel, cancel, cancel, checkIdx, defaultDestKey, defaultJobKey, dest, findJob, findJobByDest, fork, get, getState, gridParallelism, hygiene, hygiene, invoke, isCancelledOrCrashed, isCrashed, isDone, isEnded, isRunning, isRunning, onCancelled, progress, redirect, remove, runTimeMs, self, serve, speedDescription, speedValue, start, waitUntilJobEnded, waitUntilJobEnded

Methods inherited from class water.Func

cleanup, emptyLTrash, exec, execImpl, gtrash, gtrash, ltrash, ltrash

Methods inherited from class water.Request2

create, fillResponseInfo, filterNaCols, find, input, logStart, makeJsonBox, serveGrid, servePublic, set, split, superServeGrid, supportedVersions, toJSON, toString

Methods inherited from class water.api.Request

addToNavbar, addToNavbar, addToNavbar, DocExampleFail, DocExampleSucc, href, href, hrefType, HTMLHelp, htmlTemplate, initializeNavBar, log, mapTypeahead, ReSTHelp, serve, serveJava, serveResponse, toDocGET, toHTML, toJava, wrap, wrap, wrap, writeJSONFields

Methods inherited from class water.api.RequestBuilders

build, buildJSONResponseBox, buildResponseHeader, name

Methods inherited from class water.api.RequestQueries

buildQuery, checkArguments

Methods inherited from class water.api.RequestArguments

arguments, argumentsToJson, frameColumnNameToIndex

Methods inherited from class water.api.RequestStatics

checkJsonName, encodeRedirectArgs, JSON2HTML, jsonError, requestName, Str2JSON

Methods inherited from class water.Iced

clone, frozenType, init, newInstance, read, toDocField, write, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DOC_FIELDS

public static DocGen.FieldDoc[] DOC_FIELDS

_train

protected transient Vec[] _train

_valid

protected transient Vec[] _valid

_validResponse

protected transient Vec _validResponse

Validation vector extracted from validation frame.

_validResponseDomain

protected transient java.lang.String[] _validResponseDomain

Validation response domain or null if validation is not specified or null if response is float.

_sourceResponseDomain

protected transient java.lang.String[] _sourceResponseDomain

Source response domain or null if response is float.

_cmDomain

protected transient java.lang.String[] _cmDomain

CM domain derived from _validResponseDomain and _sourceResponseDomain.

_names

protected transient java.lang.String[] _names

Names of columns

_responseName

public transient java.lang.String _responseName

Name of validation response. Should be same as source response.

validation

@Request.API(help="Validation frame",
             filter=Request.Default.class,
             mustExist=true,
             json=true)
public Frame validation

n_folds

@Request.API(help="Number of folds for cross-validation (if no validation data is specified)",
             filter=Request.Default.class,
             json=true)
public int n_folds

holdout_fraction

@Request.API(help="Fraction of training data (from end) to hold out for validation (if no validation data is specified)",
             filter=Request.Default.class,
             json=true)
public float holdout_fraction

keep_cross_validation_splits

@Request.API(help="Keep cross-validation dataset splits",
             filter=Request.Default.class,
             json=true)
public boolean keep_cross_validation_splits

xval_models

@Request.API(help="Cross-validation models",
             json=true)
public Key[] xval_models

_cv_count

public int _cv_count

Constructor Detail

Job.ValidatedJob

public Job.ValidatedJob()

Method Detail

cv_progress

public float cv_progress(float p)

Helper to compute the actual progress if we're doing cross-validation. This method is supposed to be called by the progress() implementation for CV-capable algos.

Parameters:

p - Progress reported by the main job

Returns:

actual progress if CV is done, otherwise returns p

registered

protected void registered(RequestServer.API_VERSION ver)

Helper to specify which arguments trigger a refresh on change

Overrides:

registered in class Job.ColumnsResJob

Parameters:

ver -

queryArgumentValueSet

protected void queryArgumentValueSet(RequestArguments.Argument arg,
                         java.util.Properties inputArgs)

Helper to handle arguments based on existing input values

Overrides:

queryArgumentValueSet in class RequestQueries

Parameters:

arg -

inputArgs -

crossValidate

public void crossValidate(Frame[] splits,
                 Frame[] cv_preds,
                 long[] offsets,
                 int i)

Cross-Validate this Job (to be overridden for each instance, which also calls genericCrossValidation)

Parameters:

splits - Frames containing train/test splits

cv_preds - Store the predictions for each cross-validation run

offsets - Array to store the offsets of starting row indices for each cross-validation run

i - Which fold of cross-validation to perform

genericCrossValidation

protected final void genericCrossValidation(Frame[] splits,
                          long[] offsets,
                          int i)

Helper to perform the generic part of cross validation Expected to be called from each specific instance's crossValidate method

Parameters:

splits - Frames containing train/test splits

offsets - Array to store the offsets of starting row indices for each cross-validation run

i - Which fold of cross-validation to perform

toJSON

public dontweave.gson.JsonObject toJSON()

Annotate the number of columns and rows of the validation data set in the job parameter JSON

Overrides:

toJSON in class Job.ColumnsResJob

Returns:

JsonObject annotated with num_cols and num_rows of the validation data set

init

protected void init()

Description copied from class: Job

Invoked before job runs. This is the place to checks arguments are valid or throw IllegalArgumentException. It will get invoked both from the Web and Java APIs.

Overrides:

init in class Job.ModelJob

getVectorDomain

protected java.lang.String[] getVectorDomain(Vec v)

hasValidation

protected final boolean hasValidation()

Returns true if the job has specified validation dataset.

getCMDomain

protected final java.lang.String[] getCMDomain()

Returns a domain for confusion matrix.

getValidation

protected final Frame getValidation()

Return validation dataset which can be adapted to a model if it is necessary.

getOrigValidation

protected final Frame getOrigValidation()

Returns original validation dataset.

getValidAdaptor

public final Job.ValidatedJob.Response2CMAdaptor getValidAdaptor()

prepareValidationWithModel

protected final void prepareValidationWithModel(Model model)