hex.tree

Class ScoreBuildHistogram

java.lang.Object

jsr166y.ForkJoinTask<java.lang.Void>

jsr166y.CountedCompleter

water.H2O.H2OCountedCompleter<T>

water.DTask<T>

water.MRTask<ScoreBuildHistogram>

hex.tree.ScoreBuildHistogram

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.util.concurrent.Future<java.lang.Void>, jsr166y.ForkJoinPool.ManagedBlocker, water.Freezable<ScoreBuildHistogram>

Direct Known Subclasses:

ScoreBuildHistogram2

public class ScoreBuildHistogram
extends water.MRTask<ScoreBuildHistogram>

Score and Build Histogram

Fuse 2 conceptual passes into one:

Pass 1:

Score a prior partially-built tree model, and make new Node assignments to every row. This involves pulling out the current assigned DecidedNode, "scoring" the row against that Node's decision criteria, and assigning the row to a new child UndecidedNode (and giving it an improved prediction).

Pass 2:

Build new summary DHistograms on the new child UndecidedNodes every row got assigned into. Collect counts, mean, variance, min, max per bin, per column.

The result is a set of DHistogram arrays; one DHistogram array for each unique 'leaf' in the tree being histogramed in parallel. These have node ID's (nids) from 'leaf' to 'tree._len'. Each DHistogram array is for all the columns in that 'leaf'.

The other result is a prediction "score" for the whole dataset, based on the previous passes' DHistograms.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class water.DTask

water.DTask.DKeyTask<T extends water.DTask.DKeyTask,V extends water.Keyed>, water.DTask.RemoveCall

Field Summary

Fields

Modifier and Type	Field and Description
static int	DECIDED_ROW Marker for already decided row.
static int	FRESH
static int	MISSING_RESPONSE Marker for rows without a response
static int	OUT_OF_BAG Marker for sampled out rows
static int	UNDECIDED_CHILD_NODE_ID Marker for a fresh tree

Fields inherited from class water.MRTask

_appendables, _fr, _fs, _hi, _keys, _left, _lo, _nhi, _nleft, _nlo, _nrite, _rite, _run_local, _topLocal

Fields inherited from class water.DTask

_ex

Constructor Summary

Constructors

Constructor and Description
ScoreBuildHistogram(water.H2O.H2OCountedCompleter cc, int k, int ncols, int nbins, int nbins_cats, DTree tree, int leaf, DHistogram[][] hcs, hex.genmodel.utils.DistributionFamily family, int weightIdx, int workIdx, int nidIdx)

Method Summary

Methods

Modifier and Type	Method and Description
ScoreBuildHistogram	dfork2(byte[] types, water.fvec.Frame fr, boolean run_local)
static boolean	isDecidedRow(int nid)
static boolean	isOOBRow(int nid)
void	map(water.fvec.Chunk[] chks)
static int	nid2Oob(int nid)
static int	oob2Nid(int oobNid)
void	reduce(ScoreBuildHistogram sbh)
protected void	score_decide(water.fvec.Chunk[] chks, water.fvec.Chunk nids, int[] nnids)
void	setupLocal()

Methods inherited from class water.MRTask

appendables, asyncExecOnAllNodes, block, closeLocal, compute2, dfork, dfork, dfork, dfork, dfork, dinvoke, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAll, doAllNodes, getResult, getResult, isReleasable, map, map, map, map, map, map, map, map, map, map, modifiesVolatileVecs, onCompletion, onExceptionalCompletion, outputFrame, outputFrame, outputFrame, postGlobal, profile, profString, self

Methods inherited from class water.DTask

copyOver, getDException, hasException, logVerbose, onAck, onAckAck, setException

Methods inherited from class water.H2O.H2OCountedCompleter

asBytes, clone, compute, compute1, currThrPriority, frozenType, icer, priority, read, readJSON, reloadFromBytes, write, writeJSON

Methods inherited from class jsr166y.CountedCompleter

__tryComplete, addToPendingCount, compareAndSetPendingCount, complete, exec, getCompleter, getPendingCount, getRawResult, setCompleter, setPendingCount, setRawResult, tryComplete

Methods inherited from class jsr166y.ForkJoinTask

adapt, adapt, adapt, cancel, compareAndSetForkJoinTaskTag, completeExceptionally, fork, get, get, getException, getForkJoinTaskTag, getPool, getQueuedTaskCount, getSurplusQueuedTaskCount, helpQuiesce, inForkJoinPool, invoke, invokeAll, invokeAll, invokeAll, isCancelled, isCompletedAbnormally, isCompletedNormally, isDone, join, peekNextLocalTask, pollNextLocalTask, pollTask, quietlyComplete, quietlyInvoke, quietlyJoin, reinitialize, setForkJoinTaskTag, tryUnfork

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DECIDED_ROW

public static final int DECIDED_ROW

Marker for already decided row.

See Also:

Constant Field Values

OUT_OF_BAG

public static final int OUT_OF_BAG

Marker for sampled out rows

See Also:

Constant Field Values

MISSING_RESPONSE

public static final int MISSING_RESPONSE

Marker for rows without a response

See Also:

Constant Field Values

UNDECIDED_CHILD_NODE_ID

public static final int UNDECIDED_CHILD_NODE_ID

Marker for a fresh tree

See Also:

Constant Field Values

FRESH

public static final int FRESH

See Also:

Constant Field Values

Constructor Detail

ScoreBuildHistogram

public ScoreBuildHistogram(water.H2O.H2OCountedCompleter cc,
                   int k,
                   int ncols,
                   int nbins,
                   int nbins_cats,
                   DTree tree,
                   int leaf,
                   DHistogram[][] hcs,
                   hex.genmodel.utils.DistributionFamily family,
                   int weightIdx,
                   int workIdx,
                   int nidIdx)

Method Detail

dfork2

public ScoreBuildHistogram dfork2(byte[] types,
                         water.fvec.Frame fr,
                         boolean run_local)

isOOBRow

public static boolean isOOBRow(int nid)

isDecidedRow

public static boolean isDecidedRow(int nid)

oob2Nid

public static int oob2Nid(int oobNid)

nid2Oob

public static int nid2Oob(int nid)

setupLocal

public void setupLocal()

Overrides:

setupLocal in class water.MRTask<ScoreBuildHistogram>

map

public void map(water.fvec.Chunk[] chks)

Overrides:

map in class water.MRTask<ScoreBuildHistogram>

reduce

public void reduce(ScoreBuildHistogram sbh)

Overrides:

reduce in class water.MRTask<ScoreBuildHistogram>

score_decide

protected void score_decide(water.fvec.Chunk[] chks,
                water.fvec.Chunk nids,
                int[] nnids)