hex

Class FrameSplitter

java.lang.Object

jsr166y.ForkJoinTask<java.lang.Void>

jsr166y.CountedCompleter

water.H2O.H2OCountedCompleter

hex.FrameSplitter

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.util.concurrent.Future<java.lang.Void>, Freezable

public class FrameSplitter
extends H2O.H2OCountedCompleter

Frame splitter function to divide given frame into multiple partitions based on given ratios.

The task creates ratios.length+1 output frame each containing a demanded fraction of rows from source dataset

The tasks internally extract data from source chunks and create output chunks in preserving order of parts. I.e., the 1st partition contains the first P1-rows, the 2nd partition contains following P2-rows, ...

Assumptions and invariants

• number of demanding split parts is reasonable number, i.e., <10. The task is not designed to split into many small parts.
• the worker DOES NOT preserves distribution of new chunks over the cloud according to source dataset chunks.
• rows inside one output chunk are not shuffled, they are extracted deterministically in the same order as they appear in source chunk.
• workers can enforce data transfers if they need to obtain data from remote chunks.

NOTE: the implementation is data-transfer expensive and in some cases it would be beneficial to use original implementation from 9af3f4e.

.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FrameSplitter(Frame dataset, double[] ratios, Key[] destKeys, Key jobKey)
FrameSplitter(H2O.H2OCountedCompleter cc, Frame dataset, double[] ratios, Key[] destKeys, Key jobKey)

Method Summary

Methods

Modifier and Type	Method and Description
void	compute2() Override to specify actual work to do
java.lang.Throwable[]	getErrors()
Frame[]	getResult() Blocking call to obtain a result of computation.
void	onCompletion(CountedCompleter caller) Performs an action when method CountedCompleter.tryComplete() is invoked and there are no pending counts, or when the unconditional method CountedCompleter.complete(java.lang.Void) is invoked.

Methods inherited from class water.H2O.H2OCountedCompleter

clone, compute, frozenType, icer, nextThrPriority, onExceptionalCompletion, priority, read_impl, read, readJSON_impl, readJSON, write_impl, write, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON

Methods inherited from class jsr166y.CountedCompleter

addToPendingCount, compareAndSetPendingCount, complete, exec, getCompleter, getPendingCount, getRawResult, setCompleter, setPendingCount, setRawResult, tryComplete

Methods inherited from class jsr166y.ForkJoinTask

adapt, adapt, adapt, cancel, compareAndSetForkJoinTaskTag, completeExceptionally, fork, get, get, getException, getForkJoinTaskTag, getPool, getQueuedTaskCount, getSurplusQueuedTaskCount, helpQuiesce, inForkJoinPool, invoke, invokeAll, invokeAll, invokeAll, isCancelled, isCompletedAbnormally, isCompletedNormally, isDone, join, peekNextLocalTask, pollNextLocalTask, pollTask, quietlyComplete, quietlyInvoke, quietlyJoin, reinitialize, setForkJoinTaskTag, tryUnfork

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FrameSplitter

public FrameSplitter(Frame dataset,
             double[] ratios,
             Key[] destKeys,
             Key jobKey)

FrameSplitter

public FrameSplitter(H2O.H2OCountedCompleter cc,
             Frame dataset,
             double[] ratios,
             Key[] destKeys,
             Key jobKey)

Method Detail

compute2

public void compute2()

Description copied from class: H2O.H2OCountedCompleter

Override to specify actual work to do

Specified by:

compute2 in class H2O.H2OCountedCompleter

getResult

public Frame[] getResult()

Blocking call to obtain a result of computation.

getErrors

public java.lang.Throwable[] getErrors()

onCompletion

public void onCompletion(CountedCompleter caller)

Description copied from class: CountedCompleter

Performs an action when method CountedCompleter.tryComplete() is invoked and there are no pending counts, or when the unconditional method CountedCompleter.complete(java.lang.Void) is invoked. By default, this method does nothing.

Overrides:

onCompletion in class CountedCompleter

Parameters:

caller - the task invoking this method (which may be this task itself).