hex

Class FrameSplitter

java.lang.Object

jsr166y.ForkJoinTask<java.lang.Void>

jsr166y.CountedCompleter

water.H2O.H2OCountedCompleter<FrameSplitter>

hex.FrameSplitter

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, java.util.concurrent.Future<java.lang.Void>, Freezable<FrameSplitter>

public class FrameSplitter
extends H2O.H2OCountedCompleter<FrameSplitter>

Frame splitter function to divide given frame into multiple partitions based on given ratios.

The task creates ratios.length+1 output frame each containing a demanded fraction of rows from source dataset

The tasks internally extract data from source chunks and create output chunks in preserving order of parts. I.e., the 1st partition contains the first P1-rows, the 2nd partition contains following P2-rows, ...

Assumptions and invariants

• number of demanding split parts is reasonable number, i.e., <10. The task is not designed to split into many small parts.
• the worker DOES NOT preserves distribution of new chunks over the cloud according to source dataset chunks.
• rows inside one output chunk are not shuffled, they are extracted deterministically in the same order as they appear in source chunk.
• workers can enforce data transfers if they need to obtain data from remote chunks.

NOTE: the implementation is data-transfer expensive and in some cases it would be beneficial to use original implementation from 9af3f4e.

.

See Also:

Serialized Form

Field Summary

Fields inherited from class water.H2O.H2OCountedCompleter

_priority

Constructor Summary

Constructors

Constructor and Description
FrameSplitter(Frame dataset, double[] ratios, Key<Frame>[] destKeys, Key<Job> jobKey)
FrameSplitter(H2O.H2OCountedCompleter cc, Frame dataset, double[] ratios, Key<Frame>[] destKeys, Key<Job> jobKey)

Method Summary

Methods

Modifier and Type	Method and Description
void	compute2() Override to specify actual work to do
Frame[]	getResult() Blocking call to obtain a result of computation.
void	onCompletion(CountedCompleter caller) Performs an action when method CountedCompleter.tryComplete() is invoked and there are no pending counts, or when the unconditional method CountedCompleter.complete(java.lang.Void) is invoked.
boolean	onExceptionalCompletion(java.lang.Throwable ex, CountedCompleter caller) Exceptional completion path; mostly does printing if the exception was not handled earlier in the stack.

Methods inherited from class water.H2O.H2OCountedCompleter

clone, compute, compute1, frozenType, icer, nextThrPriority, priority, read_impl, read, readJSON_impl, readJSON, write_impl, write, writeJSON_impl, writeJSON

Methods inherited from class jsr166y.CountedCompleter

__tryComplete, addToPendingCount, compareAndSetPendingCount, complete, exec, getCompleter, getPendingCount, getRawResult, setCompleter, setPendingCount, setRawResult, tryComplete

Methods inherited from class jsr166y.ForkJoinTask

adapt, adapt, adapt, cancel, compareAndSetForkJoinTaskTag, completeExceptionally, fork, get, get, getException, getForkJoinTaskTag, getPool, getQueuedTaskCount, getSurplusQueuedTaskCount, helpQuiesce, inForkJoinPool, invoke, invokeAll, invokeAll, invokeAll, isCancelled, isCompletedAbnormally, isCompletedNormally, isDone, join, peekNextLocalTask, pollNextLocalTask, pollTask, quietlyComplete, quietlyInvoke, quietlyJoin, reinitialize, setForkJoinTaskTag, tryUnfork

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FrameSplitter

public FrameSplitter(Frame dataset,
             double[] ratios,
             Key<Frame>[] destKeys,
             Key<Job> jobKey)

FrameSplitter

public FrameSplitter(H2O.H2OCountedCompleter cc,
             Frame dataset,
             double[] ratios,
             Key<Frame>[] destKeys,
             Key<Job> jobKey)

Method Detail

compute2

public void compute2()

Description copied from class: H2O.H2OCountedCompleter

Override to specify actual work to do

Specified by:

compute2 in class H2O.H2OCountedCompleter<FrameSplitter>

getResult

public Frame[] getResult()

Blocking call to obtain a result of computation.

onCompletion

public void onCompletion(CountedCompleter caller)

Description copied from class: CountedCompleter

Performs an action when method CountedCompleter.tryComplete() is invoked and there are no pending counts, or when the unconditional method CountedCompleter.complete(java.lang.Void) is invoked. By default, this method does nothing.

Overrides:

onCompletion in class CountedCompleter

Parameters:

caller - the task invoking this method (which may be this task itself).

onExceptionalCompletion

public boolean onExceptionalCompletion(java.lang.Throwable ex,
                              CountedCompleter caller)

Description copied from class: H2O.H2OCountedCompleter

Exceptional completion path; mostly does printing if the exception was not handled earlier in the stack.

Overrides:

onExceptionalCompletion in class H2O.H2OCountedCompleter<FrameSplitter>

Parameters:

ex - the exception

caller - the task invoking this method (which may be this task itself).

Returns:

true if this exception should be propagated to this tasks completer, if one exists.