water.util

Class MRUtils

java.lang.Object

water.util.MRUtils

public class MRUtils
extends java.lang.Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MRUtils.ClassDist Compute the class distribution from a class label vector (not counting missing values) Usage 1: Label vector is categorical ------------------------------------ Vec label = ...; assert(label.isEnum()); double[] dist = new ClassDist(label).doAll(label).dist(); Usage 2: Label vector is numerical ---------------------------------- Vec label = ...; int num_classes = ...; assert(label.isInt()); double[] dist = new ClassDist(num_classes).doAll(label).dist();
static class	MRUtils.ParallelTasks<T extends DTask<T>>

Constructor Summary

Constructors

Constructor and Description
MRUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static Frame	sampleFrame(Frame fr, long rows, long seed) Sample rows from a frame.
static Frame	sampleFrameStratified(Frame fr, Vec label, Vec weights, float[] sampling_ratios, long seed, boolean debug) Stratified sampling
static Frame	sampleFrameStratified(Frame fr, Vec label, Vec weights, float[] sampling_ratios, long maxrows, long seed, boolean allowOversampling, boolean verbose) Stratified sampling for classifiers - FIXME: For weights, this is not accurate, as the sampling is done with uniform weights
static Frame	shuffleFramePerChunk(Frame fr, long seed) Row-wise shuffle of a frame (only shuffles rows inside of each chunk)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MRUtils

public MRUtils()

Method Detail

sampleFrame

public static Frame sampleFrame(Frame fr,
                long rows,
                long seed)

Sample rows from a frame. Can be unlucky for small sampling fractions - will continue calling itself until at least 1 row is returned.

Parameters:

fr - Input frame

rows - Approximate number of rows to sample (across all chunks)

seed - Seed for RNG

Returns:

Sampled frame

shuffleFramePerChunk

public static Frame shuffleFramePerChunk(Frame fr,
                         long seed)

Row-wise shuffle of a frame (only shuffles rows inside of each chunk)

Parameters:

fr - Input frame

Returns:

Shuffled frame

sampleFrameStratified

public static Frame sampleFrameStratified(Frame fr,
                          Vec label,
                          Vec weights,
                          float[] sampling_ratios,
                          long maxrows,
                          long seed,
                          boolean allowOversampling,
                          boolean verbose)

Stratified sampling for classifiers - FIXME: For weights, this is not accurate, as the sampling is done with uniform weights

Parameters:

fr - Input frame

label - Label vector (must be enum)

weights - Weights vector, can be null

sampling_ratios - Optional: array containing the requested sampling ratios per class (in order of domains), will be overwritten if it contains all 0s

maxrows - Maximum number of rows in the returned frame

seed - RNG seed for sampling

allowOversampling - Allow oversampling of minority classes

verbose - Whether to print verbose info

Returns:

Sampled frame, with approximately the same number of samples from each class (or given by the requested sampling ratios)

sampleFrameStratified

public static Frame sampleFrameStratified(Frame fr,
                          Vec label,
                          Vec weights,
                          float[] sampling_ratios,
                          long seed,
                          boolean debug)

Stratified sampling

Parameters:

fr - Input frame

label - Label vector (from the input frame)

weights - Weight vector (from the input frame), can be null

sampling_ratios - Given sampling ratios for each class, in order of domains

seed - RNG seed

debug - Whether to print debug info

Returns:

Stratified frame