water.util

Class FrameUtils

java.lang.Object

water.util.FrameUtils

public class FrameUtils
extends java.lang.Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FrameUtils.CategoricalBinaryEncoder Helper to convert a categorical variable into a "binary" encoding format.
static class	FrameUtils.CategoricalEigenEncoder Helper to convert a categorical variable into the first eigenvector of the dummy-expanded matrix.
static class	FrameUtils.CategoricalLabelEncoder
static class	FrameUtils.CategoricalOneHotEncoder
static class	FrameUtils.ExportTaskDriver
static class	FrameUtils.MissingInserter Helper to insert missing values into a Frame
static class	FrameUtils.WeightedMean

Constructor Summary

Constructors

Constructor and Description
FrameUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static double[]	asDoubles(Vec v)
static int[]	asInts(Vec v)
static Frame	categoricalEncoder(Frame dataset, java.lang.String[] skipCols, Model.Parameters.CategoricalEncodingScheme scheme, ToEigenVec tev)
static ChunkSummary	chunkSummary(Frame fr) Compute a chunk summary (how many chunks of each type, relative size, total size)
static void	cleanUp(IcedHashMap<Key,java.lang.String> toDelete)
static Key[]	generateNumKeys(Key mk, int num) Generate given numbers of keys by suffixing key by given numbered suffix.
static Key[]	generateNumKeys(Key mk, int num, java.lang.String delim)
static ParseSetup	guessParserSetup(ParseSetup userParserSetup, java.net.URI... uris)
static Frame	parseFrame(Key okey, java.io.File... files) Parse given file(s) into the form of single frame represented by the given key.
static Frame	parseFrame(Key okey, ParseSetup parseSetup, java.net.URI... uris)
static Frame	parseFrame(Key okey, java.net.URI... uris) Parse given set of URIs and produce a frame's key representing output.
static void	printTopCategoricalLevels(Frame fr, boolean warn, int topK)
static void	shrinkDomainsToObservedSubset(Frame frameToModifyInPlace) reduce the domains of all categorical columns to the actually observed subset
static double	sparseRatio(Chunk[] chks) compute fraction of sparse chunks in this array.
static double	sparseRatio(Frame fr)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FrameUtils

public FrameUtils()

Method Detail

parseFrame

public static Frame parseFrame(Key okey,
               java.io.File... files)
                        throws java.io.IOException

Parse given file(s) into the form of single frame represented by the given key.

Parameters:

okey - destination key for parsed frame

files - files to parse

Returns:

a new frame

Throws:

java.io.IOException

parseFrame

public static Frame parseFrame(Key okey,
               java.net.URI... uris)
                        throws java.io.IOException

Parse given set of URIs and produce a frame's key representing output.

Parameters:

okey - key for ouput frame. Can be null

uris - array of URI (file://, hdfs://, s3n://, s3a://, s3://, ...) to parse

Returns:

a frame which is saved into DKV under okey

Throws:

java.io.IOException - in case of parse error.

parseFrame

public static Frame parseFrame(Key okey,
               ParseSetup parseSetup,
               java.net.URI... uris)
                        throws java.io.IOException

Throws:

java.io.IOException

guessParserSetup

public static ParseSetup guessParserSetup(ParseSetup userParserSetup,
                          java.net.URI... uris)
                                   throws java.io.IOException

Throws:

java.io.IOException

categoricalEncoder

public static Frame categoricalEncoder(Frame dataset,
                       java.lang.String[] skipCols,
                       Model.Parameters.CategoricalEncodingScheme scheme,
                       ToEigenVec tev)

printTopCategoricalLevels

public static void printTopCategoricalLevels(Frame fr,
                             boolean warn,
                             int topK)

asDoubles

public static double[] asDoubles(Vec v)

asInts

public static int[] asInts(Vec v)

chunkSummary

public static ChunkSummary chunkSummary(Frame fr)

Compute a chunk summary (how many chunks of each type, relative size, total size)

Parameters:

fr -

Returns:

chunk summary

generateNumKeys

public static Key[] generateNumKeys(Key mk,
                    int num)

Generate given numbers of keys by suffixing key by given numbered suffix.

generateNumKeys

public static Key[] generateNumKeys(Key mk,
                    int num,
                    java.lang.String delim)

sparseRatio

public static double sparseRatio(Chunk[] chks)

compute fraction of sparse chunks in this array.

Parameters:

chks -

Returns:

sparseRatio

public static double sparseRatio(Frame fr)

cleanUp

public static void cleanUp(IcedHashMap<Key,java.lang.String> toDelete)

shrinkDomainsToObservedSubset

public static void shrinkDomainsToObservedSubset(Frame frameToModifyInPlace)

reduce the domains of all categorical columns to the actually observed subset

Parameters:

frameToModifyInPlace -