water.fvec

Class FileVec

java.lang.Object

water.Iced

water.fvec.Vec

water.fvec.ByteVec

water.fvec.FileVec

All Implemented Interfaces:

java.lang.Cloneable, Freezable

Direct Known Subclasses:

HdfsFileVec, NFSFileVec, S3FileVec, UploadFileVec

public abstract class FileVec
extends ByteVec

Nested Class Summary

Nested classes/interfaces inherited from class water.fvec.Vec

Vec.CollectDomain, Vec.VectorGroup, Vec.Writer

Field Summary

Fields

Modifier and Type	Field and Description
static int	CHUNK_SZ

Fields inherited from class water.fvec.Vec

_domain, _espc, _key, _time, KEY_PREFIX_LEN, LOG_CHK, MAX_ENUM_SIZE

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	FileVec(Key key, long len, byte be)

Method Summary

Methods

Modifier and Type	Method and Description
long	byteSize() Size of compressed vector data.
long	chunk2StartElem(int cidx) Convert a chunk-index into a starting row #.
Value	chunkIdx(int cidx) Get a Chunk's Value by index.
static int	chunkIdx(Key ckey)
static long	chunkOffset(Key ckey)
boolean	isInt() Is all integers?
long	length() Number of elements in the vector.
double	max() Return column max - lazily computed as needed.
double	mean() Return column mean - lazily computed as needed.
double	min() Return column min - lazily computed as needed.
long	naCnt() Return column missing-element-count - lazily computed as needed.
int	nChunks() Number of chunks.
double	sigma() Return column standard deviation - lazily computed as needed.
boolean	writable() Default read/write behavior for Vecs.

Methods inherited from class water.fvec.ByteVec

chunkForChunkIdx, openStream

Methods inherited from class water.fvec.Vec

align, at, at8, cardinality, checkMissing, chunkForRow, chunkKey, chunkLen, domain, domain, equals, getVecKey, group, groupKey, hash, hashCode, isBad, isByteVec, isConst, isEnum, isFloat, isNA, isTime, make1Elem, make1Elem, makeCon, makeCon, makeCon, makeCons, makeConSeq, makeNewCons, makeSeq, makeTransf, makeZero, makeZero, makeZeros, makeZeros, masterVec, newKey, open, postWrite, readable, remove, rollupStats, rollupStats, set, set, set, setNA, timeMode, timeParse, toEnum, toString

Methods inherited from class water.Iced

clone, frozenType, init, newInstance, read, toDocField, write, writeJSON, writeJSONFields

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

CHUNK_SZ

public static final int CHUNK_SZ

See Also:

Constant Field Values

Constructor Detail

FileVec

protected FileVec(Key key,
       long len,
       byte be)

Method Detail

length

public long length()

Description copied from class: Vec

Number of elements in the vector. Overridden by subclasses that compute length in an alternative way, such as file-backed Vecs.

Overrides:

length in class Vec

nChunks

public int nChunks()

Description copied from class: Vec

Number of chunks. Overridden by subclasses that compute chunks in an alternative way, such as file-backed Vecs.

Overrides:

nChunks in class Vec

writable

public boolean writable()

Description copied from class: Vec

Default read/write behavior for Vecs. AppendableVecs are write-only.

Overrides:

writable in class Vec

min

public double min()

Description copied from class: Vec

Return column min - lazily computed as needed.

Overrides:

min in class Vec

max

public double max()

Return column max - lazily computed as needed.

Overrides:

max in class Vec

mean

public double mean()

Return column mean - lazily computed as needed.

Overrides:

mean in class Vec

sigma

public double sigma()

Return column standard deviation - lazily computed as needed.

Overrides:

sigma in class Vec

naCnt

public long naCnt()

Return column missing-element-count - lazily computed as needed.

Overrides:

naCnt in class Vec

isInt

public boolean isInt()

Is all integers?

Overrides:

isInt in class Vec

byteSize

public long byteSize()

Size of compressed vector data.

Overrides:

byteSize in class Vec

chunk2StartElem

public long chunk2StartElem(int cidx)

Description copied from class: Vec

Convert a chunk-index into a starting row #. For constant-sized chunks this is a little shift-and-add math. For variable-sized chunks this is a table lookup.

Overrides:

chunk2StartElem in class Vec

chunkOffset

public static long chunkOffset(Key ckey)

chunkIdx

public static int chunkIdx(Key ckey)

chunkIdx

public Value chunkIdx(int cidx)

Description copied from class: Vec

Get a Chunk's Value by index. Basically the index-to-key map, plus the DKV.get(). Warning: this pulls the data locally; using this call on every Chunk index on the same node will probably trigger an OOM!

Overrides:

chunkIdx in class Vec