water.fvec
Class FileVec

java.lang.Object
  water.Iced
      water.fvec.Vec
          water.fvec.ByteVec
              water.fvec.FileVec

All Implemented Interfaces:

java.lang.Cloneable, Freezable

Direct Known Subclasses:

HdfsFileVec, NFSFileVec, S3FileVec

public abstract class FileVec
extends ByteVec

Nested Class Summary

Nested classes/interfaces inherited from class water.fvec.Vec

Vec.CollectDomain, Vec.VectorGroup

Field Summary

static long	CHUNK_SZ
static int	LOG_CHK

Fields inherited from class water.fvec.Vec

_domain, _key, KEY_PREFIX_LEN, MAX_ENUM_SIZE

Constructor Summary

protected

FileVec(Key key, long len, byte be)

Method Summary

long	byteSize() Size of compressed vector data.
long	chunk2StartElem(int cidx) Convert a chunk-index into a starting row #.
Value	chunkIdx(int cidx) Get a Chunk's Value by index.
static int	chunkIdx(Key ckey)
static long	chunkOffset(Key ckey)
boolean	isInt() Is all integers?
long	length() Number of elements in the vector.
double	max() Return column max - lazily computed as needed.
double	mean() Return column mean - lazily computed as needed.
double	min() Return column min - lazily computed as needed.
long	naCnt() Return column missing-element-count - lazily computed as needed.
int	nChunks() Number of chunks.
double	sigma() Return column standard deviation - lazily computed as needed.
boolean	writable() Default read/write behavior for Vecs.

Methods inherited from class water.fvec.ByteVec

elem2BV, openStream

Methods inherited from class water.fvec.Vec

adaptTo, asEnum, at, at8, chunk, chunkKey, chunkLen, defaultLevels, domain, domain, equals, espc, group, hashCode, isEnum, isNA, makeCon, makeCon, makeTransf, makeTransf, makeZero, newKey, postWrite, readable, remove, rollupStats, rollupStats, rollupStats, set, set, set, setNA, toEnum, toString

Methods inherited from class water.Iced

clone, frozenType, init, newInstance, read, toDocField, write, writeJSON, writeJSONFields

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

LOG_CHK

public static final int LOG_CHK

See Also:

Constant Field Values

CHUNK_SZ

public static final long CHUNK_SZ

See Also:

Constant Field Values

Constructor Detail

FileVec

protected FileVec(Key key,
                  long len,
                  byte be)

Method Detail

length

public long length()

Description copied from class: Vec

Number of elements in the vector. Overridden by subclasses that compute length in an alternative way, such as file-backed Vecs.

Overrides:

length in class Vec

nChunks

public int nChunks()

Description copied from class: Vec

Number of chunks. Overridden by subclasses that compute chunks in an alternative way, such as file-backed Vecs.

Overrides:

nChunks in class Vec

writable

public boolean writable()

Description copied from class: Vec

Default read/write behavior for Vecs. AppendableVecs are write-only.

Overrides:

writable in class Vec

min

public double min()

Description copied from class: Vec

Return column min - lazily computed as needed.

Overrides:

min in class Vec

max

public double max()

Return column max - lazily computed as needed.

Overrides:

max in class Vec

mean

public double mean()

Return column mean - lazily computed as needed.

Overrides:

mean in class Vec

sigma

public double sigma()

Return column standard deviation - lazily computed as needed.

Overrides:

sigma in class Vec

naCnt

public long naCnt()

Return column missing-element-count - lazily computed as needed.

Overrides:

naCnt in class Vec

isInt

public boolean isInt()

Is all integers?

Overrides:

isInt in class Vec

byteSize

public long byteSize()

Size of compressed vector data.

Overrides:

byteSize in class Vec

chunk2StartElem

public long chunk2StartElem(int cidx)

Description copied from class: Vec

Convert a chunk-index into a starting row #. For constant-sized chunks this is a little shift-and-add math. For variable-sized chunks this is a table lookup.

Overrides:

chunk2StartElem in class Vec

chunkOffset

public static long chunkOffset(Key ckey)

chunkIdx

public static int chunkIdx(Key ckey)

chunkIdx

public Value chunkIdx(int cidx)

Description copied from class: Vec

Get a Chunk's Value by index. Basically the index-to-key map, plus the DKV.get. Warning: this pulls the data locally; using this call on every Chunk index on the same node will probably trigger an OOM!

Overrides:

chunkIdx in class Vec