public class NFSFileVec extends FileVec
Vec will be lazily loaded from the NFS file on-demand. Each machine is expected to have the same filesystem view onto a file with the same byte contents. Each machine will lazily load only the sections of the file that are assigned to that machine. Basically, the file starts striped across some globally visible file system (e.g. NFS, or just replicated on local disk) and is loaded into memory - again striped across the machines - without any network traffic or data-motion.
Useful to "memory map" into RAM large datafiles, often pure text files.
Vec.CollectDomain, Vec.CollectDomainFast, Vec.VectorGroup, Vec.Writer
Keyed.BinarySerializer<X extends Keyed>
_chunkSize, _log2ChkSize, DFLT_CHUNK_SIZE, DFLT_LOG2_CHUNK_SIZE
_espc, DO_HISTOGRAMS, KEY_PREFIX_LEN, NO_HISTOGRAMS, PERCENTILES, T_BAD, T_ENUM, T_NUM, T_STR, T_TIME, T_TIMELAST, T_UUID, TYPE_STR
_key, EMPTY_KEY_LIST
Modifier and Type | Method and Description |
---|---|
static NFSFileVec |
make(java.io.File f)
Make a new NFSFileVec key which holds the filename implicitly.
|
static NFSFileVec |
make(java.io.File f,
Futures fs)
Make a new NFSFileVec key which holds the filename implicitly.
|
int |
setChunkSize(Frame fr,
int chunkSize) |
byteSize, calcOptimalChunkSize, chunkIdx, chunkOffset, length, nChunks, setChunkSize, writable
chunkForChunkIdx, getFirstBytes, getPreviewChunkBytes, isInt, naCnt, openStream
adaptTo, align, at, at16h, at16l, at8, atStr, base, bins, cardinality, checksum_impl, chunkForRow, chunkKey, chunkKey, domain, equals, factor, get_espc, get_type_str, get_type, getVecKey, group, hashCode, isBad, isBinary, isConst, isEnum, isNA, isNumeric, isString, isTime, isUUID, lazy_bins, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCons, makeCopy, makeCopy, makeRand, makeRepSeq, makeSeq, makeSeq, makeVec, makeVec, makeZero, makeZero, makeZero, makeZero, makeZeros, makeZeros, max, maxs, mean, min, mins, newKey, ninfs, nzCnt, open, pctiles, pinfs, postWrite, preWriting, remove_impl, set, set, set, set, setDomain, sigma, sparseRatio, startRollupStats, startRollupStats, stride, toByteArray, toDoubleArray, toEnum, toInt, toString, toStringVec
checksum, getBinarySerializer, getPublishedKeys, remove, remove, remove, remove
clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON
public static NFSFileVec make(java.io.File f)
public static NFSFileVec make(java.io.File f, Futures fs)
public int setChunkSize(Frame fr, int chunkSize)
setChunkSize
in class FileVec