public class NFSFileVec extends FileVec
Vec will be lazily loaded from the NFS file on-demand. Each machine is expected to have the same filesystem view onto a file with the same byte contents. Each machine will lazily load only the sections of the file that are assigned to that machine. Basically, the file starts striped across some globally visible file system (e.g. NFS, or just replicated on local disk) and is loaded into memory - again striped across the machines - without any network traffic or data-motion.
Useful to "memory map" into RAM large datafiles, often pure text files.
Vec.CollectDomain, Vec.CollectDomainFast, Vec.VectorGroup, Vec.WriterKeyed.BinarySerializer<X extends Keyed>_chunkSize, _log2ChkSize, DFLT_CHUNK_SIZE, DFLT_LOG2_CHUNK_SIZE_espc, DO_HISTOGRAMS, KEY_PREFIX_LEN, NO_HISTOGRAMS, PERCENTILES, T_BAD, T_ENUM, T_NUM, T_STR, T_TIME, T_TIMELAST, T_UUID, TYPE_STR_key, EMPTY_KEY_LIST| Modifier and Type | Method and Description |
|---|---|
static NFSFileVec |
make(java.io.File f)
Make a new NFSFileVec key which holds the filename implicitly.
|
static NFSFileVec |
make(java.io.File f,
Futures fs)
Make a new NFSFileVec key which holds the filename implicitly.
|
int |
setChunkSize(Frame fr,
int chunkSize) |
byteSize, calcOptimalChunkSize, chunkIdx, chunkOffset, length, nChunks, setChunkSize, writablechunkForChunkIdx, getFirstBytes, getPreviewChunkBytes, isInt, naCnt, openStreamadaptTo, align, at, at16h, at16l, at8, atStr, base, bins, cardinality, checksum_impl, chunkForRow, chunkKey, chunkKey, domain, equals, factor, get_espc, get_type_str, get_type, getVecKey, group, hashCode, isBad, isBinary, isConst, isEnum, isNA, isNumeric, isString, isTime, isUUID, lazy_bins, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCons, makeCopy, makeCopy, makeRand, makeRepSeq, makeSeq, makeSeq, makeVec, makeVec, makeZero, makeZero, makeZero, makeZero, makeZeros, makeZeros, max, maxs, mean, min, mins, newKey, ninfs, nzCnt, open, pctiles, pinfs, postWrite, preWriting, remove_impl, set, set, set, set, setDomain, sigma, sparseRatio, startRollupStats, startRollupStats, stride, toByteArray, toDoubleArray, toEnum, toInt, toString, toStringVecchecksum, getBinarySerializer, getPublishedKeys, remove, remove, remove, removeclone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSONpublic static NFSFileVec make(java.io.File f)
public static NFSFileVec make(java.io.File f, Futures fs)
public int setChunkSize(Frame fr, int chunkSize)
setChunkSize in class FileVec