public class NFSFileVec extends FileVec
Vec will be lazily loaded from the NFS file on-demand. Each machine is expected to have the same filesystem view onto a file with the same byte contents. Each machine will lazily load only the sections of the file that are assigned to that machine. Basically, the file starts striped across some globally visible file system (e.g. NFS, or just replicated on local disk) and is loaded into memory - again striped across the machines - without any network traffic or data-motion.
Useful to "memory map" into RAM large datafiles, often pure text files.
Vec.ESPC, Vec.Holder, Vec.Reader, Vec.VectorGroup, Vec.Writer
_chunkSize, _nChunks, DFLT_CHUNK_SIZE, DFLT_LOG2_CHUNK_SIZE
_cids, _rowLayout, DO_HISTOGRAMS, KEY_PREFIX_LEN, PERCENTILES, T_BAD, T_CAT, T_NUM, T_STR, T_TIME, T_UUID, TYPE_STR
Modifier and Type | Method and Description |
---|---|
static NFSFileVec |
make(java.io.File f)
Make a new NFSFileVec key which holds the filename implicitly.
|
static NFSFileVec |
make(java.io.File f,
Futures fs)
Make a new NFSFileVec key which holds the filename implicitly.
|
static NFSFileVec |
make(java.lang.String fname) |
byteSize, calcOptimalChunkSize, chunkIdx, chunkOffset, elem2ChunkIdx, getPathForKey, length, nChunks, setChunkSize, setChunkSize, setNChunks, writable
chunkForChunkIdx, getFirstBytes, getPreviewChunkBytes, isInt, naCnt, openStream
adaptTo, align, at, at16h, at16l, at8, atStr, base, bins, cardinality, checksum_impl, chunkForRow, chunkKey, chunkKey, copyMeta, doCopy, domain, equals, espc, factor, get_type_str, get_type, getVecKey, group, hashCode, isBad, isBinary, isCategorical, isCompatibleWith, isConst, isHomedLocally, isNA, isNumeric, isString, isTime, isUUID, isVolatile, lazy_bins, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCon, makeCons, makeCons, makeCopy, makeCopy, makeCopy, makeDoubles, makeRand, makeRepSeq, makeSeq, makeSeq, makeSeq, makeTimeVec, makeVec, makeVec, makeVec, makeVec, makeVec, makeVolatileDoubles, makeVolatileInts, makeZero, makeZero, makeZero, makeZero, makeZero, makeZeros, makeZeros, max, maxs, mean, min, mins, mode, newKey, ninfs, nzCnt, open, pctiles, pinfs, postWrite, preWriting, readAll_impl, remove_impl, rollupStatsKey, set, set, set, set, set, setBad, setChunkIdx, setDomain, setNA, sigma, sparseRatio, startRollupStats, startRollupStats, stride, stringAt, toCategoricalVec, toNumericVec, toString, toStringVec, toTwoDimTable, toTwoDimTable, writeAll_impl
checksum, makeSchema, readAll, remove, remove, remove, remove, writeAll
asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON
public static NFSFileVec make(java.io.File f)
public static NFSFileVec make(java.lang.String fname) throws java.io.IOException
java.io.IOException
public static NFSFileVec make(java.io.File f, Futures fs)