Vec (h2o-core 3.0.0 API)

java.lang.Object
- water.Iced<T>
- - water.Keyed<Vec>
  - - water.fvec.Vec

All Implemented Interfaces:: java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable<Vec>

Direct Known Subclasses:: AppendableVec, ByteVec, EnumWrappedVec, StrWrappedVec, SubsetVec

public class Vec
extends Keyed<Vec>

A distributed vector/array/column of uniform data.

A distributed vector has a count of elements, an element-to-chunk mapping, a Java-like type (mostly determines rounding on store and display), and functions to directly load elements without further indirections. The data is compressed, or backed by disk or both.

A Vec is a collection of Chunks, each of which holds between 1,000 and 1,000,000 elements. Operations on a Chunk are intended to be single-threaded; operations on a Vec are intended to be parallel and distributed on Chunk granularities, with each Chunk being manipulated by a separate CPU. The standard Map/Reduce (MRTask) paradigm handles parallel and distributed Chunk access well.

Individual elements can be directly accessed like a (very large and distributed) array - however this is not the fastest way to access the data. Direct access from Chunks is faster, avoiding several layers of indirection. In particular accessing a random row from the Vec will force the containing Chunk data to be cached locally (and network traffic to bring it local); accessing all rows from a single machine will force all the Big Data to be pulled local typically resulting in swapping and very poor performance. The main API is provided for ease of small-data manipulations and is fairly slow for writing; when touching ALL the data you are much better off using e.g. MRTask.

The main API is at(long), set(long, long), and isNA(long):

Returns	Call	Missing?	Notes
`double`	`at(long)`	`NaN`
`long`	`at8(long)`	throws
`long`	`at16l(long)`	throws	Low half of 128-bit UUID
`long`	`at16h(long)`	throws	High half of 128-bit UUID
`ValueString`	`atStr(water.parser.ValueString, long)`	`null`	Updates ValueString in-place and returns it for flow-coding
`boolean`	`isNA(long)`	`true`
	`set(long,double)`	`NaN`
	`set(long,float)`	`NaN`	Limited precision takes less memory
	`set(long,long)`	Cannot set
	`set(long,String)`	`null`	Convenience wrapper for String
	`setNA(long)`

Example manipulating some individual elements:

    double r1 = vec.at(0x123456789L);  // Access element 0x1234567889 as a double
    double r2 = vec.at(-1);            // Throws AIOOBE
    long   r3 = vec.at8_abs(1);        // Element #1, as a long
    vec.set(2,r1+r3);                  // Set element #2, as a double

Vecs have a loosely enforced type: one of numeric, UUID or String. Numeric types are further broken down into integral (long) and real (double) types. The enum type is an integral type, with a String mapping side-array. Most of the math algorithms will treat enums as small dense integers, and most enum printouts will use the String mapping. Time is another special integral type: it is represented as milliseconds since the unix epoch, and is mostly treated as an integral type when doing math but it has special time-based printout formatting. All types support the notion of a missing element; for real types this is always NaN. It is an error to attempt to fetch a missing integral type, and isNA(long) must be called first. Integral types are losslessly compressed. Real types may lose 1 or 2 ULPS due to compression.

Reading elements as doubles, or checking for an element missing is always safe. Reading a missing integral type throws an exception, since there is no NaN equivalent in the integer domain. Writing to elements may throw if the backing data is read-only (file backed), and otherwise is fully supported.

Note this dangerous scenario: loading a missing value as a double, and setting it as a long:

   set(row,(long)at(row)); // Danger!

The cast from a Double.NaN to a long produces a zero! This code will silently replace a missing value with a zero.

Vecs have a lazily computed RollupStats object and Key. The RollupStats give fast access to the common metrics: min(), max(), mean(), sigma(), the count of missing elements (naCnt()) and non-zeros (nzCnt()), amongst other stats. They are cleared if the Vec is modified and lazily recomputed after the modified Vec is closed. Clearing the RollupStats cache is fairly expensive for individual set(long, long) calls but is easy to amortize over a large count of writes; i.e., batch writing is efficient. This is normally handled by the MRTask framework; the Vec.Writer framework allows single-threaded efficient batch writing for smaller Vecs.

Example usage of common stats:

    double mean = vec.mean();  // Vec's mean; first touch computes and caches rollups
    double min  = vec.min();   // Smallest element; already computed
    double max  = vec.max();   // Largest element; already computed
    double sigma= vec.sigma(); // Standard deviation; already computed

Example: Impute (replace) missing values with the mean. Note that the use of vec.mean() in the constructor uses (and computes) the general RollupStats before the MRTask starts. Setting a value in the Chunk clears the RollupStats (since setting any value but the mean will change the mean); they will be recomputed at the next use after the MRTask.

    new MRTask{} { final double _mean = vec.mean();
      public void map( Chunk chk ) {
        for( int row=0; row < chk._len; row++ )
          if( chk.isNA(row) ) chk.set(row,_mean);
      }
    }.doAll(vec);

Vecs have a Vec.VectorGroup. Vecs in the same VectorGroup have the same Chunk and row alignment - that is, Chunks with the same index are homed to the same Node and have the same count of rows-per-Chunk. Frames are only composed of Vecs of the same VectorGroup (or very small Vecs) guaranteeing that all elements of each row are homed to the same Node and set of Chunks - such that a simple for loop over a set of Chunks all operates locally. See the example in the Chunk class.

It is common and cheap to make new Vecs in the same VectorGroup as an existing Vec and initialized to e.g. zero. Such Vecs are often used as temps, and usually immediately set to interest values in a later MRTask pass.

Example creation of temp Vecs:

    Vec tmp0 = vec.makeZero();         // New Vec with same VectorGroup and layout as vec, filled with zero
    Vec tmp1 = vec.makeCon(mean);      // Filled with 'mean'
    assert tmp1.at(0x123456789)==mean; // All elements filled with 'mean'
    for( int i=0; i<100; i++ )         // A little math on the first 100 elements
      tmp0.set(i,tmp1.at(i)+17);       // ...set into the tmp0 vec

Vec Keys have a special layout (enforced by the various Vec constructors) so there is a direct Key mapping from a Vec to a numbered Chunk and vice-versa. This mapping is crucially used in all sorts of places, basically representing a global naming scheme across a Vec and the Chunks that make it up. The basic layout created by newKey():

              byte:    0      1   2 3 4 5  6 7 8 9  10+
  Vec   Key layout: Key.VEC  -1   vec#grp    -1     normal Key bytes; often e.g. a function of original file name
  Chunk Key layout: Key.CHK  -1   vec#grp  chunk#   normal Key bytes; often e.g. a function of original file name
  Group Key layout: Key.GRP  -1     -1       -1     normal Key bytes; often e.g. a function of original file name
  RollupStats Key : Key.CHK  -1   vec#grp    -2     normal Key bytes; often e.g. a function of original file name

See Also:: Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`Vec.CollectDomain` Collect numeric domain of given vector A map-reduce task to collect up the unique values of an integer vector and returned as the domain for the vector.
`static class`	`Vec.CollectDomainFast` (Optimized for positive ints) Collect numeric domain of given vector A map-reduce task to collect up the unique values of an integer vector and returned as the domain for the vector.
`static class`	`Vec.VectorGroup` Class representing the group of vectors.
`static class`	`Vec.Writer` A more efficient way to write randomly to a Vec - still single-threaded, still slow, but much faster than Vec.set().

Nested classes/interfaces inherited from class water.Keyed
Keyed.BinarySerializer<X extends Keyed>

Field Summary

Fields
Modifier and Type	Field and Description
`long[]`	`_espc` Element-start per chunk.
`static boolean`	`DO_HISTOGRAMS`
`static int`	`KEY_PREFIX_LEN` Internally used to help build Vec and Chunk Keys; public to help PersistNFS build file mappings.
`static boolean`	`NO_HISTOGRAMS`
`static double[]`	`PERCENTILES` Default Histogram bins.
`static byte`	`T_BAD`
`static byte`	`T_ENUM`
`static byte`	`T_NUM`
`static byte`	`T_STR`
`static byte`	`T_TIME`
`static byte`	`T_TIMELAST`
`static byte`	`T_UUID`
`static java.lang.String[]`	`TYPE_STR`

Fields inherited from class water.Keyed
_key, EMPTY_KEY_LIST

Constructor Summary

Constructors
Constructor and Description
`Vec(Key<Vec> key, long[] espc)` Build a numeric-type Vec; the caller understands Chunk layout (via the `espc` array).
`Vec(Key<Vec> key, long[] espc, java.lang.String[] domain, byte type)` Main default constructor; the caller understands Chunk layout (via the `espc` array), plus enum/factor the `domain` (or null for non-enums), and the Vec type.

Method Summary

Methods
Modifier and Type	Method and Description
`EnumWrappedVec`	`adaptTo(java.lang.String[] domain)` Make a Vec adapting this Enum vector to the 'to' Enum Vec.
`Vec`	`align(Vec vec)` Always makes a copy of the given vector which shares the same group as this Vec.
`double`	`at(long i)` Fetch element the slow way, as a double, or Double.NaN is missing.
`long`	`at16h(long i)` Fetch element the slow way, as the high half of a UUID.
`long`	`at16l(long i)` Fetch element the slow way, as the low half of a UUID.
`long`	`at8(long i)` Fetch element the slow way, as a long.
`ValueString`	`atStr(ValueString vstr, long i)` Fetch element the slow way, as a `ValueString` or null if missing.
`double`	`base()` The `base` for a simple and cheap histogram of the Vec, useful for getting a broad overview of the data.
`long[]`	`bins()` A simple and cheap histogram of the Vec, useful for getting a broad overview of the data.
`long`	`byteSize()` Size of compressed vector data.
`int`	`cardinality()` Returns cardinality for enum domain or -1 for other types.
`protected long`	`checksum_impl()` A high-quality 64-bit checksum of the Vec's content, useful for establishing dataset identity.
`Chunk`	`chunkForChunkIdx(int cidx)` The Chunk for a chunk#.
`Chunk`	`chunkForRow(long i)` The Chunk for a row#.
`Value`	`chunkIdx(int cidx)` Get a Chunk's Value by index.
`Key`	`chunkKey(int cidx)` Get a Chunk Key from a chunk-index.
`static Key`	`chunkKey(Key veckey, int cidx)` Get a Chunk Key from a chunk-index and a Vec Key, without needing the actual Vec object.
`java.lang.String[]`	`domain()` Returns the enum toString mapping array, or null if not an Categorical column.
`boolean`	`equals(java.lang.Object o)` True if two Vecs are equal.
`java.lang.String`	`factor(long i)` Returns the `i`th factor for this enum column.
`long[]`	`get_espc()` Get the _espc long[].
`java.lang.String`	`get_type_str()`
`byte`	`get_type()` Get the column type.
`static Key`	`getVecKey(Key chk_key)` Get a Vec Key from Chunk Key, without loading the Chunk.
`Vec.VectorGroup`	`group()` Get the group this vector belongs to.
`int`	`hashCode()` Vec's hashcode, which is just the Vec Key hashcode.
`boolean`	`isBad()` True if the column contains only NAs
`boolean`	`isBinary()`
`boolean`	`isConst()` True if the column contains only a constant value and it is not full of NAs
`boolean`	`isEnum()` True if this is an Categorical column.
`boolean`	`isInt()` isInt is a property of numeric Vecs and not a type; this property can be changed by assigning non-integer values into the Vec (or restored by overwriting non-integer values with integers).
`boolean`	`isNA(long row)` Fetch the missing-status the slow way.
`boolean`	`isNumeric()` True if this is a numeric column, excluding enum and time types.
`boolean`	`isString()` True if this is a String column.
`boolean`	`isTime()` True if this is a time column.
`boolean`	`isUUID()` True if this is a UUID column.
`long[]`	`lazy_bins()` Optimistically return the histogram bins, or null if not computed
`long`	`length()` Number of elements in the vector; returned as a `long` instead of an `int` because Vecs support more than 2^32 elements.
`Vec`	`makeCon(double d)` Make a new vector with the same size and data layout as the current one, and initialized to the given constant value.
`static Vec`	`makeCon(double x, long len)` Make a new constant vector with the given row count, and redistribute the data evenly around the cluster.
`static Vec`	`makeCon(double x, long len, boolean redistribute)` Make a new constant vector with the given row count.
`static Vec`	`makeCon(double x, long len, int log_rows_per_chunk)` Make a new constant vector with the given row count, and redistribute the data evenly around the cluster.
`static Vec`	`makeCon(double x, long len, int log_rows_per_chunk, boolean redistribute)` Make a new constant vector with the given row count.
`static Vec`	`makeCon(Key k, double... rows)` A Vec from an array of doubles
`Vec[]`	`makeCons(int n, long l, java.lang.String[][] domains, byte[] types)`
`Vec`	`makeCopy(java.lang.String[] domain)` A new vector which is a copy of `this` one.
`Vec`	`makeCopy(java.lang.String[] domain, byte type)`
`Vec`	`makeRand(long seed)` Make a new vector initialized random numbers with the given seed
`static Vec`	`makeRepSeq(long len, long repeat)` Make a new vector initialized to increasing integers mod `repeat`.
`static Vec`	`makeSeq(long len, boolean redistribute)` Make a new vector initialized to increasing integers, starting with 1.
`static Vec`	`makeSeq(long min, long len)` Make a new vector initialized to increasing integers, starting with `min`.
`static Vec`	`makeVec(double[] vals, Key<Vec> vecKey)`
`static Vec`	`makeVec(int[] vals, java.lang.String[] domain, Key<Vec> vecKey)`
`Vec`	`makeZero()` Make a new vector with the same size and data layout as the current one, and initialized to zero.
`static Vec`	`makeZero(long len)` Make a new zero-filled vector with the given row count.
`static Vec`	`makeZero(long len, boolean redistribute)` Make a new zero-filled vec
`Vec`	`makeZero(java.lang.String[] domain)` A new vector with the same size and data layout as the current one, and initialized to zero, with the given enum domain.
`Vec[]`	`makeZeros(int n)`
`Vec[]`	`makeZeros(int n, java.lang.String[][] domain, byte[] types)`
`double`	`max()` Vec's maximum value
`double[]`	`maxs()` Vec's 5 largest values
`double`	`mean()` Vecs's mean
`double`	`min()` Vec's minimum value
`double[]`	`mins()` Vec's 5 smallest values
`long`	`naCnt()` Count of missing elements
`int`	`nChunks()` Number of chunks, returned as an `int` - Chunk count is limited by the max size of a Java `long[]`.
`static Key<Vec>`	`newKey()` Make a new random Key that fits the requirements for a Vec key.
`long`	`ninfs()` Count of negative infinities
`long`	`nzCnt()` Count of non-zero elements
`Vec.Writer`	`open()` Create a writer for bulk serial writes into this Vec.
`double[]`	`pctiles()` A simple and cheap percentiles of the Vec, useful for getting a broad overview of the data.
`long`	`pinfs()` Count of positive infinities
`Futures`	`postWrite(Futures fs)` Stop writing into this Vec.
`void`	`preWriting()` Begin writing into this Vec.
`Futures`	`remove_impl(Futures fs)` Remove associated Keys when this guy removes.
`void`	`set(long i, double d)` Write element the slow way, as a double.
`void`	`set(long i, float f)` Write element the slow way, as a float.
`void`	`set(long i, long l)` Write element the slow way, as a long.
`void`	`set(long i, java.lang.String str)` Write element the slow way, as a String.
`void`	`setDomain(java.lang.String[] domain)` Set the Categorical/factor/categorical names.
`double`	`sigma()` Vecs's standard deviation
`double`	`sparseRatio()`
`void`	`startRollupStats(Futures fs)`
`void`	`startRollupStats(Futures fs, boolean doHisto)` Check if we have local cached copy of basic Vec stats (including histogram if requested) and if not start task to compute and fetch them; useful when launching a bunch of them in parallel to avoid single threaded execution later (e.g.
`double`	`stride()` The `stride` for a a simple and cheap histogram of the Vec, useful for getting a broad overview of the data.
`byte[]`	`toByteArray()` Convert entire Vec to an array of bytes, loading all of the data into a single large array.
`double[]`	`toDoubleArray()` Convert entire Vec to an array of doubles, loading all of the data into a single large array.
`Vec`	`toEnum()` Create a new Vec (as opposed to wrapping it) that is the Enum'ified version of the original.
`Vec`	`toInt()` Transform an Enum Vec to a Int Vec.
`java.lang.String`	`toString()` Pretty print the Vec: `[#elems, min/mean/max]{chunks,...}`
`StrWrappedVec`	`toStringVec()` Transform this vector to strings.

Methods inherited from class water.Keyed
checksum, getBinarySerializer, getPublishedKeys, remove, remove, remove, remove

Methods inherited from class water.Iced
clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait

- Field Detail
  - _espc
```
public final long[] _espc
```
    Element-start per chunk. Always zero for chunk 0. One more entry than chunks, so the last entry is the total number of rows. This field is dead/ignored in subclasses that are guaranteed to have fixed-sized chunks such as file-backed Vecs.
  - T_BAD
```
public static final byte T_BAD
```
    See Also:
    Constant Field Values
  - T_UUID
```
public static final byte T_UUID
```
    See Also:
    Constant Field Values
  - T_STR
```
public static final byte T_STR
```
    See Also:
    Constant Field Values
  - T_NUM
```
public static final byte T_NUM
```
    See Also:
    Constant Field Values
  - T_ENUM
```
public static final byte T_ENUM
```
    See Also:
    Constant Field Values
  - T_TIME
```
public static final byte T_TIME
```
    See Also:
    Constant Field Values
  - T_TIMELAST
```
public static final byte T_TIMELAST
```
  - TYPE_STR
```
public static final java.lang.String[] TYPE_STR
```
  - DO_HISTOGRAMS
```
public static final boolean DO_HISTOGRAMS
```
    See Also:
    Constant Field Values
  - NO_HISTOGRAMS
```
public static final boolean NO_HISTOGRAMS
```
    See Also:
    Constant Field Values
  - PERCENTILES
```
public static final double[] PERCENTILES
```
    Default Histogram bins.
  - KEY_PREFIX_LEN
```
public static final int KEY_PREFIX_LEN
```
    Internally used to help build Vec and Chunk Keys; public to help PersistNFS build file mappings. Not intended as a public field.
    
    See Also:
    Constant Field Values
- Constructor Detail
  - Vec
```
public Vec(Key<Vec> key,
   long[] espc)
```
    Build a numeric-type Vec; the caller understands Chunk layout (via the espc array).
  - Vec
```
public Vec(Key<Vec> key,
   long[] espc,
   java.lang.String[] domain,
   byte type)
```
    Main default constructor; the caller understands Chunk layout (via the espc array), plus enum/factor the domain (or null for non-enums), and the Vec type.
- Method Detail
  - domain
```
public final java.lang.String[] domain()
```
    Returns the enum toString mapping array, or null if not an Categorical column. Not a defensive clone (to expensive to clone; coding error to change the contents).
    
    Returns:
    the enum / factor / categorical mapping array, or null if not a Categorical column
  - factor
```
public final java.lang.String factor(long i)
```
    Returns the ith factor for this enum column.
    
    Returns:
    The ith factor
  - setDomain
```
public final void setDomain(java.lang.String[] domain)
```
    Set the Categorical/factor/categorical names. No range-checking on the actual underlying numeric domain; user is responsible for maintaining a mapping which is coherent with the Vec contents.
  - cardinality
```
public final int cardinality()
```
    Returns cardinality for enum domain or -1 for other types.
  - isEnum
```
public final boolean isEnum()
```
    True if this is an Categorical column. All enum columns are also isInt(), but not vice-versa.
    
    Returns:
    true if this is an Categorical column.
  - sparseRatio
```
public final double sparseRatio()
```
  - isUUID
```
public final boolean isUUID()
```
    True if this is a UUID column.
    
    Returns:
    true if this is a UUID column.
  - isString
```
public final boolean isString()
```
    True if this is a String column.
    
    Returns:
    true if this is a String column.
  - isNumeric
```
public final boolean isNumeric()
```
    True if this is a numeric column, excluding enum and time types.
    
    Returns:
    true if this is a numeric column, excluding enum and time types
  - isTime
```
public final boolean isTime()
```
    True if this is a time column. All time columns are also isInt(), but not vice-versa.
    
    Returns:
    true if this is a time column.
  - length
```
public long length()
```
    Number of elements in the vector; returned as a long instead of an int because Vecs support more than 2^32 elements. Overridden by subclasses that compute length in an alternative way, such as file-backed Vecs.
    
    Returns:
    Number of elements in the vector
  - nChunks
```
public int nChunks()
```
    Number of chunks, returned as an int - Chunk count is limited by the max size of a Java long[]. Overridden by subclasses that compute chunks in an alternative way, such as file-backed Vecs.
    
    Returns:
    Number of chunks
  - get_espc
```
public long[] get_espc()
```
    Get the _espc long[].
  - get_type
```
public byte get_type()
```
    Get the column type.
  - get_type_str
```
public java.lang.String get_type_str()
```
  - isBinary
```
public boolean isBinary()
```
  - makeZero
```
public static Vec makeZero(long len,
           boolean redistribute)
```
    Make a new zero-filled vec
  - makeZero
```
public static Vec makeZero(long len)
```
    Make a new zero-filled vector with the given row count.
    
    Returns:
    New zero-filled vector with the given row count.
  - makeCon
```
public static Vec makeCon(double x,
          long len)
```
    Make a new constant vector with the given row count, and redistribute the data evenly around the cluster.
    
    Parameters:
    x - The value with which to fill the Vec.
    len - Number of rows.
    
    Returns:
    New cosntant vector with the given len.
  - makeCon
```
public static Vec makeCon(double x,
          long len,
          boolean redistribute)
```
    Make a new constant vector with the given row count.
    
    Returns:
    New constant vector with the given row count.
  - makeCon
```
public static Vec makeCon(double x,
          long len,
          int log_rows_per_chunk)
```
    Make a new constant vector with the given row count, and redistribute the data evenly around the cluster.
    
    Returns:
    New constant vector with the given row count.
  - makeCon
```
public static Vec makeCon(double x,
          long len,
          int log_rows_per_chunk,
          boolean redistribute)
```
    Make a new constant vector with the given row count.
    
    Returns:
    New constant vector with the given row count.
  - makeZero
```
public Vec makeZero()
```
    Make a new vector with the same size and data layout as the current one, and initialized to zero.
    
    Returns:
    A new vector with the same size and data layout as the current one, and initialized to zero.
  - makeZero
```
public Vec makeZero(java.lang.String[] domain)
```
    A new vector with the same size and data layout as the current one, and initialized to zero, with the given enum domain.
    
    Returns:
    A new vector with the same size and data layout as the current one, and initialized to zero, with the given enum domain.
  - makeCopy
```
public Vec makeCopy(java.lang.String[] domain)
```
    A new vector which is a copy of this one.
    
    Returns:
    a copy of the vector.
  - makeCopy
```
public Vec makeCopy(java.lang.String[] domain,
           byte type)
```
  - makeVec
```
public static Vec makeVec(double[] vals,
          Key<Vec> vecKey)
```
  - makeVec
```
public static Vec makeVec(int[] vals,
          java.lang.String[] domain,
          Key<Vec> vecKey)
```
  - makeCon
```
public Vec makeCon(double d)
```
    Make a new vector with the same size and data layout as the current one, and initialized to the given constant value.
    
    Returns:
    A new vector with the same size and data layout as the current one, and initialized to the given constant value.
  - makeZeros
```
public Vec[] makeZeros(int n)
```
  - makeZeros
```
public Vec[] makeZeros(int n,
              java.lang.String[][] domain,
              byte[] types)
```
  - makeCons
```
public Vec[] makeCons(int n,
             long l,
             java.lang.String[][] domains,
             byte[] types)
```
  - makeCon
```
public static Vec makeCon(Key k,
          double... rows)
```
    A Vec from an array of doubles
    
    Parameters:
    rows - Data
    
    Returns:
    The Vec
  - makeSeq
```
public static Vec makeSeq(long len,
          boolean redistribute)
```
    Make a new vector initialized to increasing integers, starting with 1.
    
    Returns:
    A new vector initialized to increasing integers, starting with 1.
  - makeSeq
```
public static Vec makeSeq(long min,
          long len)
```
    Make a new vector initialized to increasing integers, starting with `min`.
    
    Returns:
    A new vector initialized to increasing integers, starting with `min`.
  - makeRepSeq
```
public static Vec makeRepSeq(long len,
             long repeat)
```
    Make a new vector initialized to increasing integers mod repeat.
    
    Returns:
    A new vector initialized to increasing integers mod repeat.
  - makeRand
```
public Vec makeRand(long seed)
```
    Make a new vector initialized random numbers with the given seed
  - min
```
public double min()
```
    Vec's minimum value
    
    Returns:
    Vec's minimum value
  - mins
```
public double[] mins()
```
    Vec's 5 smallest values
    
    Returns:
    Vec's 5 smallest values
  - max
```
public double max()
```
    Vec's maximum value
    
    Returns:
    Vec's maximum value
  - maxs
```
public double[] maxs()
```
    Vec's 5 largest values
    
    Returns:
    Vec's 5 largeest values
  - isConst
```
public final boolean isConst()
```
    True if the column contains only a constant value and it is not full of NAs
    
    Returns:
    True if the column is constant
  - isBad
```
public final boolean isBad()
```
    True if the column contains only NAs
    
    Returns:
    True if the column contains only NAs
  - mean
```
public double mean()
```
    Vecs's mean
    
    Returns:
    Vec's mean
  - sigma
```
public double sigma()
```
    Vecs's standard deviation
    
    Returns:
    Vec's standard deviation
  - naCnt
```
public long naCnt()
```
    Count of missing elements
    
    Returns:
    Count of missing elements
  - nzCnt
```
public long nzCnt()
```
    Count of non-zero elements
    
    Returns:
    Count of non-zero elements
  - pinfs
```
public long pinfs()
```
    Count of positive infinities
    
    Returns:
    Count of positive infinities
  - ninfs
```
public long ninfs()
```
    Count of negative infinities
    
    Returns:
    Count of negative infinities
  - isInt
```
public boolean isInt()
```
    isInt is a property of numeric Vecs and not a type; this property can be changed by assigning non-integer values into the Vec (or restored by overwriting non-integer values with integers). This is a strong type for isEnum() and isTime() Vecs.
    
    Returns:
    true if the Vec is all integers
  - byteSize
```
public long byteSize()
```
    Size of compressed vector data.
  - bins
```
public long[] bins()
```
    A simple and cheap histogram of the Vec, useful for getting a broad overview of the data. Each bin is row-counts for the bin's range. The bin's range is computed from base() and stride(). The histogram is computed on first use and cached thereafter.
    
    Returns:
    A set of histogram bins.
  - lazy_bins
```
public long[] lazy_bins()
```
    Optimistically return the histogram bins, or null if not computed
    
    Returns:
    the histogram bins, or null if not computed
  - base
```
public double base()
```
    The base for a simple and cheap histogram of the Vec, useful for getting a broad overview of the data. This returns the base of bins()[0].
    
    Returns:
    the base of bins()[0]
  - stride
```
public double stride()
```
    The stride for a a simple and cheap histogram of the Vec, useful for getting a broad overview of the data. This returns the stride between any two bins.
    
    Returns:
    the stride between any two bins
  - pctiles
```
public double[] pctiles()
```
    A simple and cheap percentiles of the Vec, useful for getting a broad overview of the data. The specific percentiles are take from PERCENTILES.
    
    Returns:
    A set of percentiles
  - startRollupStats
```
public void startRollupStats(Futures fs)
```
  - startRollupStats
```
public void startRollupStats(Futures fs,
                    boolean doHisto)
```
    Check if we have local cached copy of basic Vec stats (including histogram if requested) and if not start task to compute and fetch them; useful when launching a bunch of them in parallel to avoid single threaded execution later (e.g. for-loop accessing min/max of every vec in a frame). Does *not* guarantee rollup stats will be cached locally when done since there may be racy changes to vec which will flush local caches.
    
    Parameters:
    fs - Futures allow to wait for this task to finish.
    doHisto - Also compute histogram, requires second pass over data amd is not computed by default.
  - checksum_impl
```
protected long checksum_impl()
```
    A high-quality 64-bit checksum of the Vec's content, useful for establishing dataset identity.
    
    Specified by:
    
    checksum_impl in class Keyed<Vec>
    
    Returns:
    Checksum of the Vec's content
  - preWriting
```
public void preWriting()
```
    Begin writing into this Vec. Immediately clears all the rollup stats (min(), max(), mean(), etc) since such values are not meaningful while the Vec is being actively modified. Can be called repeatedly. Per-chunk row-counts will not be changing, just row contents.
  - postWrite
```
public Futures postWrite(Futures fs)
```
    Stop writing into this Vec. Rollup stats will again (lazily) be computed.
  - getVecKey
```
public static Key getVecKey(Key chk_key)
```
    Get a Vec Key from Chunk Key, without loading the Chunk.
    
    Returns:
    the Vec Key for the Chunk Key
  - chunkKey
```
public Key chunkKey(int cidx)
```
    Get a Chunk Key from a chunk-index. Basically the index-to-key map.
    
    Returns:
    Chunk Key from a chunk-index
  - chunkKey
```
public static Key chunkKey(Key veckey,
           int cidx)
```
    Get a Chunk Key from a chunk-index and a Vec Key, without needing the actual Vec object. Basically the index-to-key map.
    
    Returns:
    Chunk Key from a chunk-index and Vec Key
  - chunkIdx
```
public Value chunkIdx(int cidx)
```
    Get a Chunk's Value by index. Basically the index-to-key map, plus the DKV.get(). Warning: this pulls the data locally; using this call on every Chunk index on the same node will probably trigger an OOM!
  - newKey
```
public static Key<Vec> newKey()
```
    Make a new random Key that fits the requirements for a Vec key.
    
    Returns:
    A new random Vec Key
  - group
```
public final Vec.VectorGroup group()
```
    Get the group this vector belongs to. In case of a group with only one vector, the object actually does not exist in KV store.
    
    Returns:
    VectorGroup this vector belongs to
  - chunkForChunkIdx
```
public Chunk chunkForChunkIdx(int cidx)
```
    The Chunk for a chunk#. Warning: this pulls the data locally; using this call on every Chunk index on the same node will probably trigger an OOM!
    
    Returns:
    Chunk for a chunk#
  - chunkForRow
```
public final Chunk chunkForRow(long i)
```
    The Chunk for a row#. Warning: this pulls the data locally; using this call on every Chunk index on the same node will probably trigger an OOM!
    
    Returns:
    Chunk for a row#
  - at8
```
public final long at8(long i)
```
    Fetch element the slow way, as a long. Floating point values are silently rounded to an integer. Throws if the value is missing.
    
    Returns:
    ith element as a long, or throw if missing
  - at
```
public final double at(long i)
```
    Fetch element the slow way, as a double, or Double.NaN is missing.
    
    Returns:
    ith element as a double, or Double.NaN if missing
  - isNA
```
public final boolean isNA(long row)
```
    Fetch the missing-status the slow way.
    
    Returns:
    the missing-status the slow way
  - at16l
```
public final long at16l(long i)
```
    Fetch element the slow way, as the low half of a UUID. Throws if the value is missing or not a UUID.
    
    Returns:
    ith element as a UUID low half, or throw if missing
  - at16h
```
public final long at16h(long i)
```
    Fetch element the slow way, as the high half of a UUID. Throws if the value is missing or not a UUID.
    
    Returns:
    ith element as a UUID high half, or throw if missing
  - atStr
```
public final ValueString atStr(ValueString vstr,
                long i)
```
    Fetch element the slow way, as a ValueString or null if missing. Throws if the value is not a String. ValueStrings are String-like objects than can be reused in-place, which is much more efficient than constructing Strings.
    
    Returns:
    ith element as ValueString or null if missing, or throw if not a String
  - set
```
public final void set(long i,
       long l)
```
    Write element the slow way, as a long. There is no way to write a missing value with this call. Under rare circumstances this can throw: if the long does not fit in a double (value is larger magnitude than 2^52), AND float values are stored in Vec. In this case, there is no common compatible data representation.
  - set
```
public final void set(long i,
       double d)
```
    Write element the slow way, as a double. Double.NaN will be treated as a set of a missing element.
  - set
```
public final void set(long i,
       float f)
```
    Write element the slow way, as a float. Float.NaN will be treated as a set of a missing element.
  - set
```
public final void set(long i,
       java.lang.String str)
```
    Write element the slow way, as a String. null will be treated as a set of a missing element.
  - open
```
public final Vec.Writer open()
```
    Create a writer for bulk serial writes into this Vec.
    
    Returns:
    A Writer for bulk serial writes
  - toString
```
public java.lang.String toString()
```
    Pretty print the Vec: [#elems, min/mean/max]{chunks,...}
    
    Overrides:
    
    toString in class java.lang.Object
    
    Returns:
    Brief string representation of a Vec
  - equals
```
public boolean equals(java.lang.Object o)
```
    True if two Vecs are equal. Checks for equal-Keys only (so it is fast) and not equal-contents.
    
    Overrides:
    
    equals in class java.lang.Object
    
    Returns:
    True if two Vecs are equal
  - hashCode
```
public int hashCode()
```
    Vec's hashcode, which is just the Vec Key hashcode.
    
    Overrides:
    
    hashCode in class java.lang.Object
    
    Returns:
    Vec's hashcode
  - remove_impl
```
public Futures remove_impl(Futures fs)
```
    Remove associated Keys when this guy removes. For Vecs, remove all associated Chunks.
    
    Overrides:
    
    remove_impl in class Keyed<Vec>
    
    Returns:
    Passed in Futures for flow-coding
  - align
```
public Vec align(Vec vec)
```
    Always makes a copy of the given vector which shares the same group as this Vec. This can be expensive operation since it can force copy of data among nodes.
    
    Parameters:
    vec - vector which is intended to be copied
    
    Returns:
    a copy of vec which shared the same Vec.VectorGroup with this vector
  - toEnum
```
public Vec toEnum()
```
    Create a new Vec (as opposed to wrapping it) that is the Enum'ified version of the original. The original Vec is not mutated.
  - toInt
```
public Vec toInt()
```
    Transform an Enum Vec to a Int Vec. If the domain of the Vec is stringified ints, then it will use those ints. Otherwise, it will use the raw domain mapping. If the domain is stringified ints, then all of the domain must be able to be parsed as an int. If it cannot be parsed as such, a NumberFormatException will be caught and rethrown as an IllegalArgumentException that declares the illegal domain value. Otherwise, the this pointer is copied to a new Vec whose domain is null.
    
    Returns:
    A new Vec
  - adaptTo
```
public EnumWrappedVec adaptTo(java.lang.String[] domain)
```
    Make a Vec adapting this Enum vector to the 'to' Enum Vec. The adapted EnumWrappedVec has 'this' as it's masterVec, but returns results in the 'to' domain (or just past it, if 'this' has elements not appearing in the 'to' domain).
  - toStringVec
```
public StrWrappedVec toStringVec()
```
    Transform this vector to strings. If the vector is enum an identity transformation vector is returned. Transformation is done by a StrWrappedVec which provides a mapping between values - without copying the underlying data.
    
    Returns:
    A new String Vec
  - toDoubleArray
```
public double[] toDoubleArray()
```
    Convert entire Vec to an array of doubles, loading all of the data into a single large array. Naturally this can easily run out of memory and throw an OOM; also due to JVM limitations often limited to 800M entries.
  - toByteArray
```
public byte[] toByteArray()
```
    Convert entire Vec to an array of bytes, loading all of the data into a single large array. Naturally this can easily run out of memory and throw an OOM; also due to JVM limitations often limited to 800M entries.

Class Vec

Nested Class Summary

Nested classes/interfaces inherited from class water.Keyed

Field Summary

Fields inherited from class water.Keyed

Constructor Summary

Method Summary

Methods inherited from class water.Keyed

Methods inherited from class water.Iced

Methods inherited from class java.lang.Object

Field Detail

_espc

T_BAD

T_UUID

T_STR

T_NUM

T_ENUM

T_TIME

T_TIMELAST

TYPE_STR

DO_HISTOGRAMS

NO_HISTOGRAMS

PERCENTILES

KEY_PREFIX_LEN

Constructor Detail

Vec

Vec

Method Detail

domain

factor

setDomain

cardinality

isEnum

sparseRatio

isUUID

isString

isNumeric

isTime

length

nChunks

get_espc

get_type

get_type_str

isBinary

makeZero

makeZero

makeCon

makeCon

makeCon

makeCon

makeZero

makeZero

makeCopy

makeCopy

makeVec

makeVec

makeCon

makeZeros

makeZeros

makeCons

makeCon

makeSeq

makeSeq

makeRepSeq

makeRand

min

mins

max

maxs

isConst

isBad

mean

sigma

naCnt

nzCnt

pinfs

ninfs

isInt

byteSize

bins