hex.tree

Class DHistogram

java.lang.Object

water.Iced

hex.tree.DHistogram

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable

public final class DHistogram
extends water.Iced

A Histogram, computed in parallel over a Vec.

A DHistogram bins every value added to it, and computes a the vec min and max (for use in the next split), and response mean and variance for each bin. DHistograms are initialized with a min, max and number-of- elements to be added (all of which are generally available from a Vec). Bins run from min to max in uniform sizes. If the DHistogram can determine that fewer bins are needed (e.g. boolean columns run from 0 to 1, but only ever take on 2 values, so only 2 bins are needed), then fewer bins are used.

DHistogram are shared per-node, and atomically updated. There's an add call to help cross-node reductions. The data is stored in primitive arrays, so it can be sent over the wire.

If we are successively splitting rows (e.g. in a decision tree), then a fresh DHistogram for each split will dynamically re-bin the data. Each successive split will logarithmically divide the data. At the first split, outliers will end up in their own bins - but perhaps some central bins may be very full. At the next split(s), the full bins will get split, and again until (with a log number of splits) each bin holds roughly the same amount of data. This dynamic binning resolves a lot of problems with picking the proper bin count or limits - generally a few more tree levels will equal any fancy but fixed-size binning strategy.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
double[]	_bins
byte	_isInt
float	_maxEx
protected float	_maxIn
float	_min
protected float	_min2
java.lang.String	_name
char	_nbin
float	_step

Constructor Summary

Constructors

Constructor and Description
DHistogram(java.lang.String name, int nbins, int nbins_cats, byte isInt, float min, float maxEx)

Method Summary

Methods

Modifier and Type	Method and Description
double	bins(int b)
static DHistogram[]	initialHist(water.fvec.Frame fr, int ncols, int nbins, int nbins_cats, DHistogram[] hs)
static DHistogram	make(java.lang.String name, int nbins, int nbins_cats, byte isInt, float min, float maxEx)
int	nbins()
DTree.Split	scoreMSE(int col, double min_rows)
java.lang.String	toString()

Methods inherited from class water.Iced

clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

_name

public final transient java.lang.String _name

_isInt

public final byte _isInt

_nbin

public final char _nbin

_step

public final float _step

_min

public final float _min

_maxEx

public final float _maxEx

_bins

public double[] _bins

_min2

protected float _min2

_maxIn

protected float _maxIn

Constructor Detail

DHistogram

public DHistogram(java.lang.String name,
          int nbins,
          int nbins_cats,
          byte isInt,
          float min,
          float maxEx)

Method Detail

nbins

public int nbins()

bins

public double bins(int b)

initialHist

public static DHistogram[] initialHist(water.fvec.Frame fr,
                       int ncols,
                       int nbins,
                       int nbins_cats,
                       DHistogram[] hs)

make

public static DHistogram make(java.lang.String name,
              int nbins,
              int nbins_cats,
              byte isInt,
              float min,
              float maxEx)

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

scoreMSE

public DTree.Split scoreMSE(int col,
                   double min_rows)