org.apache.spark.h2o

H2OContext

class H2OContext extends Logging with H2OConf with Serializable

Simple H2O context motivated by SQLContext.

It provides implicit conversion from RDD -> H2OLikeRDD and back.

Linear Supertypes
Serializable, Serializable, H2OConf, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. H2OContext
  2. Serializable
  3. Serializable
  4. H2OConf
  5. Logging
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new H2OContext(sparkContext: SparkContext)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. val PROP_CLIENT_LOG_LEVEL: (String, String)

    H2O log leve for client running in Spark driver

    H2O log leve for client running in Spark driver

    Definition Classes
    H2OConf
  7. val PROP_CLIENT_WEB_PORT: (String, Int)

    Exact client port to access web UI.

    Exact client port to access web UI. The value -1 means automatic search for free port starting at spark.ext.h2o.port.base.

    Definition Classes
    H2OConf
  8. val PROP_CLOUD_NAME: (String, String)

    Configuration property - name of H2O cloud

    Configuration property - name of H2O cloud

    Definition Classes
    H2OConf
  9. val PROP_CLOUD_TIMEOUT: (String, Int)

    Configuration property - timeout for cloud up.

    Configuration property - timeout for cloud up.

    Definition Classes
    H2OConf
  10. val PROP_CLUSTER_SIZE: (String, Int)

    Configuration property - expected number of workers of H2O cloud.

    Configuration property - expected number of workers of H2O cloud. Value -1 means automatic detection of cluster size.

    Definition Classes
    H2OConf
  11. val PROP_DEFAULT_CLUSTER_SIZE: (String, Int)

    Starting size of cluster in case that size is not explicitelly passed

    Starting size of cluster in case that size is not explicitelly passed

    Definition Classes
    H2OConf
  12. val PROP_DISABLE_GA: (String, Boolean)

    Disable GA tracking

    Disable GA tracking

    Definition Classes
    H2OConf
  13. val PROP_DUMMY_RDD_MUL_FACTOR: (String, Int)

    Configuration property - multiplication factor for dummy RDD generation.

    Configuration property - multiplication factor for dummy RDD generation. Size of dummy RDD is PROP_CLUSTER_SIZE*PROP_DUMMY_RDD_MUL_FACTOR

    Definition Classes
    H2OConf
  14. val PROP_NETWORK_MASK: (String, String)

    Subnet selector for h2o if IP guess fail - useful if 'spark.

    Subnet selector for h2o if IP guess fail - useful if 'spark.ext.h2o.flatfile' is false and we are trying to guess right IP on mi

    Definition Classes
    H2OConf
  15. val PROP_NODE_LOG_LEVEL: (String, String)

    Definition Classes
    H2OConf
  16. val PROP_NTHREADS: (String, Int)

    Limit for number of threads used by H2O, default -1 means unlimited

    Limit for number of threads used by H2O, default -1 means unlimited

    Definition Classes
    H2OConf
  17. val PROP_PORT_BASE: (String, Int)

    Configuration property - base port used for individual H2O nodes configuration.

    Configuration property - base port used for individual H2O nodes configuration.

    Definition Classes
    H2OConf
  18. val PROP_SPREADRDD_RETRIES: (String, Int)

    Configuration property - number of retries to create an RDD spreat over all executors

    Configuration property - number of retries to create an RDD spreat over all executors

    Definition Classes
    H2OConf
  19. val PROP_USE_FLATFILE: (String, Boolean)

    Configuration property - use flatfile for H2O cloud formation.

    Configuration property - use flatfile for H2O cloud formation.

    Definition Classes
    H2OConf
  20. def asDataFrame(fr: H2OFrame)(implicit sqlContext: SQLContext): DataFrame

  21. implicit def asH2OFrame[A <: Product](rdd: RDD[A])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): H2OFrame

    Implicit conversion from typed RDD to H2O's DataFrame

  22. implicit def asH2OFrame(rdd: DataFrame): H2OFrame

    Implicit conversion from Spark DataFrame to H2O's DataFrame

  23. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  24. def asRDD[A <: Product](fr: H2OFrame)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A], arg1: ClassTag[A]): RDD[A]

    Convert given H2O frame into a Product RDD type

  25. def basePort: Int

    Definition Classes
    H2OConf
  26. def clientWebPort: Int

    Definition Classes
    H2OConf
  27. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  28. def cloudName: String

    Definition Classes
    H2OConf
  29. def cloudTimeout: Int

    Definition Classes
    H2OConf
  30. implicit def createH2OFrame(fr: Frame): H2OFrame

    Implicit conversion from Frame to DataFrame

  31. def createH2ORDD[A <: Product](fr: H2OFrame)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A], arg1: ClassTag[A]): RDD[A]

  32. def createH2OSchemaRDD(fr: H2OFrame)(implicit sqlContext: SQLContext): DataFrame

  33. def defaultCloudSize: Int

    Definition Classes
    H2OConf
  34. def disableGA: Boolean

    Definition Classes
    H2OConf
  35. def drddMulFactor: Int

    Definition Classes
    H2OConf
  36. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  37. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  38. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  39. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  40. def getH2OClientArgs: Array[String]

    Get arguments for H2O client.

    Get arguments for H2O client.

    returns

    array of H2O client arguments.

    Definition Classes
    H2OConf
  41. def getH2ONodeArgs: Array[String]

    Produce arguments for H2O node based on this config.

    Produce arguments for H2O node based on this config.

    returns

    array of H2O launcher command line arguments

    Definition Classes
    H2OConf
  42. def h2oClientLogLevel: String

    Definition Classes
    H2OConf
  43. def h2oLocalClient: String

  44. def h2oNodeLogLevel: String

    Definition Classes
    H2OConf
  45. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  46. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  47. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging
  48. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging
  49. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  50. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  51. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  52. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  53. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  54. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  55. def logName: String

    Attributes
    protected
    Definition Classes
    Logging
  56. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  57. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  58. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  59. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  60. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  61. def networkMask: Option[String]

    Definition Classes
    H2OConf
  62. final def notify(): Unit

    Definition Classes
    AnyRef
  63. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  64. def nthreads: Int

    Definition Classes
    H2OConf
  65. def numH2OWorkers: Int

    Definition Classes
    H2OConf
  66. def numRddRetries: Int

    Definition Classes
    H2OConf
  67. def openFlow(): Unit

    Open H2O Flow running in this client.

  68. val sparkConf: SparkConf

    Definition Classes
    H2OContextH2OConf
  69. val sparkContext: SparkContext

  70. def start(): H2OContext

    Initialize Sparkling H2O and start H2O cloud.

  71. def start(h2oWorkers: Int): H2OContext

    Initialize Sparkling H2O and start H2O cloud with specified number of workers.

  72. implicit def symbolToString(sy: Symbol): String

    Transform given Scala symbol to String

  73. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  74. implicit def toH2OFrameKey(fr: Frame): Key[Frame]

    Returns a key of given frame

  75. implicit def toH2OFrameKey[A <: Product](rdd: RDD[A])(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A]): Key[_]

    Implicit conversion from typed RDD to H2O's DataFrame

  76. implicit def toH2OFrameKey(rdd: DataFrame): Key[Frame]

    Implicit conversion from Spark DataFrame to H2O's DataFrame

  77. def toString(): String

    Definition Classes
    H2OContextH2OConf → AnyRef → Any
  78. def useFlatFile: Boolean

    Definition Classes
    H2OConf
  79. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  80. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  81. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def asSchemaRDD(fr: H2OFrame)(implicit sqlContext: SQLContext): DataFrame

    Convert given H2O frame into DataFrame type

    Convert given H2O frame into DataFrame type

    Annotations
    @deprecated
    Deprecated

    (Since version Use asDataFrame) 1.3

  2. def toRDD[A <: Product](fr: H2OFrame)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A], arg1: ClassTag[A]): RDD[A]

    Convert given H2O frame into a RDD type

    Convert given H2O frame into a RDD type

    Annotations
    @deprecated
    Deprecated

    (Since version 0.2.3) Use asRDD instead

Inherited from Serializable

Inherited from Serializable

Inherited from H2OConf

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped