Packages

class H2OContext extends H2OContextExtensions

Create new H2OContext based on provided H2O configuration

Self Type
H2OContext
Linear Supertypes
H2OContextExtensions, ShellUtils, RestApiUtils, RestCommunication, RestEncodingUtils, Logging, Serializable, Serializable, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. H2OContext
  2. H2OContextExtensions
  3. ShellUtils
  4. RestApiUtils
  5. RestCommunication
  6. RestEncodingUtils
  7. Logging
  8. Serializable
  9. Serializable
  10. Logging
  11. AnyRef
  12. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. def asDataFrame(s: String): DataFrame
    Annotations
    @DeprecatedMethod( "asSparkFrame" , "3.32" )
  2. def asDataFrame(s: String, copyMetadata: Boolean): DataFrame
    Annotations
    @DeprecatedMethod( "asSparkFrame" , "3.32" )
  3. def asDataFrame[T <: Frame](fr: T, copyMetadata: Boolean = true): DataFrame
    Annotations
    @DeprecatedMethod( "asSparkFrame" , "3.32" )
  4. def asH2OFrame(fr: Frame): H2OFrame

    Create a new H2OFrame based on existing Frame

  5. def asH2OFrame(s: String): H2OFrame

    Create a new H2OFrame based on existing Frame referenced by its key.

  6. def asH2OFrame(ds: SupportedDataset, frameName: String): H2OFrame
  7. def asH2OFrame(ds: SupportedDataset, frameName: Option[String]): H2OFrame
  8. def asH2OFrame(ds: SupportedDataset): H2OFrame

    Transforms Dataset[Supported type] to H2OFrame

  9. def asH2OFrame(df: DataFrame, frameName: String): H2OFrame
  10. def asH2OFrame(df: DataFrame, frameName: Option[String]): H2OFrame
  11. def asH2OFrame(df: DataFrame): H2OFrame

    Transform DataFrame to H2OFrame

  12. def asH2OFrame(rdd: SupportedRDD, frameName: String): H2OFrame
  13. def asH2OFrame(rdd: SupportedRDD, frameName: Option[String]): H2OFrame
  14. def asH2OFrame(rdd: SupportedRDD): H2OFrame

    Transforms RDD[Supported type] to H2OFrame

  15. def asH2OFrameKeyString(df: DataFrame, frameName: Option[String]): String
  16. def asH2OFrameKeyString(df: DataFrame, frameName: String): String
  17. def asH2OFrameKeyString(df: DataFrame): String

    The method transforms Spark DataFrame to H2OFrame and returns String representation of its key.

    The method transforms Spark DataFrame to H2OFrame and returns String representation of its key.

    df

    Input data frame

    returns

    String representation of H2O Frame Key

  18. def asH2OFrameKeyString(rdd: SupportedRDD, frameName: Option[String]): String
  19. def asH2OFrameKeyString(rdd: SupportedRDD, frameName: String): String
  20. def asH2OFrameKeyString(rdd: SupportedRDD): String

    The method transforms RDD to H2OFrame and returns String representation of its key.

    The method transforms RDD to H2OFrame and returns String representation of its key.

    rdd

    Input RDD

    returns

    String representation of H2O Frame Key

  21. def asRDD[A <: Product](fr: ai.h2o.sparkling.H2OFrame)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A], arg1: ClassTag[A]): rdd.RDD[A]
  22. def asRDD[A <: Product](implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A], arg1: ClassTag[A]): AnyRef { def apply[T <: org.apache.spark.h2o.Frame](fr: T): org.apache.spark.h2o.RDD[A] }

    A generic convert of Frame into Product RDD type

    A generic convert of Frame into Product RDD type

    Consider using asH2OFrame since asRDD has several limitations such as that asRDD can't be used in Spark REPL in case we are RDD[T] where T is class defined in REPL. This is because class T is created as inner class and we are not able to create instance of class T without outer scope - which is impossible to get.

    This code: hc.asRDD[PUBDEV458Type](rdd) will need to be call as hc.asRDD[PUBDEV458Type].apply(rdd)

  23. def asRDD[A <: Product](fr: H2OFrame)(implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[A], arg1: ClassTag[A]): RDD[A]

    Convert given H2O frame into a Product RDD type

    Convert given H2O frame into a Product RDD type

    Consider using asH2OFrame since asRDD has several limitations such as that asRDD can't be used in Spark REPL in case we are RDD[T] where T is class defined in REPL. This is because class T is created as inner class and we are not able to create instance of class T without outer scope - which is impossible to get.

  24. def asSparkFrame(s: String): DataFrame
  25. def asSparkFrame(s: String, copyMetadata: Boolean): DataFrame
  26. def asSparkFrame[T <: Frame](fr: T, copyMetadata: Boolean = true): DataFrame
  27. def checkResponseCode(connection: HttpURLConnection): Unit
    Definition Classes
    RestCommunication
  28. def delete[ResultType](endpoint: URI, suffix: String, conf: H2OConf, params: Map[String, Any] = Map.empty, skippedFields: Seq[(Class[_], String)] = Seq.empty, encodeParamsAsJson: Boolean = false)(implicit arg0: ClassTag[ResultType]): ResultType

    endpoint

    An address of H2O node with exposed REST endpoint

    suffix

    REST relative path representing a specific call

    conf

    H2O conf object

    Definition Classes
    RestCommunication
  29. def downloadH2OLogs(destinationDir: String, logContainer: String): String
    Definition Classes
    H2OContextExtensions
  30. def flowURL(): String
  31. def getClusterEndpoint(conf: H2OConf): URI
    Definition Classes
    RestApiUtils
  32. def getClusterInfo(conf: H2OConf): CloudV3
    Definition Classes
    RestApiUtils
  33. def getConf: H2OConf

    Return a copy of this H2OContext's configuration.

    Return a copy of this H2OContext's configuration. The configuration cannot be changed at runtime.

  34. def getH2OLogLevel(): String
  35. def getH2ONodes(): Array[NodeDesc]
  36. def getLogLevel(conf: H2OConf): String
    Definition Classes
    RestApiUtils
  37. def getNodes(cloudV3: CloudV3): Array[NodeDesc]
    Definition Classes
    RestApiUtils
  38. def getNodes(conf: H2OConf): Array[NodeDesc]
    Definition Classes
    RestApiUtils
  39. def getPingInfo(conf: H2OConf): PingV3
    Definition Classes
    RestApiUtils
  40. def getTimeZone(conf: H2OConf): String
    Definition Classes
    RestApiUtils
  41. def h2oLocalClient: String

    Returns location of REST API of H2O client

  42. def h2oLocalClientIp: String

    Returns IP of H2O client

  43. def h2oLocalClientPort: Int

    Returns port where H2O REST API is exposed

  44. def importHiveTable(database: String = HiveTableImporter.DEFAULT_DATABASE, table: String, partitions: Array[Array[String]] = null, allowMultiFormat: Boolean = false): water.fvec.Frame
    Definition Classes
    H2OContextExtensions
  45. def insertToNode(node: NodeDesc, suffix: String, conf: H2OConf, params: Map[String, Any] = Map.empty): OutputStream

    node

    H2O node descriptor

    suffix

    REST relative path representing a specific call

    conf

    H2O conf object

    params

    Query parameters

    returns

    HttpUrlConnection facilitating the insertion and holding the outputStream

    Definition Classes
    RestCommunication
  46. def isStopped(): Boolean
  47. def openFlow(): Unit

    Open H2O Flow running in this client.

  48. def query[ResultType](endpoint: URI, suffix: String, conf: H2OConf, params: Map[String, Any] = Map.empty, skippedFields: Seq[(Class[_], String)] = Seq.empty)(implicit arg0: ClassTag[ResultType]): ResultType

    ResultType

    A type that the result will be deserialized to

    endpoint

    An address of H2O node with exposed REST endpoint

    suffix

    REST relative path representing a specific call

    conf

    H2O conf object

    params

    Query parameters

    skippedFields

    The list of field specifications that are skipped during deserialization. The specification consists of the class containing the field and the field name.

    returns

    A deserialized object

    Definition Classes
    RestCommunication
  49. def request[ResultType](endpoint: URI, requestType: String, suffix: String, conf: H2OConf, params: Map[String, Any] = Map.empty, skippedFields: Seq[(Class[_], String)] = Seq.empty, encodeParamsAsJson: Boolean = false)(implicit arg0: ClassTag[ResultType]): ResultType
    Definition Classes
    RestCommunication
  50. def resolveNodeEndpoint(node: NodeDesc, conf: H2OConf): URI
    Definition Classes
    RestApiUtils
  51. def setH2OClientLogLevel(level: String): Unit
    Annotations
    @DeprecatedMethod( "setH2OLogLevel" , "3.32" )
  52. def setH2OLogLevel(level: String): Unit
  53. def setH2ONodeLogLevel(level: String): Unit
    Annotations
    @DeprecatedMethod( "setH2OLogLevel" , "3.32" )
  54. def setLogLevel(conf: H2OConf, logLevel: String): Unit
    Definition Classes
    RestApiUtils
  55. def setTimeZone(conf: H2OConf, timezone: String): Unit
    Definition Classes
    RestApiUtils
  56. def shutdownCluster(conf: H2OConf): Unit
    Definition Classes
    RestApiUtils
  57. val sparkContext: SparkContext
  58. val sparkSession: SparkSession
  59. def stop(stopSparkContext: Boolean = false): Unit

    Stops H2O context.

    Stops H2O context.

    stopSparkContext

    stop also spark context

  60. def toH2OFrameKey(ds: SupportedDataset, frameName: String): Key[Frame]
  61. def toH2OFrameKey(ds: SupportedDataset, frameName: Option[String]): Key[Frame]
  62. def toH2OFrameKey(ds: SupportedDataset): Key[Frame]

    Transforms Dataset[Supported type] to H2OFrame key

  63. def toH2OFrameKey(df: DataFrame, frameName: String): Key[Frame]
  64. def toH2OFrameKey(df: DataFrame, frameName: Option[String]): Key[Frame]
  65. def toH2OFrameKey(df: DataFrame): Key[Frame]

    Transform DataFrame to H2OFrame key

  66. def toH2OFrameKey(rdd: SupportedRDD, frameName: String): Key[_]
  67. def toH2OFrameKey(rdd: SupportedRDD, frameName: Option[String]): Key[_]
  68. def toH2OFrameKey(rdd: SupportedRDD): Key[_]

    Transforms RDD[Supported type] to H2OFrame key

  69. def toString(): String
    Definition Classes
    H2OContext → AnyRef → Any
  70. def update[ResultType](endpoint: URI, suffix: String, conf: H2OConf, params: Map[String, Any] = Map.empty, skippedFields: Seq[(Class[_], String)] = Seq.empty, encodeParamsAsJson: Boolean = false)(implicit arg0: ClassTag[ResultType]): ResultType

    ResultType

    A type that the result will be deserialized to

    endpoint

    An address of H2O node with exposed REST endpoint

    suffix

    REST relative path representing a specific call

    conf

    H2O conf object

    params

    Query parameters

    skippedFields

    The list of field specifications that are skipped during deserialization. The specification consists of the class containing the field and the field name.

    returns

    A deserialized object

    Definition Classes
    RestCommunication
  71. object implicits extends H2OContextImplicits with Serializable

    Define implicits available via h2oContext.implicits._