Importing H2O MOJOs

H2O MOJOs can be imported to Sparkling Water from all data sources supported by Apache Spark such as local file, S3 or HDFS and the semantics of the import is the same as in the Spark API.

When creating a MOJO specified by a relative path and HDFS is enabled, the method attempts to load the MOJO from the HDFS home directory of the current user. In case we are not running on HDFS-enabled system, we create the mojo from a current working directory.

Scala

import ai.h2o.sparkling.ml.models._
val model = H2OMOJOModel.createFromMojo("prostate.mojo")

Python

from pysparkling.ml import *
model = H2OMOJOModel.createFromMojo("prostate.mojo")

R

library(rsparkling)
sc <- spark_connect(master = "local")
model <- H2OMOJOModel.createFromMojo("prostate.mojo")

Absolute local path can also be used. To create a MOJO model from a locally available MOJO, call:

Scala

import ai.h2o.sparkling.ml.models._
val model = H2OMOJOModel.createFromMojo("/Users/peter/prostate.mojo")

Python

from pysparkling.ml import *
model = H2OMOJOModel.createFromMojo("/Users/peter/prostate.mojo")

R

library(rsparkling)
sc <- spark_connect(master = "local")
model <- H2OMOJOModel.createFromMojo("/Users/peter/prostate.mojo")

Absolute paths on Hadoop can also be used. To create a MOJO model from a MOJO stored on HDFS, call:

Scala

import ai.h2o.sparkling.ml.models._
val model = H2OMOJOModel.createFromMojo("/user/peter/prostate.mojo")

Python

from pysparkling.ml import *
model = H2OMOJOModel.createFromMojo("/user/peter/prostate.mojo")

R

library(rsparkling)
sc <- spark_connect(master = "local")
model <- H2OMOJOModel.createFromMojo("/user/peter/prostate.mojo")

The call loads the mojo file from the following location hdfs://{server}:{port}/user/peter/prostate.mojo, where {server} and {port} is automatically filled in by Spark.

You can also manually specify the type of data source you need to use, in that case, you need to provide the schema:

Scala

import ai.h2o.sparkling.ml.models._
// HDFS
val modelHDFS = H2OMOJOModel.createFromMojo("hdfs:///user/peter/prostate.mojo")
// Local file
val modelLocal = H2OMOJOModel.createFromMojo("file:///Users/peter/prostate.mojo")

Python

from pysparkling.ml import *
# HDFS
modelHDFS = H2OMOJOModel.createFromMojo("hdfs:///user/peter/prostate.mojo")
# Local file
modelLocal = H2OMOJOModel.createFromMojo("file:///Users/peter/prostate.mojo")

R

library(rsparkling)
sc <- spark_connect(master = "local")
 # HDFS
modelHDFS <- H2OMOJOModel.createFromMojo("hdfs:///user/peter/prostate.mojo")
# Local file
modelLocal <- H2OMOJOModel.createFromMojo("file:///Users/peter/prostate.mojo")

The loaded model is an immutable instance, so it’s not possible to change the configuration of the model during its existence. On the other hand, the model can be configured during its creation via H2OMOJOSettings, in Scala:

Scala

import ai.h2o.sparkling.ml.models._
val settings = H2OMOJOSettings(convertUnknownCategoricalLevelsToNa = true, convertInvalidNumbersToNa = true)
val model = H2OMOJOModel.createFromMojo("prostate.mojo", settings)

Python

from pysparkling.ml import *
settings = H2OMOJOSettings(convertUnknownCategoricalLevelsToNa = True, convertInvalidNumbersToNa = True)
model = H2OMOJOModel.createFromMojo("prostate.mojo", settings)

R

library(rsparkling)
sc <- spark_connect(master = "local")
settings <- H2OMOJOSettings(convertUnknownCategoricalLevelsToNa = TRUE, convertInvalidNumbersToNa = TRUE)
model <- H2OMOJOModel.createFromMojo("prostate.mojo", settings)

To score the dataset using the loaded mojo, call:

Scala

model.transform(dataset)

Python

model.transform(dataset)

R

model$transform(dataset)

In Scala, the createFromMojo method returns a mojo model instance casted as a base class H2OMOJOModel. This class holds only properties that are shared accross all MOJO model types from the following type hierarchy:

• H2OMOJOModel

  • H2OUnsupervisedMOJOModel

  • H2OSupervisedMOJOModel

    • H2OTreeBasedSupervisedMOJOModel

If a Scala user wants to get a property specific for a given MOJO model type, he/she must utilize casting or call the createFromMojo method on the specific MOJO model type.

import ai.h2o.sparkling.ml.models._
val specificModel = H2OTreeBasedSupervisedMOJOModel.createFromMojo("prostate.mojo")
println(s"Ntrees: ${specificModel.getNTrees()}") // Relevant only to GBM, DRF and XGBoost