Memory Allocation

H2O resides in the same executor JVM as Spark. The memory provided for H2O is configured via Spark. Refer to Spark Configuration for more details.

Generic Configuration

  • Configure the Executor memory (i.e., memory available for H2O) via the Spark configuration property spark.executor.memory.

    For example, bin/sparkling-shell --conf spark.executor.memory=5g, or configure the property in $SPARK_HOME/conf/spark-defaults.conf

  • Configure the Driver memory (i.e., memory available for H2O client running inside Spark driver) via the Spark configuration property spark.driver.memory

    For example, bin/sparkling-shell --conf spark.driver.memory=4g, or configure the property in $SPARK_HOME/conf/spark-defaults.conf

YARN-Specific Configuration