Change Log¶

v2.2.40 (2019-05-17)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/40/index.html

Bug
- SW-1256 - Fix constructor of H2OMojoModel
- SW-1258 - Remove internal constructors & Deprecate implicit constructor parameters for H2O Algo Spark Estimators( to be the same as in PySparkling)
- SW-1270 - Fix version check in PySpakrling shell
- SW-1278 - Clean workspace on the hadoop node in integ tests
- SW-1279 - Fix inconsistencies between H2OAutoML, H2OGridSearch & H2OALgorithm
- SW-1281 - Fix bad representation of predictionCol on H2OMOJOModel
- SW-1282 - XGBoost can't be used in H2OGridSearch pipeline wrapper
- SW-1283 - Correctly return mojo model in pysparkling after fit
Story
- SW-1271 - Remove SparkContext from H2OSchemaUtils
- SW-1273 - Upgrade to H2O 3.24.0.3
New Feature
- SW-1248 - getFeaturesCols() should not return the fold column or weight column
- SW-1249 - probability calibration does not work in Sparkling Water Dataframe API
Improvement
- SW-369 - Override spark locality so we use only nodes on which h2o is running.
- SW-1216 - Improve PySparkling README
- SW-1261 - Remove binary H2O model from ML pipelines
- SW-1263 - Don't require initializer call to be called during pysparkling pipelines
- SW-1264 - Use default params reader in pipelines
- SW-1268 - Non-named columns are long time deprecated. Switch to named columns by default
- SW-1269 - Remove six as dependency from PySparkling launcher ( six is no longer dependency)
- SW-1275 - Remove unnecessary constructor in helper class
- SW-1280 - Add predictionCol to mojo pipeline model

v2.2.39 (2019-05-17)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/39/index.html

Bug
- SW-1186 - No need to pass properties defined in spark-defaults.conf to cli
- SW-1189 - Fix Sparkling Water 2.1.x compile on Scala 2.10
- SW-1194 - RSparkling Can't be used on Spark 2.4
- SW-1195 - Disable gradle daemon via gradle.properties
- SW-1196 - Fix org.apache.spark.ml.spark.models.PipelinePredictionTest
- SW-1203 - Custom metric not evaluated in internal mode of Sparkling Water
- SW-1227 - Change get-extended-jar to use https instead of http
- SW-1230 - Fix typo in GLM API - getRemoteCollinearColumns, setRemoteCollinearColumns
- SW-1232 - Fix RUnits after upgrading to Gradle 5.3.1
- SW-1234 - Deprecate asDataFrame with implicit argument
Story
- SW-1198 - Introduce new annotation deprecating legacy methods in API
- SW-1209 - Rename the 'predictionCol' model parameter to 'labelCol'
- SW-1226 - Introduce mechanism for enabling backward compatibility of MOJO files when properties are renamed
New Feature
- SW-1193 - Expose weights_column parameter
Improvement
- SW-1188 - RSparkling: Add ability to add authentication details when calling h2o_context(sc)
- SW-1190 - Improve hint description for disabling automatic usage of broadcast joins
- SW-1199 - Improve memory efficiency of H2OMOJOPipelineModel
- SW-1202 - Simplify Sparkling Water build
- SW-1204 - Fix formating in python tests
- SW-1208 - Create pysparkling tests report file if it does not exist
- SW-1210 - Add fold column to python and scala pipelines
- SW-1211 - Automatically download H2O Wheel
- SW-1213 - Upgrade to H2O 3.24.0.2
- SW-1214 - Remove PySparkling six dependency as it was removed in H2O
- SW-1215 - Automatically generate PySparkling README
- SW-1217 - Automatically generate last pieces of doc subproject
- SW-1219 - Remove suport for testing external cluster in manual mode
- SW-1221 - Remove unnecessary branch check
- SW-1222 - Remove duplicate readme file (contains old info & the correct info is in doc)
- SW-1223 - Remove confusing meetup dir
- SW-1224 - Upgrade to Gradle 5.3.1
- SW-1228 - Rename the 'ignoredColumns' parameter of H2OAutoML to 'ignoredCols'
- SW-1229 - Remove dependencies to Scala 2.10
- SW-1236 - Reformat few python classes
- SW-1238 - Parametrize EMR version in templates generation
- SW-1239 - Remove old README and DEVEL doc files (not just pointer to new doc)
- SW-1240 - Use minSupportedJava for source and target compatibility in build.gradle

v2.2.39 (2019-04-25)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/39/index.html

Bug
- SW-1186 - No need to pass properties defined in spark-defaults.conf to cli
- SW-1189 - Fix Sparkling Water 2.1.x compile on Scala 2.10
- SW-1194 - RSparkling Can't be used on Spark 2.4
- SW-1195 - Disable gradle daemon via gradle.properties
- SW-1196 - Fix org.apache.spark.ml.spark.models.PipelinePredictionTest
- SW-1203 - Custom metric not evaluated in internal mode of Sparkling Water
- SW-1227 - Change get-extended-jar to use https instead of http
- SW-1230 - Fix typo in GLM API - getRemoteCollinearColumns, setRemoteCollinearColumns
- SW-1232 - Fix RUnits after upgrading to Gradle 5.3.1
- SW-1234 - Deprecate asDataFrame with implicit argument
Story
- SW-1198 - Introduce new annotation deprecating legacy methods in API
- SW-1209 - Rename the 'predictionCol' model parameter to 'labelCol'
- SW-1226 - Introduce mechanism for enabling backward compatibility of MOJO files when properties are renamed
New Feature
- SW-1193 - Expose weights_column parameter
Improvement
- SW-1188 - RSparkling: Add ability to add authentication details when calling h2o_context(sc)
- SW-1190 - Improve hint description for disabling automatic usage of broadcast joins
- SW-1199 - Improve memory efficiency of H2OMOJOPipelineModel
- SW-1202 - Simplify Sparkling Water build
- SW-1204 - Fix formating in python tests
- SW-1208 - Create pysparkling tests report file if it does not exist
- SW-1210 - Add fold column to python and scala pipelines
- SW-1211 - Automatically download H2O Wheel
- SW-1213 - Upgrade to H2O 3.24.0.2
- SW-1214 - Remove PySparkling six dependency as it was removed in H2O
- SW-1215 - Automatically generate PySparkling README
- SW-1217 - Automatically generate last pieces of doc subproject
- SW-1219 - Remove suport for testing external cluster in manual mode
- SW-1221 - Remove unnecessary branch check
- SW-1222 - Remove duplicate readme file (contains old info & the correct info is in doc)
- SW-1223 - Remove confusing meetup dir
- SW-1224 - Upgrade to Gradle 5.3.1
- SW-1228 - Rename the 'ignoredColumns' parameter of H2OAutoML to 'ignoredCols'
- SW-1229 - Remove dependencies to Scala 2.10
- SW-1236 - Reformat few python classes
- SW-1238 - Parametrize EMR version in templates generation
- SW-1239 - Remove old README and DEVEL doc files (not just pointer to new doc)
- SW-1240 - Use minSupportedJava for source and target compatibility in build.gradle

v2.2.38 (2019-04-03)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/38/index.html

Bug
- SW-1162 - Exception when there is a column with BOOLEAN type in dataset during H2OMOJOModel transformation
- SW-1177 - In Pysparkling script, setting –driver-class-path influences the environment
- SW-1178 - Upgrade to h2O 3.24.0.1
- SW-1180 - Use specific metrics in grid search, in the same way as H2O Grid
- SW-1181 - Document off heap memory configuration for Spark in Standalone mode/IBM conductor
- SW-1182 - Fix random project name generation in H2OAutoML Spark Wrapper
New Feature
- SW-1167 - Expose search_criteria for H2OGridSearch
- SW-1174 - expose H2OGridSearch models
- SW-1183 - Add includeAlgos to H2o AutoML pipeline stage & ability to ignore XGBoost
Improvement
- SW-1164 - Add Sparkling Water to Jupyter spark/pyspark kernels in EMR terraform template
- SW-1171 - Upgrade build to Gradle 5.2.1
- SW-1175 - Integrate with H2O native hive support

v2.2.37 (2019-03-15)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/37/index.html

Bug
- SW-1163 - Expose missing variables in shared TF EMR SW tamplate
Improvement
- SW-1145 - Start jupyter notebook with Scala & Python Spark in AWS EMR Terraform template
- SW-1165 - Upgrade to H2O 3.22.1.6

v2.2.36 (2019-03-07)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/36/index.html

Bug
- SW-1150 - hc.stop() shows 'exit' not defined error
- SW-1152 - Fix RSparkling in case the jars are being fetched from maven
- SW-1156 - H2OXgboost pipeline stage does not define updateH2OParams method
- SW-1159 - Unique project name in automl to avoid sharing one leaderboard
- SW-1161 - Fix grid search pipeline step on pyspark side
Improvement
- SW-1052 - Document teraform scripts for AWS
- SW-1089 - Document using Google Cloud Storage In Sparkling Water
- SW-1135 - Speed up conversion between sparse spark vectors and h2o frames by using sparse new chunk
- SW-1141 - Improve terraform templates for AWS EMR and make them part of the release process
- SW-1149 - Allow login via ssh to created cluster using terraform
- SW-1153 - Add H2OGridSearch pipeline stage to PySpark
- SW-1155 - Test GBM Grid Search Scala pipeline step
- SW-1158 - Generalize H2OGridSearch Pipeline step to support other available algos
- SW-1160 - Upgrade to H2O 3.22.1.5

v2.2.35 (2019-02-18)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/35/index.html

Bug
- SW-1136 - Fix bug affecting loading pipeline in python when stored in scala
- SW-1138 - Fix several cases in spark vector -> h2o conversion
Improvement
- SW-1134 - Add H2OGLM Wrapper to Sparkling Water
- SW-1139 - Update mojo2 to 0.3.16
- SW-1143 - Fix s3 bootstrap templates for nightly builds
- SW-1144 - Upgrade to H2O 3.22.1.4

v2.2.34 (2019-01-29)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/34/index.html

Bug
- SW-1133 - Upgrade to H2O 3.22.1.3

v2.2.33 (2019-01-21)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/33/index.html

Bug
- SW-1129 - Fix support for unsupervised mojo models
Improvement
- SW-1101 - Update code to work with latest jetty changes
- SW-1127 - Upgrade H2O to 3.22.1.2

v2.2.32 (2019-01-17)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/32/index.html

Bug
- SW-1116 - Cannot serialize DAI model
Improvement
- SW-1113 - Update to H2O 3.22.0.5
- SW-1115 - Enable tabs in the documentation based on the language
- SW-1120 - Prepare Terraform scripts for Sparkling Water on EMR
- SW-1121 - Use getTimestamp method instead of _timestamp directly
- SW-1125 - Upgrade to Spark 2.2.3

v2.2.31 (2019-01-08)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/31/index.html

Bug
- SW-1107 - NullPointerException at water.H2ONode.openChan(H2ONode.java:417) after upgrade to H2O 3.22.0.3
- SW-1110 - Fix test suite to test PySparkling YARN integration tests on external backend as well
Task
- SW-1109 - Docs: Change copyright year in docs to include 2019
Improvement
- SW-464 - Publish PySparkling as conda package
- SW-1111 - Update H2O to 3.22.0.4

v2.2.30 (2018-12-27)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/30/index.html

Bug
- SW-1084 - Documentation link does not work on the Nightly Bleeding Edge download page
- SW-1100 - Fix Travis builds
- SW-1102 - Fix Travis builds (test just scala unit tests)
Improvement
- SW-464 - Publish PySparkling as conda package
- SW-1080 - Fix deprecation warning regarding automl -> AutoML
- SW-1090 - Upgrade shadowJar plugin
- SW-1091 - Upgrade to Gradle 5.0
- SW-1092 - Updates to streaming app
- SW-1093 - Update to H2O 3.22.0.3
- SW-1095 - Enable GCS in Sparkling Water
- SW-1097 - Properly integrate GCS with Sparkling Water, including test in PySparkling
- SW-1106 - Remove deprecated Gradle option in Gradle 5
Docs
- SW-1083 - Add Installation and Starting instructions to the docs

v2.2.29 (2018-11-27)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/29/index.html

Improvement
- SW-1078 - Upgrade H2O to 3.22.0.2

v2.2.28 (2018-10-27)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/28/index.html

Bug
- SW-1071 - Fallback to original IP discovery in case we can't find the same network
- SW-1072 - Fix handling time column for mojo pipeline
- SW-1073 - Upgrade MOJO to 0.3.17
Improvement
- SW-1045 - Upgrade H2O to 3.22.0.1

v2.2.27 (2018-10-17)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/27/index.html

Bug
- SW-930 - Enable AutoML tests in Sparkling Water
- SW-1065 - Fix isssue with empty queue name by default
- SW-1066 - In PySparkling, don't reconnect if already connected
- SW-1068 - Fix warning in doc
Improvement
- SW-1057 - Sparkling shell ignores parameters after last updates
- SW-1058 - Automatic detection of client ip in external backend
- SW-1059 - Pysparkling in external backend, manual mode stops the backend cluster, but the cluster should be left intact
- SW-1060 - Create nightly release for 2.1, 2.2 and 2.3
- SW-1061 - Upgrade to Mojo 0.3.15
- SW-1062 - Don't expose mojo internal types
- SW-1063 - More explicit checks for valid values of Backend mode and external backend start mode
- SW-1064 - Expose run_as_user for External H2O Backend
- SW-1069 - Upgrade H2O to 3.20.0.10

v2.2.26 (2018-10-02)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/26/index.html

Bug
- SW-1041 - Fix passing –jars to sparkling-shell
- SW-1042 - More robust check for python package in PySparkling shell
- SW-1048 - Add missing six dependency to setup.py for PySparkling
Improvement
- SW-1043 - Mojo pipeline with multiple output columns (and also with dots in the names) does not work in SW
- SW-1054 - Upgrade H2O dependency to 3.20.0.9

v2.2.25 (2018-09-24)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/25/index.html

New Feature
- SW-1020 - Expose leaderboard on H2OAutoML
- SW-1022 - Display Release creation date on the download page
Improvement
- SW-1024 - remove call to ./gradlew –help in jenkins pipeline
- SW-1025 - Ensure that release does not depend on build id
- SW-1030 - [RSparkling] In case only path to SW jar file is specified, discover the version from JAR file instead of requiring it as parameter
- SW-1031 - Enable installation ot RSparkling using devtools from Github repo
- SW-1032 - Upgrade mojo pipeline to 0.13.2
- SW-1033 - Document automatic certificate creation for Flow UI
- SW-1034 - PySparkling fails if we specify https argument as part of getOrCreate()
- SW-1035 - Document using s3a and s3n on Sparkling Water
- SW-1036 - Upgrade to H2O 3.20.0.8
- SW-1038 - The shell script bin/pysparkling should print missing dependencies
- SW-1039 - Upgrade Gradle to 4.10.2
Docs
- SW-1018 - Fix link to Installing RSparkling on Windows

v2.2.24 (2018-09-14)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/24/index.html

New Feature
- SW-1023 - Upgrade Gradle to 4.10.1
Improvement
- SW-1019 - Upgrade H2O to 3.20.0.7
- SW-1027 - Revert Upgrade to Gradle 4.10.1(bug in Gradle) and upgrade to Gradle 4.0
- SW-1028 - Update docs and mention that ORC is supported
Docs
- SW-1017 - Docs: Add Parquet to list of supported data formats

v2.2.23 (2018-08-28)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/23/index.html

Bug
- SW-270 - Add test for RDD[TimeStamp] -> H2OFrame[Time] -> RDD[Timestamp] conversion
- SW-319 - SVMModelTest is failing
- SW-986 - Fix links on RSparkling Readme page
- SW-996 - Fix typos in documentation
- SW-997 - Fix javadoc on JavaH2OContext
- SW-1000 - Setting context path in pysparkling fails to launch h2o
- SW-1001 - RSparkling does not respect context path
- SW-1002 - Automatically generate the keystore for H2O Flow ssl (self-signed certificates)
- SW-1003 - When running in Local mode, we ignore some configuration
- SW-1004 - Fix context path value checks
- SW-1005 - Use correct scheme in sparkling water when ssl on flow is enabled
- SW-1006 - Fix context path setting on RSparkling
- SW-1015 - Add context path after value of spark.ext.h2o.client.flow.baseurl.override when specified
New Feature
- SW-980 - Integrate XGBoost in Sparkling Water
- SW-1012 - Sparkling water External Backend Support in kerberized cluster
Task
- SW-988 - Add to docs that pysparkling has a new dependency pyspark
Improvement
- SW-175 - JavaH2OContext#asRDD implementation is missing
- SW-920 - Sparkling Water/RSparkling needs to declare additional repository
- SW-989 - Improve Scala Doc API of the support classes
- SW-991 - Update Gradle Spinx libraries - faster documentation builds
- SW-992 - Create abstract class from creating parameters from Enum for Sparkling Water pipelines
- SW-993 - [PySparkling] Fix Wrong H2O version detection on latest bundled H2Os
- SW-994 - Add timeouts & retries for docker pull
- SW-998 - Document using PySparkling on the edge node ( EMR)
- SW-1007 - Upgrade H2O to 3.20.0.6
- SW-1011 - Fix EMR bootstrap scripts
- SW-1013 - Add option which can be used to change the flow address which is printed out after H2OConetext started
- SW-1014 - Document how to run Sparkling Water on kerberized cluster

v2.2.22 (2018-08-09)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/22/index.html

Bug
- SW-971 - Change maintainer of RSparkling to jakub@h2o.ai
- SW-972 - Fix Content of RSparkling release table
- SW-973 - Allow passing custom cars when running ./bin/sparkling/shell
- SW-975 - Fix CRAN issues of Rsparkling
- SW-981 - Fix wrong comparison of versions when detecing other h2o versions in PySparkling
- SW-982 - Set up client_disconnect_timeout correctly in context on External backend, auto mode
- SW-983 - Fix missing mojo impl artifact when running pysparkling tests in jenkins
Task
- SW-633 - Add to doc that 100 columns are displayed in the preview data by default
Improvement
- SW-528 - Update PySparkling Notebooks to work for Python 3
- SW-548 - List nodes and driver memory in Spark UI - SParkling Water Tab
- SW-910 - Use Mojo Pipeline API in Sparkling Water
- SW-969 - Port documentation for mojo pipeline on Spark to SW repo
- SW-970 - Upgrade Mojo 2 in SW to 0.11.0
- SW-976 - Upgrade H2O to 3.20.0.5
- SW-977 - Need ability to disable Flow UI for Sparkling-Water
- SW-979 - Verify that we are running on correct Spark for PySparkling at init time
- SW-984 - Cache also test and runtime dependencies in docker image
Docs
- SW-946 - Add "How to" for using Sparkling Water on Google Cloud Dataproc

v2.2.21 (2018-08-01)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/21/index.html

Bug
- SW-903 - Automate releases of RSparkling and create release pipeline for this release proccess
- SW-911 - Add missing repository to the documentation
- SW-944 - Fix Sphinx gradle plugin, the latest version does not work
- SW-945 - Stabilize releasing to Nexus Repository
- SW-953 - Do not stop external H2O backend in case of manual start mode
- SW-958 - Fix RSparkling README style issues
- SW-959 - Fix address for fetching H2O R package in nightly tests
- SW-961 - Add option to ignore SPARK_PUBLIC_DNS
- SW-962 - Add option which ensures that items in flatfile are translated to IP address
- SW-967 - Deprecate old behaviour of mojo pipeline output in SW
Improvement
- SW-233 - Warn if user's h2o in python env is different then the one bundled in pysparkling
- SW-921 - Move Rsparkling to Sparkling Water repo
- SW-941 - Upgrade Gradle to 4.9
- SW-952 - Fix issues when stopping Sparkling Water (Scala) in yarn-cluster mode for external Backend
- SW-957 - RSparkling should run tests in both, external and internal mode
- SW-963 - Upgrade H2O to 3.20.0.4
- SW-965 - Expose port offset in Sparkling Water
- SW-968 - Remove confusing message about stopping H2OContext in PySparkling

v2.2.20 (2018-07-16)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/20/index.html

Bug
- SW-902 - Upgrade Gradle to 4.8.1
- SW-904 - Upgrade Mojo2 version to 0.10.7
- SW-908 - Exclude Hadoop dependencies as they are provided by Spark
- SW-909 - Fix issues when stopping Sparkling Water (Scala) in yarn-cluster mode
- SW-925 - Fix missing aposthrope in documentation
- SW-929 - Disable temporarily AutoML tests in Sparkling Water
New Feature
- SW-826 - Implement Synchronous and Asynchronous Scala cell behaviour
Improvement
- SW-846 - Don't parse types again when passing data to mojo pipeline
- SW-886 - Several Scala cell improvements in H2O flow
- SW-887 - Make sure that we can use schemes unsupported by H2O in H2O Confoguration
- SW-889 - Port AWS preparation scripts into SW codebase
- SW-894 - Add support for queuing of Scala cell jobs
- SW-914 - Wrong Spark version in documentation
- SW-917 - Dockerize Sparkling Water release pipeline
- SW-919 - Clean gradle build with regards to mojo2
- SW-922 - Upgrade H2O to 3.20.0.3
- SW-928 - Expose AutoML max models
- SW-933 - Upgrade Spark to 2.2.2
Docs
- SW-878 - Add section for using Sparkling Water with AWS

v2.2.19 (2018-06-18)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/19/index.html

Improvement
- SW-885 - Upgrade H2O to 3.20.0.2

v2.2.18 (2018-06-18)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/18/index.html

Bug
- SW-861 - Upgrade Gradle to 4.8 (publishing plugin)
- SW-872 - Fix reference to local-cluster on download page
- SW-880 - Update Hadoop version on download page
- SW-881 - Fix Script tests on Dockerized Jenkins infrastructure
- SW-882 - Call h2oContext.stop after ham or spam Scala example
- SW-883 - Add mising description in publish.gradle
Improvement
- SW-860 - Modify the hadoop launch command on download page
- SW-873 - Upgrade H2O to 3.20.0.1
- SW-874 - Update Mojo2 to 0.10.4
- SW-876 - FIx local PySparkling integtest on jenkins infrastracture
- SW-879 - Print output of script tests

v2.2.17 (2018-06-13)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/17/index.html

Bug
- SW-712 - Test non-distributed sparkling water tests in docker
- SW-850 - Expose methods to get input/output names in H2OMOJOPipelineModel
- SW-859 - Print Warning when spark-home is defined on PATH
- SW-862 - Create & fix test in PySparkling for named mojo columns
- SW-864 - Fix & more readable test
- SW-865 - Better Naming of the UDF method to obtain predictions
- SW-869 - Add repository to build required by xgboost-predictor
Story
- SW-856 - Upgrade Mojo2 to latest version
Improvement
- SW-839 - Verify that Spark time column representation can be digested by Mojo2
- SW-848 - Document Kerberos on Sparkling Water
- SW-849 - Update use from maven on sparkling water download page
- SW-851 - Make use of output types when creating Spark DataFrame out of mojo2 predicted values
- SW-852 - Create spark UDF used to extract predicted values
- SW-853 - Sparkling Water py should require pyspark dependency
- SW-854 - Upgrade MojoPipeline to 0.10.0
- SW-855 - Upgrade H2O to 3.18.0.11

v2.2.16 (2018-05-23)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/16/index.html

Bug
- SW-842 - Enforce system level properties in SW
Improvement
- SW-845 - Upgrade H2O to 3.18.0.10
- SW-847 - Remove GA from Sparkling Water

v2.2.15 (2018-05-18)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/15/index.html

Bug
- SW-836 - Add support for converting empty dataframe/RDD in Python and Scala to H2OFrame
- SW-841 - Remove withCustomCommitsState in pipelines as it's now duplicating Github
- SW-843 - Fix data obtaining for mojo pipeline
- SW-844 - Upgrade Mojo pipeline to 0.9.9

v2.2.14 (2018-05-15)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/14/index.html

Bug
- SW-817 - Enable running MOJO spark pipeline without H2O init
- SW-825 - Local creation of Sparkling Water does not work anymore.
- SW-831 - Check shape of H2O frame after the conversion from Spark frame
- SW-834 - External Backend stored sparse vector values incorrectly
Improvement
- SW-829 - Type checking in PySparkling pipelines
- SW-832 - Small refactoring in identifiers
- SW-833 - Explicitly set source and target java versions
- SW-837 - Upgrade H2O to 3.18.0.9
- SW-838 - Upgrade Mojo pipeline dependency to 0.9.8
- SW-840 - Add test checking column names and types between spark and mojo2

v2.2.13 (2018-05-02)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/13/index.html

Bug
- SW-574 - Process steam handle and use it for connection to external h2o cluster
- SW-822 - Require correct colorama version
- SW-823 - Fix Windows starting scripts
- SW-824 - Fix NPE in mojo pipeline predictions
New Feature
- SW-827 - Change color highlight in scala cell as it is too dark
Improvement
- SW-815 - Upgrade H2O to 3.18.0.8
- SW-816 - Update Mojo2 dependency to one which is compatible with Java7
- SW-818 - Spark Pipeline imports do not work in PySparkling
- SW-819 - Add ability to convert specific columns to categoricals in Sparkling Water pipelines
- SW-820 - Sparkling Water pipelines add duplicate response column to the list of features

v2.2.12 (2018-04-19)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/12/index.html

Bug
- SW-672 - Enable using sparkling water maven packages in databricks cloud
- SW-787 - Documentation fixes
- SW-790 - Add missing seed argument to H2OAutoml pipeline step
- SW-794 - Point to proper web-based docs
- SW-796 - Use parquet provided by Spark
- SW-797 - Automatically update redirect table as part of release pipeline
- SW-806 - Fix exporting and importing of pipeline steps and mojo models to and from HDFS
Improvement
- SW-772 - Integrate & Test Mojo Pipeline with Sparkling Water
- SW-789 - Upgrade H2O to 3.18.0.7
- SW-791 - Expose context_path in Sparkling Water
- SW-793 - Create additional test verifying that the new light endpoint works as expected
- SW-798 - Additional link to documentation
- SW-800 - Remove references to Sparkling Water 2.0
- SW-804 - Reduce time of H2OAutoml step in pipeline tests to 1 minute
- SW-808 - Upgrade to Gradle 4.7

v2.2.11 (2018-03-29)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/11/index.html

Bug
- SW-696 - Intermittent script test issue on external backend
- SW-726 - Mark Spark dependencies as provided on artefacts published to maven
- SW-740 - Increase timeout for conversion in pyunit test for external cluster
- SW-760 - Fix doc artefact publication
- SW-763 - Remove support for downloading H2O logs from Spark UI
- SW-766 - Fix coding style issue
- SW-769 - Fix import
- SW-776 - sparkling water from maven does not know the stacktrace_collector_interval option
- SW-778 - Handle nulls properly in H2OMojoModel
- SW-783 - Make H2OAutoML pipeline tests deterministic by setting the seed
New Feature
- SW-722 - [PySparkling] Check for correct data type as part of as_h2o_frame
Improvement
- SW-733 - Parametrize pipeline scripts to be able to specify different algorithms
- SW-746 - Log chunk layout after the conversion of data to external H2O cluster
- SW-755 - Document GBM Grid Search Pipeline Step
- SW-765 - Remove test artefacts from the sparkling-water assembly
- SW-768 - Add missing import
- SW-771 - Travis edits - no longer need the workaround for JDK7
- SW-773 - Don't use default value for output dir in external backend, it's not required
- SW-780 - Upgrade H2O to 3.18.0.5
Docs
- SW-775 - Fix link for documentation on DEVEL.md

v2.2.10 (2018-03-08)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/10/index.html

Bug
- SW-739 - Sparkling Water Doc artefact is still missing Scala version
- SW-742 - Fix setting up node network mask on external cluster
- SW-743 - Allow to set LDAP and different security options in external backend as well
- SW-747 - Fix bug in documentation for manual mode of external backend
- SW-757 - Fix tests after enabling the stack-trace collection
Improvement
- SW-744 - Document how to use Sparkling Water with LDAP in Sparkling Water docs
- SW-745 - Expose Grid search as Spark pipeline step in the Scala API
- SW-748 - Upgrade to Gradle 4.6
- SW-752 - Collect stack traces on each h2o node as part of log collecting extension
- SW-754 - Upgrade H2O to 3.18.0.3
- SW-756 - Upgrade H2O to 3.18.0.4
Docs
- SW-753 - Add "How to" for changing the default H2O port

v2.2.9 (2018-02-26)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/9/index.html

Bug
- SW-723 - Sparkling water doc artefact is missing scala version
- SW-727 - Improve method for downloading H2O logs
- SW-728 - Use new light endpoint introduced in 3.18.0.1
- SW-734 - Make sure we use the unique key names in split method
- SW-736 - Document how to download logs on Databricks cluster
- SW-737 - Expose downloadH2OLogs on H2OContext in PySparkling
- SW-738 - Move spark.ext.h2o.node.network.mask setter to SharedArguments
Improvement
- SW-702 - Create Spark Transformer for AutoML
- SW-725 - create an an equvivalent of h2o.download_all_logs in scala
- SW-730 - Upgrade H2O to 3.18.0.2

v2.2.8 (2018-02-14)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/8/index.html

Technical task
- SW-652 - Deliver SW documentation in HTML output
Bug
- SW-685 - Fix Typo in documentation
- SW-695 - Make printHadoopDistributions gradle task available again for testing
- SW-701 - Kill the client when one of the h2o nodes went OOM in external mode
- SW-706 - Fix pysparkling.ml import for non-interactive sessions
- SW-707 - parquet import fails on HDP with Spark 2.0 (azure hdi cluster)
- SW-708 - Make sure H2OMojoModel does not required H2OContext to be initialized
- SW-709 - Fix mojo predictions tests
- SW-710 - In PySparkling pipelines, ensure that if users pass integer to double type we handle that correctly for all possible double values
- SW-713 - Write a simple test for parquet import in Sparkling Water
- SW-714 - Add option to H2OModel pipeline step allowing us to convert unknown categoricals to NAs
- SW-715 - Fix driverif configuration on the external backend
Improvement
- SW-606 - Verify & Document run of RSparkling on top of Databricks Azure cluster
- SW-678 - Document how to change log location
- SW-683 - H2OContext can't be initalized on Databricks cloud
- SW-686 - Fix typo in documentation
- SW-687 - Upgrade Gradle to 4.5
- SW-688 - Update docs - SparklyR supports Spark 2.2.1 in the latest release
- SW-690 - Log Sparkling Water version during startup of Sparkling Water
- SW-693 - Allow to set driverIf on external H2O backend
- SW-694 - Fix creation of Extended JAR in gradle task
- SW-700 - Report Yarn App ID of spark application in H2OContext
- SW-703 - Upload generated sphinx documentation to S3
- SW-704 - Update links on the download page to point to the new docs
- SW-705 - Increase memory for JUNIT tests
- SW-718 - Upgrade to Gradle 4.5.1
- SW-719 - Upgrade to H2O 3.18.0.1
- SW-720 - Fix parquet import test on external backend
Docs
- SW-697 - Final updates for Sparkling Water html output
- SW-698 - Update "Contributing" section in Sparkling Water

v2.2.7 (2018-01-18)¶

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/7/index.html

Bug
- SW-273 - Remove workaround introduced by SW-272 for yarn/cluster mode
- SW-551 - Remove hotfix introduced by [SW-541] and implement proper fix
- SW-661 - Use always correct Spark version on the R download page
- SW-662 - Remove extra files that got into the repo
- SW-666 - Kill the cluster when a new executors joins in the internal backend
- SW-668 - Generate download link as part of the release notes
- SW-669 - Remove mentions of local-cluster in public docs
- SW-670 - Deprecated call in H2OContextInitDemo
- SW-671 - Fix jenkinsfile for builds again specific h2o branches
Improvement
- SW-674 - Update H2O to 3.16.0.4
- SW-675 - Tiny clean up of the release code
- SW-679 - Cleaner release script
- SW-680 - Ensure S3 in release pipeline does depend only on credentials provided from Jenkins
- SW-681 - Separate releasing on Github and Publishing artifacts

v2.2.6 (2018-01-03)¶

Bug
- SW-627 - [PySparkling] calling as_spark_frame for the second time results in exception
- SW-630 - Fix ham or spam flow to reflect latest changes in pipelines
- SW-631 - Ensure that we do not access RDDs in pipelines ( to unblock the deployment)
- SW-645 - Fix H2OInterpreter on Scala 2.10
- SW-646 - Fix incosistencies in ham or spam examples between scala and python
- SW-647 - PySparkling shell is failing in Spark 2.2.0
- SW-648 - Fix ham or spam pipeline tests
- SW-649 - Fix ham or spam tests for deeplearning pipeline
Improvement
- SW-608 - Measure time of conversions to H2OFrame in debug mode
- SW-612 - Port all arguments available to Scala ML to PySparkling ML
- SW-617 - Support for exporting mojo to hdfs
- SW-632 - Dump full spark configuration during H2OContext.getOrCreate into DEBUG
- SW-634 - Integrate with Spark 2.2.1
- SW-635 - Fix wrong instruction at PySparkling download page
- SW-637 - Create new DataFrame with new schema when it actually contain any dot in names
- SW-638 - Port release script into the sw repo
- SW-639 - Use persist layer for exportPOJOModel
- SW-640 - export H2OMOJOMOdel.createFromMOJO to pysparkling
- SW-642 - Create test for mojo predictions in PySparkling
- SW-643 - Add tests for H2ODeeplearning in Scala and Python and Fix potential problems
- SW-644 - Log spark configuration to INFO level
- SW-650 - Upgrade Gradle to 4.4.1
- SW-656 - Upgrade ShadowJar to 2.0.2

v2.2.5 (2017-12-11)¶

Bug
- SW-615 - pysparkling.__version__ returns incorrectly ‘SUBST_PROJECT_VERSION’
- SW-616 - PySparkling fails on python 3.6 because long time does not exist in python 3.6
- SW-621 - PySParkling failing on Python3.6
- SW-624 - Python build does not support H2O_PYTHON_WHEEL when building against h2o older then 3.16.0.1
- SW-628 - PySparkling fails when installed from pypi
Improvement
- SW-626 - Upgrade Gradle to 4.4

v2.2.4 (2017-12-01)¶

Bug
- SW-602 - conversion of sparse data DataFrame to H2OFrame is slow
- SW-620 - Fix obtaining version from bundled h2o inside pysparkling
Improvement
- SW-613 - Append dynamic allocation option into SW tuning documentation.
- SW-618 - Integration with H2O 3.16.0.2

v2.2.3 (2017-11-25)¶

Bug
- SW-320 - H2OConfTest Python test blocks test run
- SW-499 - BinaryType handling is not implemented in SparkDataFrameConverter
- SW-535 - asH2OFrame gives error if column names have DOT in it
- SW-547 - Don’t use md5skip in external mode
- SW-569 - pysparkling: h2o on exit does not shut down cleanly
- SW-572 - Additional fix for [SW-571]
- SW-573 - Minor Gradle build improvements and fixes
- SW-575 - Incorrect comment in hamOrSpamMojo pipeline
- SW-576 - Cleanup pysparkling test infrastructure
- SW-577 - Fix conditions in jenkins file
- SW-580 - Fix composite build in Jenkins
- SW-581 - Fix H2OConf test on external cluster
- SW-582 - Opening Chicago Crime Demo Notebook errors on the first opening
- SW-584 - Create extended directory automatically
- SW-588 - Fix links in README
- SW-589 - Wrap stages in try finally in jenkins file
- SW-592 - Properly pass all parameters to algorithm
- SW-593 - H2Conf cannot be initialized on windows
- SW-594 - Gradle ml submodule reports success even though tests fail
- SW-595 - Fix ML tests
New Feature
- SW-519 - Introduce SW Models into Spark python pipelines
Task
- SW-609 - Upgrade H2O dependency to 3.16.0.1
Improvement
- SW-318 - Keep H2O version inside sparklin-water-core.jar and provide utility to query it
- SW-420 - Shell scripts miss-leading error message
- SW-504 - Provides Sparkling Water Spark Uber package which can be used in –packages
- SW-570 - Stop previous jobs in jenkins in case of PR
- SW-571 - In PySparkling, getOrCreate(spark) still incorrectly complains that we should use spark session
- SW-583 - Upgrade to Gradle 4.3
- SW-585 - Add the custom commit status for internal and external pipelines
- SW-586 - [ML] Remove some duplicities, enable mojo for deep learning
- SW-590 - Replace deprecated method call in ChicagoCrime python example
- SW-591 - Repl doesn’t require H2O dependencies to compile
- SW-596 - Minor build improvements
- SW-603 - Upgrade Gradle to 4.3.1
- SW-605 - addFiles doesn’t accept sparkSession
- SW-610 - Change default client mode to INFO, let user to change it at runtime

v2.2.2 (2017-10-23)¶

Bug
- SW-555 - Fix documentation issue in PySparkling
- SW-558 - Increase default value for client connection retry timeout in
- SW-560 - SW documentation for nthreads is inconsistent with code
- SW-561 - Fix reporting artefacts in Jenkins and remove use of h2o-3-shared-lib
- SW-564 - Clean test workspace in jenkins
- SW-565 - Fix creation of extended jar in jenkins
- SW-567 - Fix failing tests on external backend
- SW-568 - Remove obsolete and failing idea configuration
- SW-559 - GLM fails to build model when weights are specified
Improvement
- SW-557 - Create 2 jenkins files ( for internal and external backend ) backed by configurable pipeline
- SW-562 - Disable web on external H2O nodes in external cluster mode
- SW-563 - In external cluster mode, print also YARN job ID of the external cluster once context is available
- SW-566 - Upgrade H2O to 3.14.0.7
- SW-553 - Improve handling of sparse vectors in internal cluster

v2.2.1 (2017-10-10)¶

Bug
- SW-423 - Tests of External Cluster mode fails
- SW-516 - External cluster improperly convert RDD[ml.linalg.Vector]
- SW-524 - Ensure Jenkins uses Java 8 for Sparkling Water 2.2.x
- SW-525 - Don’t use GPU nodes for sparkling water testing in Jenkins
- SW-526 - Add missing when clause to scripts test stage in Jenkinsfile
- SW-527 - Use dX cluster for Jenkins testing
- SW-529 - Code defect in Scala example
- SW-531 - Use code which is compatible between Scala 2.10 and 2.11
- SW-532 - Make auto mode in external cluster default for tests in jenkins
- SW-534 - Ensure that all tests run on both, internal and external backends
- SW-536 - Allow to test sparkling water against specific h2o branch
- SW-537 - Update Gradle to 4.2RC2
- SW-538 - Fix problem in Jenkinsfile where H2O_HOME has higher priority then H2O_PYTHON_WHEEL
- SW-539 - Fix PySparkling issue when running multiple times on the same node
- SW-541 - Model training hangs in SW
- SW-542 - Sparkling Water does not support parquet import
- SW-552 - Fix documentation bug
New Feature
- SW-521 - Fix typo in documentation
- SW-523 - Use linux label to determine which nodes are used for Jenkins testing
- SW-533 - In external cluster, remove notification file at the end. This affects nothing, it is just cleanup.
Improvement
- SW-543 - Upgrade Gradle to 4.2
- SW-544 - Improve exception in ExternalH2OBackend
- SW-545 - Stop H2O in afterAll in tests
- SW-546 - Add sw version to name of h2odriver obtained using get-extended-h2o script
- SW-549 - Upgrade gradle to 4.2.1
- SW-550 - Upgrade H2O to 3.14.0.6

v2.2.0 (2017-08-23)¶

Bug
- SW-449 - Support Sparse Data during spark-h2o conversions
- SW-510 - the link Demo Example from Git is broken on the download page
New Feature
- SW-481 - MOJO for Spark SVM
- SW-497 - Integration with Spark 2.2
Improvement
- SW-395 - bin/sparkling-shell should fail if assembly jar file does not exist
- SW-471 - Use mojo in pipelines if possible, remove H2OPipeline and OneTimeTransformers
- SW-512 - Make JenkinsFile up-to-date with sparkling_yarn_branch
- SW-513 - Upgrade to Gradle 4.1
- SW-514 - Upgrade H2O to 3.14.0.2

v2.1.x (2017-03-02)¶

Sparkling Water 2.1 brings support of Spark 2.1.
For detailed changelog, please read rel-2.1/CHANGELOG.

v2.0.x (2016-09-26)¶

Sparkling Water 2.0 brings support of Spark 2.0.
For detailed changelog, please read rel-2.0/CHANGELOG.

v1.6.x (2016-03-15)¶

Sparkling Water 1.6 brings support of Spark 1.6.
For detailed changelog, please read rel-1.6/CHANGELOG.

v1.5.x (2015-09-28)¶

Sparkling Water 1.5 brings support of Spark 1.5.
For detailed changelog, please read rel-1.5/CHANGELOG.

v1.4.x (2015-07-06)¶

Sparkling Water 1.4 brings support of Spark 1.4.
For detailed changelog, please read rel-1.4/CHANGELOG.

v1.3.x (2015-05-25)¶

Sparkling Water 1.3 brings support of Spark 1.3.
For detailed changelog, please read rel-1.3/CHANGELOG.

v1.2.x (2015-05-18) and older¶

Sparkling Water 1.2 brings support of Spark 1.2.
For detailed changelog, please read rel-1.2/CHANGELOG.