Change Log

v2.2.9 (2018-02-26)

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/9/index.html

  • Bug
    • SW-723 - Sparkling water doc artefact is missing scala version
    • SW-727 - Improve method for downloading H2O logs
    • SW-728 - Use new light endpoint introduced in 3.18.0.1
    • SW-734 - Make sure we use the unique key names in split method
    • SW-736 - Document how to download logs on Databricks cluster
    • SW-737 - Expose downloadH2OLogs on H2OContext in PySparkling
    • SW-738 - Move spark.ext.h2o.node.network.mask setter to SharedArguments
  • Improvement
    • SW-702 - Create Spark Transformer for AutoML
    • SW-725 - create an an equvivalent of h2o.download_all_logs in scala
    • SW-730 - Upgrade H2O to 3.18.0.2

v2.2.8 (2018-02-14)

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/8/index.html

  • Technical task
    • SW-652 - Deliver SW documentation in HTML output
  • Bug
    • SW-685 - Fix Typo in documentation
    • SW-695 - Make printHadoopDistributions gradle task available again for testing
    • SW-701 - Kill the client when one of the h2o nodes went OOM in external mode
    • SW-706 - Fix pysparkling.ml import for non-interactive sessions
    • SW-707 - parquet import fails on HDP with Spark 2.0 (azure hdi cluster)
    • SW-708 - Make sure H2OMojoModel does not required H2OContext to be initialized
    • SW-709 - Fix mojo predictions tests
    • SW-710 - In PySparkling pipelines, ensure that if users pass integer to double type we handle that correctly for all possible double values
    • SW-713 - Write a simple test for parquet import in Sparkling Water
    • SW-714 - Add option to H2OModel pipeline step allowing us to convert unknown categoricals to NAs
    • SW-715 - Fix driverif configuration on the external backend
  • Improvement
    • SW-606 - Verify & Document run of RSparkling on top of Databricks Azure cluster
    • SW-678 - Document how to change log location
    • SW-683 - H2OContext can't be initalized on Databricks cloud
    • SW-686 - Fix typo in documentation
    • SW-687 - Upgrade Gradle to 4.5
    • SW-688 - Update docs - SparklyR supports Spark 2.2.1 in the latest release
    • SW-690 - Log Sparkling Water version during startup of Sparkling Water
    • SW-693 - Allow to set driverIf on external H2O backend
    • SW-694 - Fix creation of Extended JAR in gradle task
    • SW-700 - Report Yarn App ID of spark application in H2OContext
    • SW-703 - Upload generated sphinx documentation to S3
    • SW-704 - Update links on the download page to point to the new docs
    • SW-705 - Increase memory for JUNIT tests
    • SW-718 - Upgrade to Gradle 4.5.1
    • SW-719 - Upgrade to H2O 3.18.0.1
    • SW-720 - Fix parquet import test on external backend
  • Docs
    • SW-697 - Final updates for Sparkling Water html output
    • SW-698 - Update "Contributing" section in Sparkling Water

v2.2.7 (2018-01-18)

Download at: http://h2o-release.s3.amazonaws.com/sparkling-water/rel-2.2/7/index.html

  • Bug
    • SW-273 - Remove workaround introduced by SW-272 for yarn/cluster mode
    • SW-551 - Remove hotfix introduced by [SW-541] and implement proper fix
    • SW-661 - Use always correct Spark version on the R download page
    • SW-662 - Remove extra files that got into the repo
    • SW-666 - Kill the cluster when a new executors joins in the internal backend
    • SW-668 - Generate download link as part of the release notes
    • SW-669 - Remove mentions of local-cluster in public docs
    • SW-670 - Deprecated call in H2OContextInitDemo
    • SW-671 - Fix jenkinsfile for builds again specific h2o branches
  • Improvement
    • SW-674 - Update H2O to 3.16.0.4
    • SW-675 - Tiny clean up of the release code
    • SW-679 - Cleaner release script
    • SW-680 - Ensure S3 in release pipeline does depend only on credentials provided from Jenkins
    • SW-681 - Separate releasing on Github and Publishing artifacts

v2.2.6 (2018-01-03)

  • Bug
    • SW-627 - [PySparkling] calling as_spark_frame for the second time results in exception
    • SW-630 - Fix ham or spam flow to reflect latest changes in pipelines
    • SW-631 - Ensure that we do not access RDDs in pipelines ( to unblock the deployment)
    • SW-645 - Fix H2OInterpreter on Scala 2.10
    • SW-646 - Fix incosistencies in ham or spam examples between scala and python
    • SW-647 - PySparkling shell is failing in Spark 2.2.0
    • SW-648 - Fix ham or spam pipeline tests
    • SW-649 - Fix ham or spam tests for deeplearning pipeline
  • Improvement
    • SW-608 - Measure time of conversions to H2OFrame in debug mode
    • SW-612 - Port all arguments available to Scala ML to PySparkling ML
    • SW-617 - Support for exporting mojo to hdfs
    • SW-632 - Dump full spark configuration during H2OContext.getOrCreate into DEBUG
    • SW-634 - Integrate with Spark 2.2.1
    • SW-635 - Fix wrong instruction at PySparkling download page
    • SW-637 - Create new DataFrame with new schema when it actually contain any dot in names
    • SW-638 - Port release script into the sw repo
    • SW-639 - Use persist layer for exportPOJOModel
    • SW-640 - export H2OMOJOMOdel.createFromMOJO to pysparkling
    • SW-642 - Create test for mojo predictions in PySparkling
    • SW-643 - Add tests for H2ODeeplearning in Scala and Python and Fix potential problems
    • SW-644 - Log spark configuration to INFO level
    • SW-650 - Upgrade Gradle to 4.4.1
    • SW-656 - Upgrade ShadowJar to 2.0.2

v2.2.5 (2017-12-11)

  • Bug
    • SW-615 - pysparkling.__version__ returns incorrectly ‘SUBST_PROJECT_VERSION’
    • SW-616 - PySparkling fails on python 3.6 because long time does not exist in python 3.6
    • SW-621 - PySParkling failing on Python3.6
    • SW-624 - Python build does not support H2O_PYTHON_WHEEL when building against h2o older then 3.16.0.1
    • SW-628 - PySparkling fails when installed from pypi
  • Improvement
    • SW-626 - Upgrade Gradle to 4.4

v2.2.4 (2017-12-01)

  • Bug
    • SW-602 - conversion of sparse data DataFrame to H2OFrame is slow
    • SW-620 - Fix obtaining version from bundled h2o inside pysparkling
  • Improvement
    • SW-613 - Append dynamic allocation option into SW tuning documentation.
    • SW-618 - Integration with H2O 3.16.0.2

v2.2.3 (2017-11-25)

  • Bug
    • SW-320 - H2OConfTest Python test blocks test run
    • SW-499 - BinaryType handling is not implemented in SparkDataFrameConverter
    • SW-535 - asH2OFrame gives error if column names have DOT in it
    • SW-547 - Don’t use md5skip in external mode
    • SW-569 - pysparkling: h2o on exit does not shut down cleanly
    • SW-572 - Additional fix for [SW-571]
    • SW-573 - Minor Gradle build improvements and fixes
    • SW-575 - Incorrect comment in hamOrSpamMojo pipeline
    • SW-576 - Cleanup pysparkling test infrastructure
    • SW-577 - Fix conditions in jenkins file
    • SW-580 - Fix composite build in Jenkins
    • SW-581 - Fix H2OConf test on external cluster
    • SW-582 - Opening Chicago Crime Demo Notebook errors on the first opening
    • SW-584 - Create extended directory automatically
    • SW-588 - Fix links in README
    • SW-589 - Wrap stages in try finally in jenkins file
    • SW-592 - Properly pass all parameters to algorithm
    • SW-593 - H2Conf cannot be initialized on windows
    • SW-594 - Gradle ml submodule reports success even though tests fail
    • SW-595 - Fix ML tests
  • New Feature
    • SW-519 - Introduce SW Models into Spark python pipelines
  • Task
    • SW-609 - Upgrade H2O dependency to 3.16.0.1
  • Improvement
    • SW-318 - Keep H2O version inside sparklin-water-core.jar and provide utility to query it
    • SW-420 - Shell scripts miss-leading error message
    • SW-504 - Provides Sparkling Water Spark Uber package which can be used in –packages
    • SW-570 - Stop previous jobs in jenkins in case of PR
    • SW-571 - In PySparkling, getOrCreate(spark) still incorrectly complains that we should use spark session
    • SW-583 - Upgrade to Gradle 4.3
    • SW-585 - Add the custom commit status for internal and external pipelines
    • SW-586 - [ML] Remove some duplicities, enable mojo for deep learning
    • SW-590 - Replace deprecated method call in ChicagoCrime python example
    • SW-591 - Repl doesn’t require H2O dependencies to compile
    • SW-596 - Minor build improvements
    • SW-603 - Upgrade Gradle to 4.3.1
    • SW-605 - addFiles doesn’t accept sparkSession
    • SW-610 - Change default client mode to INFO, let user to change it at runtime

v2.2.2 (2017-10-23)

  • Bug
    • SW-555 - Fix documentation issue in PySparkling
    • SW-558 - Increase default value for client connection retry timeout in
    • SW-560 - SW documentation for nthreads is inconsistent with code
    • SW-561 - Fix reporting artefacts in Jenkins and remove use of h2o-3-shared-lib
    • SW-564 - Clean test workspace in jenkins
    • SW-565 - Fix creation of extended jar in jenkins
    • SW-567 - Fix failing tests on external backend
    • SW-568 - Remove obsolete and failing idea configuration
    • SW-559 - GLM fails to build model when weights are specified
  • Improvement
    • SW-557 - Create 2 jenkins files ( for internal and external backend ) backed by configurable pipeline
    • SW-562 - Disable web on external H2O nodes in external cluster mode
    • SW-563 - In external cluster mode, print also YARN job ID of the external cluster once context is available
    • SW-566 - Upgrade H2O to 3.14.0.7
    • SW-553 - Improve handling of sparse vectors in internal cluster

v2.2.1 (2017-10-10)

  • Bug
    • SW-423 - Tests of External Cluster mode fails
    • SW-516 - External cluster improperly convert RDD[ml.linalg.Vector]
    • SW-524 - Ensure Jenkins uses Java 8 for Sparkling Water 2.2.x
    • SW-525 - Don’t use GPU nodes for sparkling water testing in Jenkins
    • SW-526 - Add missing when clause to scripts test stage in Jenkinsfile
    • SW-527 - Use dX cluster for Jenkins testing
    • SW-529 - Code defect in Scala example
    • SW-531 - Use code which is compatible between Scala 2.10 and 2.11
    • SW-532 - Make auto mode in external cluster default for tests in jenkins
    • SW-534 - Ensure that all tests run on both, internal and external backends
    • SW-536 - Allow to test sparkling water against specific h2o branch
    • SW-537 - Update Gradle to 4.2RC2
    • SW-538 - Fix problem in Jenkinsfile where H2O_HOME has higher priority then H2O_PYTHON_WHEEL
    • SW-539 - Fix PySparkling issue when running multiple times on the same node
    • SW-541 - Model training hangs in SW
    • SW-542 - Sparkling Water does not support parquet import
    • SW-552 - Fix documentation bug
  • New Feature
    • SW-521 - Fix typo in documentation
    • SW-523 - Use linux label to determine which nodes are used for Jenkins testing
    • SW-533 - In external cluster, remove notification file at the end. This affects nothing, it is just cleanup.
  • Improvement
    • SW-543 - Upgrade Gradle to 4.2
    • SW-544 - Improve exception in ExternalH2OBackend
    • SW-545 - Stop H2O in afterAll in tests
    • SW-546 - Add sw version to name of h2odriver obtained using get-extended-h2o script
    • SW-549 - Upgrade gradle to 4.2.1
    • SW-550 - Upgrade H2O to 3.14.0.6

v2.2.0 (2017-08-23)

  • Bug
    • SW-449 - Support Sparse Data during spark-h2o conversions
    • SW-510 - the link Demo Example from Git is broken on the download page
  • New Feature
    • SW-481 - MOJO for Spark SVM
    • SW-497 - Integration with Spark 2.2
  • Improvement
    • SW-395 - bin/sparkling-shell should fail if assembly jar file does not exist
    • SW-471 - Use mojo in pipelines if possible, remove H2OPipeline and OneTimeTransformers
    • SW-512 - Make JenkinsFile up-to-date with sparkling_yarn_branch
    • SW-513 - Upgrade to Gradle 4.1
    • SW-514 - Upgrade H2O to 3.14.0.2

v2.1.x (2017-03-02)

  • Sparkling Water 2.1 brings support of Spark 2.1.
  • For detailed changelog, please read rel-2.1/CHANGELOG.

v2.0.x (2016-09-26)

  • Sparkling Water 2.0 brings support of Spark 2.0.
  • For detailed changelog, please read rel-2.0/CHANGELOG.

v1.6.x (2016-03-15)

  • Sparkling Water 1.6 brings support of Spark 1.6.
  • For detailed changelog, please read rel-1.6/CHANGELOG.

v1.5.x (2015-09-28)

  • Sparkling Water 1.5 brings support of Spark 1.5.
  • For detailed changelog, please read rel-1.5/CHANGELOG.

v1.4.x (2015-07-06)

  • Sparkling Water 1.4 brings support of Spark 1.4.
  • For detailed changelog, please read rel-1.4/CHANGELOG.

v1.3.x (2015-05-25)

  • Sparkling Water 1.3 brings support of Spark 1.3.
  • For detailed changelog, please read rel-1.3/CHANGELOG.

v1.2.x (2015-05-18) and older

  • Sparkling Water 1.2 brings support of Spark 1.2.
  • For detailed changelog, please read rel-1.2/CHANGELOG.