water

Class ExternalFrameWriterClient

java.lang.Object

water.ExternalFrameWriterClient

public final class ExternalFrameWriterClient
extends java.lang.Object

This class is used to create and write data to H2O Frames from non-H2O environments, such as Spark Executors.

Example usage of this class:

First we need to open the connection to H2O and initialize the writer:

 
 // Prepare expected bytes from Java Classes.
 // We don't specify vector types since they are deterministically inferred from the expected types
 byte[] expectedBytes = ExternalFrameUtils.prepareExpectedTypes(new Class[]{Boolean.class, Integer.class});
 ByteChannel channel = ExternalFrameUtils.getConnection("ip:port");
 ExternalFrameWriter writer = new ExternalFrameWriter(channel);
 writer.createChunks("frameName", expectedTypes, chunkIdx, numOfRowsToBeWritten);
 
 

Then we can write the data:

 int rowsWritten = 0;
 while(rowsWritten < totalNumOfRows){
  writer.sendBool(true);
  writer.sendInt(657);
 }
 
 

And at the end we need to make sure to force to code wait for all data to be written

 
 writer.waitUntilAllWritten();
 
 

Constructor Summary

Constructors

Constructor and Description
ExternalFrameWriterClient(java.nio.channels.ByteChannel channel) Initialize the External frame writer This method expects expected types in order to ensure we send the data in optimal way.

Method Summary

Modifier and Type	Method and Description
void	createChunks(java.lang.String frameKey, byte[] expectedTypes, int chunkId, int totalNumRows, int[] maxVecSizes) Create chunks on the h2o backend.
void	sendBoolean(boolean data)
void	sendByte(byte data)
void	sendChar(char data)
void	sendDenseVector(double[] values)
void	sendDouble(double data)
void	sendFloat(float data)
void	sendInt(int data)
void	sendLong(long data)
void	sendNA()
void	sendShort(short data)
void	sendSparseVector(int[] indices, double[] values)
void	sendString(java.lang.String data)
void	sendTimestamp(java.sql.Timestamp timestamp)
void	waitUntilAllWritten(int timeout) This method ensures the application waits for all bytes to be written before continuing in the control flow.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExternalFrameWriterClient

public ExternalFrameWriterClient(java.nio.channels.ByteChannel channel)

Initialize the External frame writer This method expects expected types in order to ensure we send the data in optimal way.

Parameters:

channel - communication channel to h2o node

Method Detail

createChunks

public void createChunks(java.lang.String frameKey,
                         byte[] expectedTypes,
                         int chunkId,
                         int totalNumRows,
                         int[] maxVecSizes)
                  throws java.io.IOException

Create chunks on the h2o backend. This method creates chunk in en empty frame.

Parameters:

frameKey - name of the frame

expectedTypes - expected types

chunkId - chunk index

totalNumRows - total number of rows which is about to be sent

Throws:

java.io.IOException

sendBoolean

public void sendBoolean(boolean data)
                 throws java.io.IOException

Throws:

java.io.IOException

sendByte

public void sendByte(byte data)
              throws java.io.IOException

Throws:

java.io.IOException

sendChar

public void sendChar(char data)
              throws java.io.IOException

Throws:

java.io.IOException

sendShort

public void sendShort(short data)
               throws java.io.IOException

Throws:

java.io.IOException

sendInt

public void sendInt(int data)
             throws java.io.IOException

Throws:

java.io.IOException

sendLong

public void sendLong(long data)
              throws java.io.IOException

Throws:

java.io.IOException

sendFloat

public void sendFloat(float data)
               throws java.io.IOException

Throws:

java.io.IOException

sendDouble

public void sendDouble(double data)
                throws java.io.IOException

Throws:

java.io.IOException

sendString

public void sendString(java.lang.String data)
                throws java.io.IOException

Throws:

java.io.IOException

sendTimestamp

public void sendTimestamp(java.sql.Timestamp timestamp)
                   throws java.io.IOException

Throws:

java.io.IOException

sendNA

public void sendNA()
            throws java.io.IOException

Throws:

java.io.IOException

sendSparseVector

public void sendSparseVector(int[] indices,
                             double[] values)
                      throws java.io.IOException

Throws:

java.io.IOException

sendDenseVector

public void sendDenseVector(double[] values)
                     throws java.io.IOException

Throws:

java.io.IOException

waitUntilAllWritten

public void waitUntilAllWritten(int timeout)
                         throws ExternalFrameConfirmationException

This method ensures the application waits for all bytes to be written before continuing in the control flow. It has to be called at the end of writing.

Parameters:

timeout - timeout in seconds

Throws:

ExternalFrameConfirmationException