water

Class ExternalFrameWriterClient

java.lang.Object

water.ExternalFrameWriterClient

public final class ExternalFrameWriterClient
extends java.lang.Object

This class is used to create and write data to H2O Frames from non-H2O environments, such as Spark Executors.

Example usage of this class:

First we need to open the connection to H2O and initialize the writer:

 
 // Prepare expected bytes from Java Classes.
 // We don't specify vector types since they are deterministically inferred from the expected types
 byte[] expectedBytes = ExternalFrameUtils.prepareExpectedTypes(new Class[]{Boolean.class, Integer.class});
 ByteChannel channel = ExternalFrameUtils.getConnection("ip:port");
 ExternalFrameWriter writer = new ExternalFrameWriter(channel);

 writer.initFrame("frameName", columns)
 writer.createChunk("frameName", expectedTypes, chunkIdx, numOfRowsToBeWritten);
 

Then we can write the data:

 {@code
 int rowsWritten = 0;
 while(rowsWritten < totalNumOfRows){
     writer.sendBool(true);
     writer.sendInt(657);
     rowsWritten++;
   }
 }
 

At last, finalize the frame

 writer.finalizeFrame("frameName, rowsPerChunk, colTypes, domains);
 

Constructor Summary

Constructors

Constructor and Description
ExternalFrameWriterClient(java.nio.channels.ByteChannel channel, int timeout) Initialize the External frame writer

Method Summary

Modifier and Type	Method and Description
void	close()
static ExternalFrameWriterClient	create(java.lang.String ip, int port, short timestamp, int timeout)
void	createChunk(java.lang.String frameKey, byte[] expectedTypes, int chunkId, int totalNumRows, int[] maxVecSizes) Create single chunk on the already existing empty frame.
void	finalizeFrame(java.lang.String keyName, long[] rowsPerChunk, byte[] colTypes, java.lang.String[][] domains)
int	getNumberOfWrittenRows()
void	initFrame(java.lang.String keyName, java.lang.String[] names)
void	sendBoolean(boolean data)
void	sendByte(byte data)
void	sendChar(char data)
void	sendDenseVector(double[] values)
void	sendDouble(double data)
void	sendFloat(float data)
void	sendInt(int data)
void	sendLong(long data)
void	sendNA()
void	sendShort(short data)
void	sendSparseVector(int[] indices, double[] values)
void	sendString(java.lang.String data)
void	sendTimestamp(java.sql.Timestamp timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExternalFrameWriterClient

public ExternalFrameWriterClient(java.nio.channels.ByteChannel channel,
                                 int timeout)

Initialize the External frame writer

This method expects expected types in order to ensure we send the data in optimal way.

Parameters:

channel - communication channel to h2o node

Method Detail

create

public static ExternalFrameWriterClient create(java.lang.String ip,
                                               int port,
                                               short timestamp,
                                               int timeout)
                                        throws java.io.IOException

Throws:

java.io.IOException

initFrame

public void initFrame(java.lang.String keyName,
                      java.lang.String[] names)
               throws java.io.IOException,
                      ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

createChunk

public void createChunk(java.lang.String frameKey,
                        byte[] expectedTypes,
                        int chunkId,
                        int totalNumRows,
                        int[] maxVecSizes)
                 throws java.io.IOException

Create single chunk on the already existing empty frame.

Parameters:

frameKey - name of the frame

expectedTypes - expected types

chunkId - chunk index

totalNumRows - total number of rows which is about to be sent

Throws:

java.io.IOException

finalizeFrame

public void finalizeFrame(java.lang.String keyName,
                          long[] rowsPerChunk,
                          byte[] colTypes,
                          java.lang.String[][] domains)
                   throws java.io.IOException,
                          ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

close

public void close()
           throws java.io.IOException

Throws:

java.io.IOException

sendBoolean

public void sendBoolean(boolean data)
                 throws java.io.IOException,
                        ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendByte

public void sendByte(byte data)
              throws java.io.IOException,
                     ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendChar

public void sendChar(char data)
              throws java.io.IOException,
                     ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendShort

public void sendShort(short data)
               throws java.io.IOException,
                      ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendInt

public void sendInt(int data)
             throws java.io.IOException,
                    ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendLong

public void sendLong(long data)
              throws java.io.IOException,
                     ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendFloat

public void sendFloat(float data)
               throws java.io.IOException,
                      ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendDouble

public void sendDouble(double data)
                throws java.io.IOException,
                       ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendString

public void sendString(java.lang.String data)
                throws java.io.IOException,
                       ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendTimestamp

public void sendTimestamp(java.sql.Timestamp timestamp)
                   throws java.io.IOException,
                          ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendNA

public void sendNA()
            throws java.io.IOException,
                   ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendSparseVector

public void sendSparseVector(int[] indices,
                             double[] values)
                      throws java.io.IOException,
                             ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendDenseVector

public void sendDenseVector(double[] values)
                     throws java.io.IOException,
                            ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

getNumberOfWrittenRows

public int getNumberOfWrittenRows()