water

Class ExternalFrameWriterClient

java.lang.Object

water.ExternalFrameWriterClient

public final class ExternalFrameWriterClient
extends java.lang.Object

This class is used to create and write data to H2O Frames from non-H2O environments, such as Spark Executors.

Example usage of this class:

First we need to open the connection to H2O and initialize the writer:

 
 // Prepare expected bytes from Java Classes.
 // We don't specify vector types since they are deterministically inferred from the expected types
 byte[] expectedBytes = ExternalFrameUtils.prepareExpectedTypes(new Class[]{Boolean.class, Integer.class});
 ByteChannel channel = ExternalFrameUtils.getConnection("ip:port");
 ExternalFrameWriter writer = new ExternalFrameWriter(channel);

 writer.initFrame("frameName", columns)
 writer.createChunk("frameName", expectedTypes, chunkIdx, numOfRowsToBeWritten);
 

Then we can write the data:

 {@code
 int rowsWritten = 0;
 while(rowsWritten < totalNumOfRows){
     writer.sendBool(true);
     writer.sendInt(657);
     rowsWritten++;
   }
 }
 

At last, finalize the frame

 writer.finalizeFrame("frameName, rowsPerChunk, colTypes, domains);
 

Constructor Summary

Constructors

Constructor and Description
ExternalFrameWriterClient(java.nio.channels.ByteChannel channel, int timeout, long blockSize) Initialize the External frame writer

Method Summary

Modifier and Type	Method and Description
void	close()
static ExternalFrameWriterClient	create(java.lang.String ip, int port, short timestamp, int timeout, long blockSize)
void	createChunk(java.lang.String frameKey, byte[] expectedTypes, int chunkId, int totalNumRows, int[] maxVecSizes) Create single chunk on the already existing empty frame.
void	finalizeFrame(java.lang.String keyName, long[] rowsPerChunk, byte[] colTypes, java.lang.String[][] domains)
int	getNumberOfWrittenRows()
void	initFrame(java.lang.String keyName, java.lang.String[] names)
void	sendBoolean(boolean data)
void	sendByte(byte data)
void	sendChar(char data)
void	sendDenseVector(double[] values)
void	sendDouble(double data)
void	sendFloat(float data)
void	sendInt(int data)
void	sendLong(long data)
void	sendNA()
void	sendShort(short data)
void	sendSparseVector(int[] indices, double[] values)
void	sendString(java.lang.String data)
void	sendTimestamp(java.sql.Timestamp timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExternalFrameWriterClient

public ExternalFrameWriterClient(java.nio.channels.ByteChannel channel,
                                 int timeout,
                                 long blockSize)

Initialize the External frame writer

This method expects expected types in order to ensure we send the data in optimal way.

Parameters:

channel - communication channel to h2o node

Method Detail

create

public static ExternalFrameWriterClient create(java.lang.String ip,
                                               int port,
                                               short timestamp,
                                               int timeout,
                                               long blockSize)
                                        throws java.io.IOException

Throws:

java.io.IOException

initFrame

public void initFrame(java.lang.String keyName,
                      java.lang.String[] names)
               throws java.io.IOException,
                      ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

createChunk

public void createChunk(java.lang.String frameKey,
                        byte[] expectedTypes,
                        int chunkId,
                        int totalNumRows,
                        int[] maxVecSizes)
                 throws java.io.IOException

Create single chunk on the already existing empty frame.

Parameters:

frameKey - name of the frame

expectedTypes - expected types

chunkId - chunk index

totalNumRows - total number of rows which is about to be sent

Throws:

java.io.IOException

finalizeFrame

public void finalizeFrame(java.lang.String keyName,
                          long[] rowsPerChunk,
                          byte[] colTypes,
                          java.lang.String[][] domains)
                   throws java.io.IOException,
                          ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

close

public void close()
           throws java.io.IOException,
                  ExternalFrameConfirmationException

Throws:

java.io.IOException

ExternalFrameConfirmationException

sendBoolean

public void sendBoolean(boolean data)
                 throws java.io.IOException

Throws:

java.io.IOException

sendByte

public void sendByte(byte data)
              throws java.io.IOException

Throws:

java.io.IOException

sendChar

public void sendChar(char data)
              throws java.io.IOException

Throws:

java.io.IOException

sendShort

public void sendShort(short data)
               throws java.io.IOException

Throws:

java.io.IOException

sendInt

public void sendInt(int data)
             throws java.io.IOException

Throws:

java.io.IOException

sendLong

public void sendLong(long data)
              throws java.io.IOException

Throws:

java.io.IOException

sendFloat

public void sendFloat(float data)
               throws java.io.IOException

Throws:

java.io.IOException

sendDouble

public void sendDouble(double data)
                throws java.io.IOException

Throws:

java.io.IOException

sendString

public void sendString(java.lang.String data)
                throws java.io.IOException

Throws:

java.io.IOException

sendTimestamp

public void sendTimestamp(java.sql.Timestamp timestamp)
                   throws java.io.IOException

Throws:

java.io.IOException

sendNA

public void sendNA()
            throws java.io.IOException

Throws:

java.io.IOException

sendSparseVector

public void sendSparseVector(int[] indices,
                             double[] values)
                      throws java.io.IOException

Throws:

java.io.IOException

sendDenseVector

public void sendDenseVector(double[] values)
                     throws java.io.IOException

Throws:

java.io.IOException

getNumberOfWrittenRows

public int getNumberOfWrittenRows()