public final class ExternalFrameWriterClient
extends java.lang.Object
This class is used to create and write data to H2O Frames from non-H2O environments, such as Spark Executors.
Example usage of this class:First we need to open the connection to H2O and initialize the writer:
// Prepare expected bytes from Java Classes.
// We don't specify vector types since they are deterministically inferred from the expected types
byte[] expectedBytes = ExternalFrameUtils.prepareExpectedTypes(new Class[]{Boolean.class, Integer.class});
ByteChannel channel = ExternalFrameUtils.getConnection("ip:port");
ExternalFrameWriter writer = new ExternalFrameWriter(channel);
writer.createChunks("frameName", expectedTypes, chunkIdx, numOfRowsToBeWritten);
Then we can write the data:
int rowsWritten = 0;
while(rowsWritten < totalNumOfRows){
writer.sendBool(true);
writer.sendInt(657);
}
And at the end we need to make sure to force to code wait for all data to be written
writer.waitUntilAllWritten();
Constructor and Description |
---|
ExternalFrameWriterClient(java.nio.channels.ByteChannel channel)
Initialize the External frame writer
This method expects expected types in order to ensure we send the data in optimal way.
|
Modifier and Type | Method and Description |
---|---|
void |
createChunks(java.lang.String frameKey,
byte[] expectedTypes,
int chunkId,
int totalNumRows,
int[] maxVecSizes)
Create chunks on the h2o backend.
|
void |
sendBoolean(boolean data) |
void |
sendByte(byte data) |
void |
sendChar(char data) |
void |
sendDenseVector(double[] values) |
void |
sendDouble(double data) |
void |
sendFloat(float data) |
void |
sendInt(int data) |
void |
sendLong(long data) |
void |
sendNA() |
void |
sendShort(short data) |
void |
sendSparseVector(int[] indices,
double[] values) |
void |
sendString(java.lang.String data) |
void |
sendTimestamp(java.sql.Timestamp timestamp) |
void |
waitUntilAllWritten(int timeout)
This method ensures the application waits for all bytes to be written before continuing in the control flow.
|
public ExternalFrameWriterClient(java.nio.channels.ByteChannel channel)
channel
- communication channel to h2o nodepublic void createChunks(java.lang.String frameKey, byte[] expectedTypes, int chunkId, int totalNumRows, int[] maxVecSizes) throws java.io.IOException
frameKey
- name of the frameexpectedTypes
- expected typeschunkId
- chunk indextotalNumRows
- total number of rows which is about to be sentjava.io.IOException
public void sendBoolean(boolean data) throws java.io.IOException
java.io.IOException
public void sendByte(byte data) throws java.io.IOException
java.io.IOException
public void sendChar(char data) throws java.io.IOException
java.io.IOException
public void sendShort(short data) throws java.io.IOException
java.io.IOException
public void sendInt(int data) throws java.io.IOException
java.io.IOException
public void sendLong(long data) throws java.io.IOException
java.io.IOException
public void sendFloat(float data) throws java.io.IOException
java.io.IOException
public void sendDouble(double data) throws java.io.IOException
java.io.IOException
public void sendString(java.lang.String data) throws java.io.IOException
java.io.IOException
public void sendTimestamp(java.sql.Timestamp timestamp) throws java.io.IOException
java.io.IOException
public void sendNA() throws java.io.IOException
java.io.IOException
public void sendSparseVector(int[] indices, double[] values) throws java.io.IOException
java.io.IOException
public void sendDenseVector(double[] values) throws java.io.IOException
java.io.IOException
public void waitUntilAllWritten(int timeout) throws ExternalFrameConfirmationException
timeout
- timeout in secondsExternalFrameConfirmationException