water.persist

Class PersistHdfs

java.lang.Object

water.persist.Persist

water.persist.PersistHdfs

public final class PersistHdfs
extends Persist

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.hadoop.conf.Configuration	CONF

Fields inherited from class water.persist.Persist

I, UNKNOWN

Method Summary

Methods

Modifier and Type	Method and Description
static void	addFolder(org.apache.hadoop.fs.Path p, com.google.gson.JsonArray succeeded, com.google.gson.JsonArray failed)
static void	addFolder2(org.apache.hadoop.fs.Path p, java.util.ArrayList<java.lang.String> keys, java.util.ArrayList<java.lang.String> failed)
void	clear()
void	delete(Value v)
java.lang.String	getPath()
Value	lazyArrayChunk(Key key) Lazily manifest data chunks on demand.
byte[]	load(Value v) Value should already be persisted to disk.
void	loadExisting() Load all Key/Value pairs that can be found on the backend.
static java.io.InputStream	openStream(Key k, Job.ProgressMonitor pmon)
static void	store(org.apache.hadoop.fs.Path path, byte[] data)
void	store(Value v)

Methods inherited from class water.persist.Persist

getIce, getTotalSpace, getUsableSpace, initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CONF

public static final org.apache.hadoop.conf.Configuration CONF

Method Detail

getPath

public java.lang.String getPath()

Specified by:

getPath in class Persist

loadExisting

public void loadExisting()

Description copied from class: Persist

Load all Key/Value pairs that can be found on the backend.

Specified by:

loadExisting in class Persist

clear

public void clear()

Specified by:

clear in class Persist

openStream

public static java.io.InputStream openStream(Key k,
                             Job.ProgressMonitor pmon)
                                      throws java.io.IOException

Throws:

java.io.IOException

load

public byte[] load(Value v)

Description copied from class: Persist

Value should already be persisted to disk. A racing delete can trigger a failure where we get a null return, but no crash (although one could argue that a racing load&delete is a bug no matter what).

Specified by:

load in class Persist

store

public void store(Value v)

Specified by:

store in class Persist

store

public static void store(org.apache.hadoop.fs.Path path,
         byte[] data)

delete

public void delete(Value v)

Specified by:

delete in class Persist

lazyArrayChunk

public Value lazyArrayChunk(Key key)

Description copied from class: Persist

Lazily manifest data chunks on demand.

Specified by:

lazyArrayChunk in class Persist

addFolder

public static void addFolder(org.apache.hadoop.fs.Path p,
             com.google.gson.JsonArray succeeded,
             com.google.gson.JsonArray failed)
                      throws java.io.IOException

Throws:

java.io.IOException

addFolder2

public static void addFolder2(org.apache.hadoop.fs.Path p,
              java.util.ArrayList<java.lang.String> keys,
              java.util.ArrayList<java.lang.String> failed)
                       throws java.io.IOException

Throws:

java.io.IOException