water.persist

Class PersistManager

java.lang.Object

water.persist.PersistManager

public class PersistManager
extends java.lang.Object

One true persistence manager which hides the implementations from H2O. In particular, HDFS support or S3 support may or may not exist depending on what is on the classpath.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PersistManager.PersistStatsEntry
static class	PersistManager.Schemes Persistence schemes; used as file prefixes eg "hdfs://some_hdfs_path/some_file"

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAX_BACKENDS

Constructor Summary

Constructors

Constructor and Description
PersistManager(java.net.URI iceRoot)

Method Summary

Methods

Modifier and Type	Method and Description
Key	anyURIToKey(java.net.URI uri) Convert given URI into a specific H2O key representation.
java.util.ArrayList<java.lang.String>	calcTypeaheadMatches(java.lang.String filter, int limit) Calculate typeahead matches for src
java.io.OutputStream	create(java.lang.String path, boolean overwrite)
void	delete(int backend, Value v)
boolean	delete(java.lang.String path)
boolean	exists(java.lang.String path)
java.lang.String	getHdfsHomeDirectory()
Persist	getIce() Get the current Persist flavor for user-mode swapping.
Persist	getPersistForURI(java.net.URI uri)
PersistManager.PersistStatsEntry[]	getStats()
void	importFiles(java.lang.String path, java.util.ArrayList<java.lang.String> files, java.util.ArrayList<java.lang.String> keys, java.util.ArrayList<java.lang.String> fails, java.util.ArrayList<java.lang.String> dels) From a path produce a list of files and keys for parsing.
static boolean	isHdfsPath(java.lang.String path)
long	length(java.lang.String path)
Persist.PersistEntry[]	list(java.lang.String path)
byte[]	load(int backend, Value v)
boolean	mkdirs(java.lang.String path)
java.io.InputStream	open(java.lang.String path)
boolean	rename(java.lang.String fromPath, java.lang.String toPath)
void	store(int backend, Value v)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_BACKENDS

public static final int MAX_BACKENDS

See Also:

Constant Field Values

Constructor Detail

PersistManager

public PersistManager(java.net.URI iceRoot)

Method Detail

getStats

public PersistManager.PersistStatsEntry[] getStats()

isHdfsPath

public static boolean isHdfsPath(java.lang.String path)

store

public void store(int backend,
         Value v)

delete

public void delete(int backend,
          Value v)

load

public byte[] load(int backend,
          Value v)
            throws java.io.IOException

Throws:

java.io.IOException

getIce

public Persist getIce()

Get the current Persist flavor for user-mode swapping.

anyURIToKey

public final Key anyURIToKey(java.net.URI uri)
                      throws java.io.IOException

Convert given URI into a specific H2O key representation. The representation depends on persistent backend, since it will deduce file location from the key content. The method will look at scheme of URI and based on it, it will ask a backend to provide a conversion to a key (i.e., URI with scheme 'hdfs' will be forwared to HDFS backend).

Parameters:

uri - file location

Returns:

a key encoding URI

Throws:

java.io.IOException - in the case of uri conversion problem

H2OIllegalArgumentException - in case of unsupported scheme

calcTypeaheadMatches

public java.util.ArrayList<java.lang.String> calcTypeaheadMatches(java.lang.String filter,
                                                         int limit)

Calculate typeahead matches for src

Parameters:

filter - Source string to match for typeahead

limit - Max number of entries to return

Returns:

List of matches

importFiles

public void importFiles(java.lang.String path,
               java.util.ArrayList<java.lang.String> files,
               java.util.ArrayList<java.lang.String> keys,
               java.util.ArrayList<java.lang.String> fails,
               java.util.ArrayList<java.lang.String> dels)

From a path produce a list of files and keys for parsing. Use as follows: ArrayList files = new ArrayList(); ArrayList keys = new ArrayList(); ArrayList fails = new ArrayList(); ArrayList dels = new ArrayList(); importFiles(importFiles.path, files, keys, fails, dels);

Parameters:

path - (Input) Path to import data from

files - (Output) List of files found

keys - (Output) List of keys corresponding to files

fails - (Output) List of failed files which mismatch among nodes

dels - (Output) I don't know what this is

getHdfsHomeDirectory

public java.lang.String getHdfsHomeDirectory()

list

public Persist.PersistEntry[] list(java.lang.String path)

exists

public boolean exists(java.lang.String path)

length

public long length(java.lang.String path)

open

public java.io.InputStream open(java.lang.String path)

mkdirs

public boolean mkdirs(java.lang.String path)

rename

public boolean rename(java.lang.String fromPath,
             java.lang.String toPath)

create

public java.io.OutputStream create(java.lang.String path,
                          boolean overwrite)

delete

public boolean delete(java.lang.String path)

getPersistForURI

public Persist getPersistForURI(java.net.URI uri)