water.parser

Class ParseSetup

java.lang.Object

water.Iced

water.parser.ParseSetup

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, Freezable

public final class ParseSetup
extends Iced

Configuration and base guesser for a parse;

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParseSetup.GuessSetupTsk Try to determine the ParseSetup on a file by file basis and merge results.

Field Summary

Fields

Modifier and Type	Field and Description
static int	GUESS_COL_CNT
static int	GUESS_HEADER
static byte	GUESS_SEP
static int	HAS_HEADER
static int	NO_HEADER

Constructor Summary

Constructors

Constructor and Description
ParseSetup() Create a default ParseSetup Used by Ray's schema magic
ParseSetup(ParserType t, byte sep, boolean singleQuotes, int checkHeader, int ncols, java.lang.String[][] data) Create a ParseSetup without any column information Typically used by file type parsers for returning final invalid results
ParseSetup(ParserType t, byte sep, boolean singleQuotes, int checkHeader, int ncols, java.lang.String[] columnNames, byte[] ctypes, java.lang.String[][] domains, java.lang.String[][] naStrings, java.lang.String[][] data) Create a ParseSetup with all parameters except chunk size.
ParseSetup(ParserType t, byte sep, boolean singleQuotes, int checkHeader, int ncols, java.lang.String[] columnNames, byte[] ctypes, java.lang.String[][] domains, java.lang.String[][] naStrings, java.lang.String[][] data, int chunkSize)
ParseSetup(ParseSetup ps)
ParseSetup(ParseSetupV3 ps) Create a ParseSetup with parameters from the client.

Method Summary

Methods

Modifier and Type	Method and Description
static java.lang.String	createHexName(java.lang.String n) Cleans up the file name to make .hex name to be used as a destination key.
java.lang.String[]	getColumnNames()
java.lang.String[]	getColumnTypeStrings()
java.lang.String[][]	getData()
static ParseSetup	guessSetup(byte[] bits, ParserType pType, byte sep, int ncols, boolean singleQuotes, int checkHeader, java.lang.String[] columnNames, byte[] columnTypes, java.lang.String[][] domains, java.lang.String[][] naStrings)
static ParseSetup	guessSetup(byte[] bits, ParseSetup userSetup) Guess everything from a single pile-o-bits.
static ParseSetup	guessSetup(Key[] fkeys, boolean singleQuote, int checkHeader) Used by test harnesses for simple parsing of test data.
static ParseSetup	guessSetup(Key[] fkeys, ParseSetup userSetup) Discover the parse setup needed to correctly parse all files.
Parser	parser()
static byte[]	strToColumnTypes(java.lang.String[] strs)
java.lang.String	toString()

Methods inherited from class water.Iced

clone, frozenType, read_impl, read, readExternal, readJSON_impl, readJSON, toJsonString, write_impl, write, writeExternal, writeHTML_impl, writeHTML, writeJSON_impl, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

GUESS_SEP

public static final byte GUESS_SEP

See Also:

Constant Field Values

NO_HEADER

public static final int NO_HEADER

See Also:

Constant Field Values

GUESS_HEADER

public static final int GUESS_HEADER

See Also:

Constant Field Values

HAS_HEADER

public static final int HAS_HEADER

See Also:

Constant Field Values

GUESS_COL_CNT

public static final int GUESS_COL_CNT

See Also:

Constant Field Values

Constructor Detail

ParseSetup

public ParseSetup(ParseSetup ps)

ParseSetup

public ParseSetup(ParserType t,
          byte sep,
          boolean singleQuotes,
          int checkHeader,
          int ncols,
          java.lang.String[] columnNames,
          byte[] ctypes,
          java.lang.String[][] domains,
          java.lang.String[][] naStrings,
          java.lang.String[][] data,
          int chunkSize)

ParseSetup

public ParseSetup(ParseSetupV3 ps)

Create a ParseSetup with parameters from the client. Typically used to guide sampling in the data to verify chosen settings, and fill in missing settings.

Parameters:

ps - Parse setup settings from client

ParseSetup

public ParseSetup(ParserType t,
          byte sep,
          boolean singleQuotes,
          int checkHeader,
          int ncols,
          java.lang.String[] columnNames,
          byte[] ctypes,
          java.lang.String[][] domains,
          java.lang.String[][] naStrings,
          java.lang.String[][] data)

Create a ParseSetup with all parameters except chunk size. Typically used by file type parsers for returning final valid results _chunk_size will be set later using results from all files.

ParseSetup

public ParseSetup(ParserType t,
          byte sep,
          boolean singleQuotes,
          int checkHeader,
          int ncols,
          java.lang.String[][] data)

Create a ParseSetup without any column information Typically used by file type parsers for returning final invalid results

ParseSetup

public ParseSetup()

Create a default ParseSetup Used by Ray's schema magic

Method Detail

getColumnNames

public java.lang.String[] getColumnNames()

getData

public java.lang.String[][] getData()

getColumnTypeStrings

public java.lang.String[] getColumnTypeStrings()

strToColumnTypes

public static byte[] strToColumnTypes(java.lang.String[] strs)

parser

public Parser parser()

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

guessSetup

public static ParseSetup guessSetup(Key[] fkeys,
                    boolean singleQuote,
                    int checkHeader)

Used by test harnesses for simple parsing of test data. Presumes auto-detection for file and separator types.

Parameters:

fkeys - Keys to input vectors to be parsed

singleQuote - single quotes quote fields

checkHeader - check for a header

Returns:

ParseSetup settings from looking at all files

guessSetup

public static ParseSetup guessSetup(Key[] fkeys,
                    ParseSetup userSetup)

Discover the parse setup needed to correctly parse all files. This takes a ParseSetup as guidance. Each file is examined individually and then results merged. If a conflict exists between any results all files are re-examined using the best guess from the first examination.

Parameters:

fkeys - Keys to input vectors to be parsed

userSetup - Setup guidance from user

Returns:

ParseSetup settings from looking at all files

guessSetup

public static ParseSetup guessSetup(byte[] bits,
                    ParseSetup userSetup)

Guess everything from a single pile-o-bits. Used in tests, or in initial parser inspections when the user has not told us anything about separators or headers.

Parameters:

bits - Initial bytes from a parse source

Returns:

ParseSetup settings from looking at all files

guessSetup

public static ParseSetup guessSetup(byte[] bits,
                    ParserType pType,
                    byte sep,
                    int ncols,
                    boolean singleQuotes,
                    int checkHeader,
                    java.lang.String[] columnNames,
                    byte[] columnTypes,
                    java.lang.String[][] domains,
                    java.lang.String[][] naStrings)

createHexName

public static java.lang.String createHexName(java.lang.String n)

Cleans up the file name to make .hex name to be used as a destination key. Eliminates common file extensions, and replaces odd characters.

Parameters:

n - filename to be cleaned

Returns:

cleaned name