org.apache.arrow.vector.ipc

Class ArrowReader

java.lang.Object

org.apache.arrow.vector.ipc.ArrowReader

All Implemented Interfaces:

AutoCloseable, DictionaryProvider

Direct Known Subclasses:

ArrowFileReader, ArrowStreamReader

public abstract class ArrowReader
extends Object
implements DictionaryProvider, AutoCloseable

Abstract class to read Schema and ArrowRecordBatches.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.arrow.vector.dictionary.DictionaryProvider

DictionaryProvider.MapDictionaryProvider

Field Summary

Fields

Modifier and Type	Field and Description
protected BufferAllocator	allocator

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ArrowReader(BufferAllocator allocator)

Method Summary

Modifier and Type	Method and Description
abstract long	bytesRead() Return the number of bytes read from the ReadChannel.
void	close() Close resources, including vector schema root and dictionary vectors, and the underlying read source.
void	close(boolean closeReadSource) Close resources, including vector schema root and dictionary vectors.
protected abstract void	closeReadSource() Close the underlying read source.
protected void	ensureInitialized() Initialize if not done previously.
Map<Long,Dictionary>	getDictionaryVectors() Returns any dictionaries that were loaded along with ArrowRecordBatches.
VectorSchemaRoot	getVectorSchemaRoot() Returns the vector schema root.
protected void	loadDictionary(ArrowDictionaryBatch dictionaryBatch) Load an ArrowDictionaryBatch to the readers dictionary vectors.
abstract boolean	loadNextBatch() Load the next ArrowRecordBatch to the vector schema root if available.
protected void	loadRecordBatch(ArrowRecordBatch batch) Load an ArrowRecordBatch to the readers VectorSchemaRoot.
Dictionary	lookup(long id) Lookup a dictionary that has been loaded using the dictionary id.
protected void	prepareLoadNextBatch() Ensure the reader has been initialized and reset the VectorSchemaRoot row count to 0.
protected abstract ArrowDictionaryBatch	readDictionary() Read a dictionary batch from the source, will be invoked after the schema has been read and called N times, where N is the number of dictionaries indicated by the schema Fields.
protected abstract Schema	readSchema() Read the Schema from the source, will be invoked at the beginning the initialization.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

allocator

protected final BufferAllocator allocator

Constructor Detail

ArrowReader

protected ArrowReader(BufferAllocator allocator)

Method Detail

getVectorSchemaRoot

public VectorSchemaRoot getVectorSchemaRoot()
                                     throws IOException

Returns the vector schema root. This will be loaded with new values on every call to loadNextBatch.

Returns:

the vector schema root

Throws:

IOException - if reading of schema fails

getDictionaryVectors

public Map<Long,Dictionary> getDictionaryVectors()
                                          throws IOException

Returns any dictionaries that were loaded along with ArrowRecordBatches.

Returns:

Map of dictionaries to dictionary id, empty if no dictionaries loaded

Throws:

IOException - if reading of schema fails

lookup

public Dictionary lookup(long id)

Lookup a dictionary that has been loaded using the dictionary id.

Specified by:

lookup in interface DictionaryProvider

Parameters:

id - Unique identifier for a dictionary

Returns:

the requested dictionary or null if not found

loadNextBatch

public abstract boolean loadNextBatch()
                               throws IOException

Load the next ArrowRecordBatch to the vector schema root if available.

Returns:

true if a batch was read, false on EOS

Throws:

IOException

bytesRead

public abstract long bytesRead()

Return the number of bytes read from the ReadChannel.

Returns:

number of bytes read

close

public void close()
           throws IOException

Close resources, including vector schema root and dictionary vectors, and the underlying read source.

Specified by:

close in interface AutoCloseable

Throws:

IOException

close

public void close(boolean closeReadSource)
           throws IOException

Close resources, including vector schema root and dictionary vectors. If the flag closeReadChannel is true then close the underlying read source, otherwise leave it open.

Parameters:

closeReadSource - Flag to control if closing the underlying read source

Throws:

IOException

closeReadSource

protected abstract void closeReadSource()
                                 throws IOException

Close the underlying read source.

Throws:

IOException

readSchema

protected abstract Schema readSchema()
                              throws IOException

Read the Schema from the source, will be invoked at the beginning the initialization.

Returns:

the read Schema

Throws:

IOException

readDictionary

protected abstract ArrowDictionaryBatch readDictionary()
                                                throws IOException

Read a dictionary batch from the source, will be invoked after the schema has been read and called N times, where N is the number of dictionaries indicated by the schema Fields.

Returns:

the read ArrowDictionaryBatch

Throws:

IOException

ensureInitialized

protected void ensureInitialized()
                          throws IOException

Initialize if not done previously.

Throws:

IOException

prepareLoadNextBatch

protected void prepareLoadNextBatch()
                             throws IOException

Ensure the reader has been initialized and reset the VectorSchemaRoot row count to 0.

Throws:

IOException

loadRecordBatch

protected void loadRecordBatch(ArrowRecordBatch batch)

Load an ArrowRecordBatch to the readers VectorSchemaRoot.

Parameters:

batch - the record batch to load

loadDictionary

protected void loadDictionary(ArrowDictionaryBatch dictionaryBatch)

Load an ArrowDictionaryBatch to the readers dictionary vectors.

Parameters:

dictionaryBatch -