org.apache.parquet.hadoop

Class ParquetWriter.Builder<T,SELF extends ParquetWriter.Builder<T,SELF>>

java.lang.Object

org.apache.parquet.hadoop.ParquetWriter.Builder<T,SELF>

Type Parameters:

T - The type of objects written by the constructed ParquetWriter.

SELF - The type of this builder that is returned by builder methods

Direct Known Subclasses:

ExampleParquetWriter.Builder

Enclosing class:

ParquetWriter<T>

public abstract static class ParquetWriter.Builder<T,SELF extends ParquetWriter.Builder<T,SELF>>
extends Object

An abstract builder class for ParquetWriter instances. Object models should extend this builder to provide writer configuration options.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Builder(OutputFile path)
protected	Builder(org.apache.hadoop.fs.Path path)

Method Summary

Modifier and Type	Method and Description
ParquetWriter<T>	build() Build a ParquetWriter with the accumulated configuration.
SELF	config(String property, String value) Set a property that will be available to the read path.
SELF	enableDictionaryEncoding() Enables dictionary encoding for the constructed writer.
SELF	enableValidation() Enables validation for the constructed writer.
protected abstract WriteSupport<T>	getWriteSupport(org.apache.hadoop.conf.Configuration conf)
protected abstract SELF	self()
SELF	withCompressionCodec(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName) Set the compression codec used by the constructed writer.
SELF	withConf(org.apache.hadoop.conf.Configuration conf) Set the Configuration used by the constructed writer.
SELF	withDictionaryEncoding(boolean enableDictionary) Enable or disable dictionary encoding for the constructed writer.
SELF	withDictionaryPageSize(int dictionaryPageSize) Set the Parquet format dictionary page size used by the constructed writer.
SELF	withMaxPaddingSize(int maxPaddingSize) Set the maximum amount of padding, in bytes, that will be used to align row groups with blocks in the underlying filesystem.
SELF	withPageSize(int pageSize) Set the Parquet format page size used by the constructed writer.
SELF	withRowGroupSize(int rowGroupSize) Set the Parquet format row group size used by the constructed writer.
SELF	withValidation(boolean enableValidation) Enable or disable validation for the constructed writer.
SELF	withWriteMode(ParquetFileWriter.Mode mode) Set the write mode used when creating the backing file for this writer.
SELF	withWriterVersion(ParquetProperties.WriterVersion version) Set the format version used by the constructed writer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

protected Builder(org.apache.hadoop.fs.Path path)

Builder

protected Builder(OutputFile path)

Method Detail

self

protected abstract SELF self()

Returns:

this as the correct subclass of ParquetWriter.Builder.

getWriteSupport

protected abstract WriteSupport<T> getWriteSupport(org.apache.hadoop.conf.Configuration conf)

Parameters:

conf - a configuration

Returns:

an appropriate WriteSupport for the object model.

withConf

public SELF withConf(org.apache.hadoop.conf.Configuration conf)

Set the Configuration used by the constructed writer.

Parameters:

conf - a Configuration

Returns:

this builder for method chaining.

withWriteMode

public SELF withWriteMode(ParquetFileWriter.Mode mode)

Set the write mode used when creating the backing file for this writer.

Parameters:

mode - a ParquetFileWriter.Mode

Returns:

this builder for method chaining.

withCompressionCodec

public SELF withCompressionCodec(org.apache.parquet.hadoop.metadata.CompressionCodecName codecName)

Set the compression codec used by the constructed writer.

Parameters:

codecName - a CompressionCodecName

Returns:

this builder for method chaining.

withRowGroupSize

public SELF withRowGroupSize(int rowGroupSize)

Set the Parquet format row group size used by the constructed writer.

Parameters:

rowGroupSize - an integer size in bytes

Returns:

this builder for method chaining.

withPageSize

public SELF withPageSize(int pageSize)

Set the Parquet format page size used by the constructed writer.

Parameters:

pageSize - an integer size in bytes

Returns:

this builder for method chaining.

withDictionaryPageSize

public SELF withDictionaryPageSize(int dictionaryPageSize)

Set the Parquet format dictionary page size used by the constructed writer.

Parameters:

dictionaryPageSize - an integer size in bytes

Returns:

this builder for method chaining.

withMaxPaddingSize

public SELF withMaxPaddingSize(int maxPaddingSize)

Set the maximum amount of padding, in bytes, that will be used to align row groups with blocks in the underlying filesystem. If the underlying filesystem is not a block filesystem like HDFS, this has no effect.

Parameters:

maxPaddingSize - an integer size in bytes

Returns:

this builder for method chaining.

enableDictionaryEncoding

public SELF enableDictionaryEncoding()

Enables dictionary encoding for the constructed writer.

Returns:

this builder for method chaining.

withDictionaryEncoding

public SELF withDictionaryEncoding(boolean enableDictionary)

Enable or disable dictionary encoding for the constructed writer.

Parameters:

enableDictionary - whether dictionary encoding should be enabled

Returns:

this builder for method chaining.

enableValidation

public SELF enableValidation()

Enables validation for the constructed writer.

Returns:

this builder for method chaining.

withValidation

public SELF withValidation(boolean enableValidation)

Enable or disable validation for the constructed writer.

Parameters:

enableValidation - whether validation should be enabled

Returns:

this builder for method chaining.

withWriterVersion

public SELF withWriterVersion(ParquetProperties.WriterVersion version)

Set the format version used by the constructed writer.

Parameters:

version - a WriterVersion

Returns:

this builder for method chaining.

config

public SELF config(String property,
                   String value)

Set a property that will be available to the read path. For writers that use a Hadoop configuration, this is the recommended way to add configuration values.

Parameters:

property - a String property name

value - a String property value

Returns:

this builder for method chaining.

build

public ParquetWriter<T> build()
                       throws IOException

Build a ParquetWriter with the accumulated configuration.

Returns:

a configured ParquetWriter instance.

Throws:

IOException - if there is an error while creating the writer