| Package | Description |
|---|---|
| com.google.cloud.hadoop.fs.gcs |
| Modifier and Type | Field and Description |
|---|---|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.BLOCK_SIZE
Configuration key for default block size of a file.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.DELEGATION_TOKEN_BINDING_CLASS
Configuration key for Delegation Token binding class.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCE_BUCKET_DELETE_ENABLE
If true, recursive delete on a path that refers to a GCS bucket itself ('/' for any
bucket-rooted GoogleHadoopFileSystem) or delete on that path when it's empty will result in
fully deleting the GCS bucket.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_APPLICATION_NAME_SUFFIX
Configuration key for adding a suffix to the GHFS application name sent to GCS.
|
static HadoopConfigurationProperty<Class<? extends AuthorizationHandler>> |
GoogleHadoopFileSystemConfiguration.GCS_AUTHORIZATION_HANDLER_IMPL
Configuration for authorization handler implementation class.
|
static HadoopConfigurationProperty<Map<String,String>> |
GoogleHadoopFileSystemConfiguration.GCS_AUTHORIZATION_HANDLER_PROPERTIES_PREFIX
Configuration prefix for custom authorization handler properties.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_BATCH_THREADS
Configuration key for a number of threads to execute batch requests.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_COOPERATIVE_LOCKING_ENABLE
Configuration key for using cooperative locking to achieve a directory mutation operations
isolation.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_COOPERATIVE_LOCKING_EXPIRATION_TIMEOUT_MS
Configuration key for lock expiration when using cooperative locking.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_COOPERATIVE_LOCKING_MAX_CONCURRENT_OPERATIONS
Configuration key for maximum allowed concurrent operations when using cooperative locking.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_COPY_WITH_REWRITE_ENABLE
Configuration key for enabling the use of Rewrite requests for copy operations.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_CREATE_ITEMS_CONFLICT_CHECK_ENABLE
Configuration key for enabling check to ensure that conflicting directories do not exist when
creating files and conflicting files do not exist when creating directories.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_ENCRYPTION_ALGORITHM
Configuration key for the CSEK encryption algorithm.
|
static HadoopConfigurationProperty<RedactedString> |
GoogleHadoopFileSystemConfiguration.GCS_ENCRYPTION_KEY
Configuration key for the CSEK encryption key.
|
static HadoopConfigurationProperty<RedactedString> |
GoogleHadoopFileSystemConfiguration.GCS_ENCRYPTION_KEY_HASH
Configuration key for sha256 hash of the CSEK encryption key.
|
static HadoopConfigurationProperty<GoogleHadoopFileSystemBase.GcsFileChecksumType> |
GoogleHadoopFileSystemConfiguration.GCS_FILE_CHECKSUM_TYPE
Configuration key for which type of FileChecksum to return; if a particular file doesn't
support the requested type, then getFileChecksum() will return null for that file.
|
static HadoopConfigurationProperty<GoogleHadoopFileSystemBase.GlobAlgorithm> |
GoogleHadoopFileSystemConfiguration.GCS_GLOB_ALGORITHM
Configuration key for customizing glob search algorithm.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_CHECK_INTERVAL_TIMEOUT_MS
Configuration key for check interval (in millisecond) for gRPC request timeout to GCS.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_CHECKSUMS_ENABLE
Configuration key for enabling checksum validation for the gRPC API.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_DIRECTPATH_ENABLE
Configuration key for enabling use of directpath gRPC API for read/write.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_ENABLE
Configuration key for enabling use of the gRPC API for read/write.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_READ_MESSAGE_TIMEOUT_MS
Configuration key for the message timeout (in millisecond) for gRPC read requests to GCS.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_READ_METADATA_TIMEOUT_MS
Configuration key for the connection timeout (in millisecond) for gRPC metadata requests to
GCS.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_READ_TIMEOUT_MS
Configuration key for the connection timeout (in millisecond) for gRPC read requests to GCS.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_READ_ZEROCOPY_ENABLE
Configuration key for enabling the zero-copy deserializer for the gRPC API.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_SERVER_ADDRESS
Configuration key for the Cloud Storage gRPC server address.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_TRAFFICDIRECTOR_ENABLE
Configuration key for enabling use of traffic director gRPC API for read/write.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_UPLOAD_BUFFERED_REQUESTS
Configuration key for the number of requests to be buffered for uploads to GCS.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_WRITE_MESSAGE_TIMEOUT_MS
Configuration key for the message timeout (in millisecond) for gRPC write requests to GCS.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_GRPC_WRITE_TIMEOUT_MS
Configuration key for the connect timeout (in millisecond) for gRPC write requests to GCS.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_CONNECT_TIMEOUT
Configuration key for the connect timeout (in millisecond) for HTTP request to GCS.
|
static HadoopConfigurationProperty<Map<String,String>> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_HEADERS
Configuration key for the headers for HTTP request to GCS.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_MAX_RETRY
Configuration key for the max number of retries for failed HTTP request to GCS.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_HTTP_READ_TIMEOUT
Configuration key for the connect timeout (in millisecond) for HTTP request to GCS.
|
static HadoopConfigurationProperty<GoogleCloudStorageReadOptions.Fadvise> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FADVISE
Tunes reading objects behavior to optimize HTTP GET requests for various use cases.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_FAST_FAIL_ON_NOT_FOUND_ENABLE
If
true, on opening a file we will proactively perform a metadata GET to check
whether the object exists, even though the underlying channel will not open a data stream until
read() is actually called. |
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_INPLACE_SEEK_LIMIT
If forward seeks are within this many bytes of the current position, seeks are performed by
reading and discarding bytes in-place rather than opening a new underlying stream.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_MIN_RANGE_REQUEST_SIZE
Minimum size in bytes of the HTTP Range header set in GCS request when opening new stream to
read an object.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_INPUT_STREAM_SUPPORT_GZIP_ENCODING_ENABLE
If true, reading a file with GZIP content encoding (HTTP header "Content-Encoding: gzip") will
result in failure (IOException is thrown).
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_LAZY_INITIALIZATION_ENABLE
Configuration key for enabling lazy initialization of GCS FS instance.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_MARKER_FILE_PATTERN
Configuration key for marker file pattern.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_MAX_LIST_ITEMS_PER_CALL
Configuration key for number of items to return per call to the list* GCS RPCs.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_MAX_REQUESTS_PER_BATCH
Configuration key for a max number of GCS RPCs in batch request.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_MAX_WAIT_MILLIS_EMPTY_OBJECT_CREATE
Configuration key for modifying the maximum amount of time to wait for empty object creation.
|
static HadoopConfigurationProperty<GoogleCloudStorageOptions.MetricsSink> |
GoogleHadoopFileSystemConfiguration.GCS_METRICS_SINK
Configuration key to enable publishing metrics to Google cloud monitoring.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_BUFFER_SIZE
Configuration key for setting write buffer size.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_DIRECT_UPLOAD_ENABLE
Configuration key for enabling GCS direct upload.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_PIPE_BUFFER_SIZE
Configuration key for setting pipe buffer size.
|
static HadoopConfigurationProperty<AsyncWriteChannelOptions.PipeType> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_PIPE_TYPE
Configuration key for setting pipe type.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_SYNC_MIN_INTERVAL_MS
Configuration key for the minimal time interval between consecutive sync/hsync/hflush calls.
|
static HadoopConfigurationProperty<GoogleHadoopFileSystemBase.OutputStreamType> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_TYPE
Configuration key for which type of output stream to use; different options may have different
degrees of support for advanced features like
hsync() and different performance
characteristics. |
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_UPLOAD_CACHE_SIZE
Configuration for setting GCS upload cache size.
|
static HadoopConfigurationProperty<Integer> |
GoogleHadoopFileSystemConfiguration.GCS_OUTPUT_STREAM_UPLOAD_CHUNK_SIZE
Configuration key for setting GCS upload chunk size.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_ENABLE
Configuration key for using a local item cache to supplement GCS API "getFile" results.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_PERFORMANCE_CACHE_MAX_ENTRY_AGE_MILLIS
Configuration key for maximum number of milliseconds a GoogleCloudStorageItemInfo will remain
"valid" in the performance cache before it's invalidated.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_PROJECT_ID
Configuration key for GCS project ID.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_REPAIR_IMPLICIT_DIRECTORIES_ENABLE
Configuration key for enabling automatic repair of implicit directories whenever detected
inside delete and rename calls.
|
static HadoopConfigurationProperty<Collection<String>> |
GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_BUCKETS
Configuration key for GCS Requester Pays Buckets.
|
static HadoopConfigurationProperty<RequesterPaysOptions.RequesterPaysMode> |
GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_MODE
Configuration key for GCS project ID.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_REQUESTER_PAYS_PROJECT_ID
Configuration key for GCS Requester Pays Project ID.
|
static HadoopConfigurationProperty<Long> |
GoogleHadoopFileSystemConfiguration.GCS_REWRITE_MAX_BYTES_PER_CALL
Configuration key for specifying max number of bytes rewritten in a single rewrite request when
fs.gs.copy.with.rewrite.enable is set to 'true'.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_ROOT_URL
Configuration key for the Cloud Storage API endpoint root URL.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_SERVICE_PATH
Configuration key for the Cloud Storage API endpoint service path.
|
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_STATUS_PARALLEL_ENABLE
If true, executes GCS requests in
listStatus and getFileStatus methods in
parallel to reduce latency. |
static HadoopConfigurationProperty<Boolean> |
GoogleHadoopFileSystemConfiguration.GCS_TRACE_LOG_ENABLE
Configuration key to enable logging of additional trace details.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.GCS_WORKING_DIRECTORY
Configuration key for initial working directory of a GHFS instance.
|
static HadoopConfigurationProperty<String> |
GoogleHadoopFileSystemConfiguration.PERMISSIONS_TO_REPORT
Key for the permissions that we report a file or directory to have.
|
Copyright © 2022. All rights reserved.