class TransportClient extends Closeable
Client for fetching consecutive chunks of a pre-negotiated stream. This API is intended to allow efficient transfer of a large amount of data, broken up into chunks with size ranging from hundreds of KB to a few MB.
Note that while this client deals with the fetching of chunks from a stream (i.e., data plane), the actual setup of the streams is done outside the scope of the transport layer. The convenience method "sendRPC" is provided to enable control plane communication between the client and server to perform this setup.
For example, a typical workflow might be: client.sendRPC(new OpenFile("/foo")) --> returns StreamId = 100 client.fetchChunk(streamId = 100, chunkIndex = 0, callback) client.fetchChunk(streamId = 100, chunkIndex = 1, callback) ... client.sendRPC(new CloseStream(100))
Construct an instance of TransportClient using TransportClientFactory. A single
TransportClient may be used for multiple streams, but any given stream must be restricted to a
single client, in order to avoid out-of-order responses.
NB: This class is used to make requests to the server, while TransportResponseHandler is
responsible for handling responses from the server.
Concurrency: thread safe and can be called from multiple threads.
- Alphabetic
- By Inheritance
- TransportClient
- Closeable
- AutoCloseable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new TransportClient(channel: Channel, handler: TransportResponseHandler)
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def close(): Unit
- Definition Classes
- TransportClient → Closeable → AutoCloseable
- Annotations
- @Override()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def fetchChunk(streamId: Long, chunkIndex: Int, callback: ChunkReceivedCallback): Unit
Requests a single chunk from the remote side, from the pre-negotiated streamId.
Requests a single chunk from the remote side, from the pre-negotiated streamId.
Chunk indices go from 0 onwards. It is valid to request the same chunk multiple times, though some streams may not support this.
Multiple fetchChunk requests may be outstanding simultaneously, and the chunks are guaranteed to be returned in the same order that they were requested, assuming only a single TransportClient is used to fetch the chunks.
- streamId
Identifier that refers to a stream in the remote StreamManager. This should be agreed upon by client and server beforehand.
- chunkIndex
0-based index of the chunk to fetch
- callback
Callback invoked upon successful receipt of chunk, or upon any failure.
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- def getChannel(): Channel
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def getClientId(): String
Returns the ID used by the client to authenticate itself when authentication is enabled.
Returns the ID used by the client to authenticate itself when authentication is enabled.
- returns
The client ID, or null if authentication is disabled.
- def getHandler(): TransportResponseHandler
- Annotations
- @VisibleForTesting()
- def getSocketAddress(): SocketAddress
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def isActive(): Boolean
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def removeRpcRequest(requestId: Long): Unit
Removes any state associated with the given RPC.
Removes any state associated with the given RPC.
- requestId
The RPC id returned by
RpcResponseCallback).
- def send(message: ByteBuffer): Unit
Sends an opaque message to the RpcHandler on the server-side.
Sends an opaque message to the RpcHandler on the server-side. No reply is expected for the message, and no delivery guarantees are made.
- message
The message to send.
- def sendMergedBlockMetaReq(appId: String, shuffleId: Int, shuffleMergeId: Int, reduceId: Int, callback: MergedBlockMetaResponseCallback): Unit
Sends a MergedBlockMetaRequest message to the server.
Sends a MergedBlockMetaRequest message to the server. The response of this message is either a
MergedBlockMetaSuccessorRpcFailure.- appId
applicationId.
- shuffleId
shuffle id.
- shuffleMergeId
shuffleMergeId is used to uniquely identify merging process of shuffle by an indeterminate stage attempt.
- reduceId
reduce id.
- callback
callback the handle the reply.
- def sendRpc(message: ByteBuffer, callback: RpcResponseCallback): Long
Sends an opaque message to the RpcHandler on the server-side.
Sends an opaque message to the RpcHandler on the server-side. The callback will be invoked with the server's response or upon any failure.
- message
The message to send.
- callback
Callback to handle the RPC's reply.
- returns
The RPC's id.
- def sendRpcSync(message: ByteBuffer, timeoutMs: Long): ByteBuffer
Synchronously sends an opaque message to the RpcHandler on the server-side, waiting for up to a specified timeout for a response.
- def setClientId(id: String): Unit
Sets the authenticated client ID.
Sets the authenticated client ID. This is meant to be used by the authentication layer.
Trying to set a different client ID after it's been set will result in an exception.
- def stream(streamId: String, callback: StreamCallback): Unit
Request to stream the data with the given stream ID from the remote end.
Request to stream the data with the given stream ID from the remote end.
- streamId
The stream to fetch.
- callback
Object to call with the stream data.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def timeOut(): Unit
Mark this channel as having timed out.
- def toString(): String
- Definition Classes
- TransportClient → AnyRef → Any
- Annotations
- @Override()
- def uploadStream(meta: ManagedBuffer, data: ManagedBuffer, callback: RpcResponseCallback): Long
Send data to the remote end as a stream.
Send data to the remote end as a stream. This differs from stream() in that this is a request to *send* data to the remote end, not to receive it from the remote.
- meta
meta data associated with the stream, which will be read completely on the receiving end before the stream itself.
- data
this will be streamed to the remote end to allow for transferring large amounts of data without reading into memory.
- callback
handles the reply -- onSuccess will only be called when both message and data are received successfully.
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()