Packages

c

org.apache.spark.network

TransportContext

class TransportContext extends Closeable

Contains the context to create a TransportServer, TransportClientFactory, and to setup Netty Channel pipelines with a org.apache.spark.network.server.TransportChannelHandler.

There are two communication protocols that the TransportClient provides, control-plane RPCs and data-plane "chunk fetching". The handling of the RPCs is performed outside of the scope of the TransportContext (i.e., by a user-provided handler), and it is responsible for setting up streams which can be streamed through the data plane in chunks using zero-copy IO.

The TransportServer and TransportClientFactory both create a TransportChannelHandler for each channel. As each TransportChannelHandler contains a TransportClient, this enables server processes to send messages back to the client on an existing channel.

Linear Supertypes
Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TransportContext
  2. Closeable
  3. AutoCloseable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new TransportContext(conf: TransportConf, rpcHandler: RpcHandler, closeIdleConnections: Boolean, isClientOnly: Boolean)

    Enables TransportContext initialization for underlying client and server.

    Enables TransportContext initialization for underlying client and server.

    conf

    TransportConf

    rpcHandler

    RpcHandler responsible for handling requests and responses.

    closeIdleConnections

    Close idle connections if it is set to true.

    isClientOnly

    This config indicates the TransportContext is only used by a client. This config is more important when external shuffle is enabled. It stops creating extra event loop and subsequent thread pool for shuffle clients to handle chunked fetch requests.

  2. new TransportContext(conf: TransportConf, rpcHandler: RpcHandler, closeIdleConnections: Boolean)
  3. new TransportContext(conf: TransportConf, rpcHandler: RpcHandler)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. def close(): Unit
    Definition Classes
    TransportContext → Closeable → AutoCloseable
  7. def createClientFactory(): TransportClientFactory
  8. def createClientFactory(bootstraps: List[TransportClientBootstrap]): TransportClientFactory

    Initializes a ClientFactory which runs the given TransportClientBootstraps prior to returning a new Client.

    Initializes a ClientFactory which runs the given TransportClientBootstraps prior to returning a new Client. Bootstraps will be executed synchronously, and must run successfully in order to create a Client.

  9. def createServer(): TransportServer
  10. def createServer(bootstraps: List[TransportServerBootstrap]): TransportServer

    Creates a new server, binding to any available ephemeral port.

  11. def createServer(host: String, port: Int, bootstraps: List[TransportServerBootstrap]): TransportServer

    Create a server which will attempt to bind to a specific host and port.

  12. def createServer(port: Int, bootstraps: List[TransportServerBootstrap]): TransportServer

    Create a server which will attempt to bind to a specific port.

  13. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  15. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  16. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  17. def getConf(): TransportConf
  18. def getRegisteredConnections(): Counter
  19. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  20. def initializePipeline(channel: SocketChannel, channelRpcHandler: RpcHandler): TransportChannelHandler

    Initializes a client or server Netty Channel Pipeline which encodes/decodes messages and has a org.apache.spark.network.server.TransportChannelHandler to handle request or response messages.

    Initializes a client or server Netty Channel Pipeline which encodes/decodes messages and has a org.apache.spark.network.server.TransportChannelHandler to handle request or response messages.

    channel

    The channel to initialize.

    channelRpcHandler

    The RPC handler to use for the channel.

    returns

    Returns the created TransportChannelHandler, which includes a TransportClient that can be used to communicate on this channel. The TransportClient is directly associated with a ChannelHandler to ensure all users of the same channel get the same TransportClient object.

  21. def initializePipeline(channel: SocketChannel): TransportChannelHandler
  22. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  23. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  24. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  25. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  26. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  27. def toString(): String
    Definition Classes
    AnyRef → Any
  28. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  29. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  30. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped