c

org.apache.spark.network.server

OneForOneStreamManager

class OneForOneStreamManager extends StreamManager

StreamManager which allows registration of an Iterator<ManagedBuffer>, which are individually fetched as chunks by the client. Each registered buffer is one chunk.

Linear Supertypes
StreamManager, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. OneForOneStreamManager
  2. StreamManager
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new OneForOneStreamManager()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def checkAuthorization(client: TransportClient, streamId: Long): Unit

    Verify that the client is authorized to read from the given stream.

    Verify that the client is authorized to read from the given stream.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
    Exceptions thrown

    SecurityException If client is not authorized.

  6. def chunkBeingSent(streamId: Long): Unit

    Called when start sending a chunk.

    Called when start sending a chunk.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  7. def chunkSent(streamId: Long): Unit

    Called when a chunk is successfully sent.

    Called when a chunk is successfully sent.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  8. def chunksBeingTransferred(): Long

    Return the number of chunks being transferred and not finished yet in this StreamManager.

    Return the number of chunks being transferred and not finished yet in this StreamManager.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  9. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  10. def connectionTerminated(channel: Channel): Unit

    Indicates that the given channel has been terminated.

    Indicates that the given channel has been terminated. After this occurs, we are guaranteed not to read from the associated streams again, so any state can be cleaned up.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  14. def getChunk(streamId: Long, chunkIndex: Int): ManagedBuffer

    Called in response to a fetchChunk() request.

    Called in response to a fetchChunk() request. The returned buffer will be passed as-is to the client. A single stream will be associated with a single TCP connection, so this method will not be called in parallel for a particular stream.

    Chunks may be requested in any order, and requests may be repeated, but it is not required that implementations support this behavior.

    The returned ManagedBuffer will be release()'d after being written to the network.

    streamId

    id of a stream that has been previously registered with the StreamManager.

    chunkIndex

    0-indexed chunk of the stream that's requested

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  15. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  17. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  18. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  19. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. def numStreamStates(): Int
    Annotations
    @VisibleForTesting()
  22. def openStream(streamChunkId: String): ManagedBuffer

    Called in response to a stream() request.

    Called in response to a stream() request. The returned data is streamed to the client through a single TCP connection.

    Note the streamId argument is not related to the similarly named argument in the int) method.

    returns

    A managed buffer for the stream, or null if the stream was not found.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  23. def registerStream(appId: String, buffers: Iterator[ManagedBuffer], channel: Channel): Long
  24. def registerStream(appId: String, buffers: Iterator[ManagedBuffer], channel: Channel, isBufferMaterializedOnNext: Boolean): Long

    Registers a stream of ManagedBuffers which are served as individual chunks one at a time to callers.

    Registers a stream of ManagedBuffers which are served as individual chunks one at a time to callers. Each ManagedBuffer will be release()'d after it is transferred on the wire. If a client connection is closed before the iterator is fully drained, then the remaining materialized buffers will all be release()'d, but some buffers like ShuffleManagedBufferIterator, ShuffleChunkManagedBufferIterator, ManagedBufferIterator should not release, because they have not been materialized before requesting the iterator by the next method.

    If an app ID is provided, only callers who've authenticated with the given app ID will be allowed to fetch from this stream.

    This method also associates the stream with a single client connection, which is guaranteed to be the only reader of the stream. Once the connection is closed, the stream will never be used again, enabling cleanup by connectionTerminated.

  25. def streamBeingSent(streamId: String): Unit

    Called when start sending a stream.

    Called when start sending a stream.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  26. def streamSent(streamId: String): Unit

    Called when a stream is successfully sent.

    Called when a stream is successfully sent.

    Definition Classes
    OneForOneStreamManager → StreamManager
    Annotations
    @Override()
  27. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  28. def toString(): String
    Definition Classes
    AnyRef → Any
  29. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  30. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  31. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from StreamManager

Inherited from AnyRef

Inherited from Any

Ungrouped