Package org.apache.curator.framework.recipes.queue

Class QueueSharder<U,​T extends QueueBase<U>>

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable
    public class QueueSharder<U,​T extends QueueBase<U>>
extends java.lang.Object
implements java.io.Closeable

    A utility for shard a distributed queue.

    Due to limitations in ZooKeeper's transport layer, a single queue will break if it has more than 10K-ish items in it. This class provides a facade over multiple distributed queues. It monitors the queues and if any one of them goes over a threshold, a new queue is added. Puts are distributed amongst the queues.

    NOTE: item ordering is maintained within each managed queue but cannot be maintained across queues. i.e. items might get consumed out of order if they are in different managed queues.

    • Constructor Summary

      Constructors 
      Constructor Description
      QueueSharder​(org.apache.curator.framework.CuratorFramework client, QueueAllocator<U,​T> queueAllocator, java.lang.String queuePath, java.lang.String leaderPath, QueueSharderPolicies policies)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      T getQueue()
      Return one of the managed queues - the selection method cannot be relied on.
      java.util.Collection<java.lang.String> getQueuePaths()
      Return the current set of shard paths
      int getShardQty()
      Return the current number of mananged queues
      void start()
      The sharder must be started

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • QueueSharder

        public QueueSharder​(org.apache.curator.framework.CuratorFramework client,
                    QueueAllocator<U,​T> queueAllocator,
                    java.lang.String queuePath,
                    java.lang.String leaderPath,
                    QueueSharderPolicies policies)
        Parameters:
        client - client
        queueAllocator - allocator for new queues
        queuePath - path for the queues
        leaderPath - path for the leader that monitors queue sizes (must be different than queuePath)
        policies - sharding policies

    • Method Detail

      • start

        public void start()
           throws java.lang.Exception
        The sharder must be started
        Throws:
        java.lang.Exception - errors

      • close

        public void close()
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable

      • getQueue

        public T getQueue()
        Return one of the managed queues - the selection method cannot be relied on. It should be considered a random managed queue.
        Returns:
        a queue

      • getShardQty

        public int getShardQty()
        Return the current number of mananged queues
        Returns:
        qty

      • getQueuePaths

        public java.util.Collection<java.lang.String> getQueuePaths()
        Return the current set of shard paths
        Returns:
        paths