Design a distributed job queue.

Question

Anonymous · Accepted Answer

We can use any queue framework to design a distributed job queue. First, we have
to identify a attributes of the data inside the queue record to be used for
partitioning. For example, if we have a queue that contains records for say
deposit transactions. We can use customer id or transaction id as partitioning
key. This way we can create hash for transaction id and evenly distribute the
transactions records over multiple queues. Now, we can assign application
process to de-queue these records for processing them from each of the
partitioned queues. This will provide parallel processing of deposit
transactions by multiple processers reading from these queues.
We can add the time-stamp such transaction create date-time also in the hashing
algorithm for the partition keys. This way transaction in each distributed
queues will be process as first-in-first-out sequence.

Design a distributed job queue.

Practice this question with AI

Go Premium

Community Answers

Unlock Full Access

Practice More Questions