The Queue-Based Load Leveling Pattern

Why use Queue-Based Load Leveling Pattern?

Applications can be subjected to heavy peaks of traffic in intermittent phases. If the applications can’t handle these peaks, this can lead to performance, availability and reliability issues. For example, imagine we have an application that stores state temporarily in a cache. We could have a single task within our application that performs this for us, and we could have a good idea of how many times a single instance of our application would perform this task.

However, if the same service has multiple instances running concurrently, the volume of prequests made to the cache becomes difficult to predict. Peaks ind emand could cause our application to overload the cache and become unresponsive due to the number of requests flooding in.

This is where Queue-Based Load Leveling patterns can help us out.

How Queue-Based Load Leveling Helps the Problem

Introduce a queue between the task and the service. The task and the service run asynchonously. The task posts a message containing the data required by the service to a queue. The queue acts as a buffer and stores the message.