AWS Simple Queue Service, How SQS Works? | by Kubernetes Advocate | AVM Consulting Blog | Medium

AWS Simple Queue Service, How SQS Works?

  • SQS stands for Simple Queue Service.
  • SQS was the first service available in AWS.
  • Amazon SQS is a web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.
  • Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component where a queue is a temporary repository for messages that are awaiting processing.
  • With the help of SQS, you can send, store and receive messages between software components at any volume without losing messages.
  • Using Amazon sqs, you can separate the components of an application so that they can run independently, easing message management between components.
  • Any component of a distributed application can store the messages in the queue.
  • Messages can contain up to 256 KB of text in any format such as json, xml, etc.
  • Any component of an application can later retrieve the messages programmatically using the Amazon SQS API.
  • The queue acts as a buffer between the component producing and saving data, and the component receives the data for processing. This means that the queue resolves issues that arise if the producer is producing work faster than the consumer can process it, or if the producer or consumer is only intermittently connected to the network.
  • If you got two EC2 instances that are pulling the SQS Queue. You can configure the autoscaling group if several messages go over a certain limit. Suppose the number of messages exceeds 10, then you can add an EC2 instance to process the job faster. In this way, SQS provides elasticity.

Queue Types

  • Standard Queues (default)
  • FIFO Queues (First-In-First-Out)
  • Standard Queue

  • SQS offers a standard queue as the default queue type.
  • It allows you to have an unlimited number of transactions per second.
  • It guarantees that a message is delivered at least once. However, sometime, more than one copy of a message might be delivered out of order.
  • It provides best-effort ordering which ensures that messages are generally delivered in the same order as they are sent but it does not provide a guarantee.
  • FIFO Queue
  • The FIFO Queue complements the standard Queue.
  • It guarantees to order, i.e., the order in which they are sent is also received in the same order.
  • The most important features of a queue are FIFO Queue and exactly-once processing, i.e., a message is delivered once and remains available until the consumer processes and deletes it.
  • FIFO Queue does not allow duplicates to be introduced into the Queue.
  • It also supports message groups that allow multiple ordered message groups within a single Queue.
  • FIFO Queues are limited to 300 transactions per second but have all the capabilities of standard queues.

SQS Visibility Timeout

  • The visibility timeout is the amount of time that the message is invisible in the SQS Queue after a reader picks up that message.
  • If the provided job is processed before the visibility time-out expires, the message will then be deleted from the Queue. If the job is not processed within that time, the message will become visible again and another reader will process it. This could result in the same message being delivered twice.
  • The Default Visibility Timeout is 30 seconds.
  • Visibility Timeout can be increased if your task takes more than 30 seconds.
  • The maximum Visibility Timeout is 12 hours.
  • SQS is pull-based, not push-based.
  • Messages are 256 KB in size.
  • Messages are kept in a queue from 1 minute to 14 days.
  • The default retention period is 4 days.
  • It guarantees that your messages will be processed at least once.