How can I get my Amazon SQS subscription to successfully receive a notification from my Amazon SNS topic?

Short description

Before you get started, configure Amazon CloudWatch delivery status logging for your SNS topic. For more information, see Monitoring Amazon SNS topics using CloudWatch.

Note:

• For information on alerts for threshold breaches, see Set CloudWatch alarms for Amazon SNS metrics.
• For information on the message delivery status to endpoints, see Configuring delivery status logging using the AWS Management Console.
• To confirm the delivery status, access the topic's delivery logs from CloudWatch. To check that the message is delivered to the queue, compare the SNS topic's NumberOfMessagesPublished and NumberOfNotificationsDelivered metrics with the Amazon SQS queue's NumberOfMessagesSent CloudWatch metric.

Resolution

Validate the subscription status

Check that the subscription is in the Confirmed state and not in the Pending Confirmation or Deleted state. If the status is Pending Confirmation, then confirm the subscription. If the status is Deleted, then recreate the deleted subscription.

The following reasons are why a subscription is in the Confirmed state, but the messages aren't delivered to the queues:

• You didn't configure your SQS queue's access policy or AWS Key Management Service (AWS KMS) key policy.
• The messages are filtered out.
• The messages are deduplicated first in first out (FIFO) topics.
• You activated raw message delivery for the subscriptions.
• The messages were delivered, but you don't see the messages in Amazon SQS.

You receive an Access denied error

Configure your SQS queue's access policy to allow Amazon SNS to send messages

To view the access policy of your SQS queue, configure your access policy.

If your SQS queue's access policy doesn't include the SendMessage API action for your SNS topic, then update your policy with the correct permissions. The permissions must allow Amazon SNS to send messages to the SQS queue.

Configure your AWS KMS key policy to work with SSE on your SQS queue

If server-side encryption (SSE) is turned on for your SQS queue, then follow these steps:

1.    Make sure that the AWS KMS key is activated.

2.    Check if your SQS queue uses a customer managed KMS key. The key policy must grant kms:Decrypt and kms:GenerateDataKey permissions to the SNS topic.

Note: You can't use AWS managed keys in cross-account subscriptions.

3.    To allow the Amazon SNS service to perform kms:GenerateDataKey and kms:Decrypt API actions, add this statement to the KMS key policy:

{    "Sid": "Allow Amazon SNS to use this key",
    "Effect": "Allow",
    "Principal": {
        "Service": "sns.amazonaws.com"
   
    },
    "Action": [
        "kms:Decrypt",
        "kms:GenerateDataKey*"
    ],
    "Resource": "*"
}

If the KMS key policy doesn't grant access to the Amzon SNS principal, then the message delivery status logs show the KMS.AccessDeniedException error:

{    "notification": {
        "messageMD5Sum": "9ecb1c62c0de86ac894c5fd6c447fe1f",
        "messageId": "f762cbcb-8643-5ef3-95c8-9739fe5f07f6",
        "topicArn": "arn:aws:sns:::",
        "timestamp": "2023-06-01 10:41:32.114"
    },
    "delivery": {
        "deliveryId": "f7a48fb3-1d98-5fe8-ae45-d34165842b39",
        "destination": "arn:aws:sqs:::",
        "providerResponse": "{\"ErrorCode\":\"KMS.AccessDeniedException\",\"ErrorMessage\":\"null (Service: AWSKMS; Status Code: 400; Error Code: AccessDeniedException; Request ID: 00df5c7d-1834-4cfb-bc05-e5a7fccfa816; Proxy: null)\",\"sqsRequestId\":\"Unrecoverable\"}",
        "dwellTimeMs": 59,
        "statusCode": 400
    },
    "status": "FAILURE"
}

For more information, see Why aren't messages that I publish to my Amazon SNS topic getting delivered to my subscribed Amazon SQS queue that has server-side encryption activated?

The messages are filtered out

On the CloudWatch console, check the CloudWatch metrics for Amazon SNS. To confirm that the messages are filtered out, check the NumberOfNotificationsFilteredOut metric. If the messages are filtered out, then check the SNS topic subscription filter policy of your SQS queue.

To check the SNS topic subscription policy, complete the following steps:

1.    Open the Amazon SNS console.

2.    On the navigation pane, choose Subscriptions.

3.    Select your subscription, and then choose Edit.

4.    Expand the Subscription filter policy section. Check if the Message attributes or Message body matches the SNS subscription filter policy scope. If the attributes or body don't match, then update the filter policy.

5.    Choose Save changes.

For more information, see Amazon SNS message filtering.

The messages are deduplicated

Based on message content or a deduplication ID, Amazon SNS FIFO topics and Amazon SQS FIFO queues support message deduplication. If ContentBasedDeduplication is set on the topic, then the system generates a MessageDeduplicationId based on the contents of the message. If ContentBasedDeduplication isn't set, then every message that's published to the topic must have a unique MessageDeduplicationId.

Use the MessageDeduplicationId parameter to dedupe messages that are published to the SNS topics within the 5-minute deduplication interval. For more information, see Message deduplication for FIFO topics.

You activated raw message delivery for the subscriptions

If you use raw message delivery, then check that you send no more than 10 message attributes in the published notification. For messages that you activate with raw message delivery, Amazon SNS maps the message attributes to SQS message attributes.

The following example is a delivery status log with more than 10 message attributes:

{    "notification": {
        "messageMD5Sum": "8fa14cdd754f91cc6554c9e71929cce7",
        "messageId": "22540375-973d-5c6d-8d9e-5f870de34c49",
        "topicArn": "arn:aws:sns:<region>:<account>:<topic name>",
        "timestamp": "2023-06-01 13:09:50.947"
    },
    "delivery": {
        "deliveryId": "9ccf7066-19e5-5bbb-a973-2c94833bc5fc",
        "destination": "arn:aws:sqs:<region>:<account>:<queue name>",
        "providerResponse": "{\\"ErrorCode\\":\\"InvalidParameterValue\\",\\"ErrorMessage\\":\\"Number of message attributes \[13\] exceeds the allowed maximum \[10\].\\",\\"sqsRequestId\\":\\"Unrecoverable\\"}",
        "dwellTimeMs": 31,
        "statusCode": 400
    },
    "status": "FAILURE"
}

The messages were delivered, but you don't see the messages in Amazon SQS

A message might get delivered to the queue but isn't available for polling because of these reasons:

• It's currently in flight, or the message is invisible because a consumer received and processed it. To confirm whether the consumer received and processed the message, use the NumberOfMessagesReceived or ApproximateNumberOfMessagesNotVisible CloudWatch metric.
• You configured the message with a delay queue, or the messages were sent with message timer. To check the message, use the ApproximateNumberOfMessagesDelayed CloudWatch metric.

The SNS subscription isn't visible in the SQS console

If you created the SQS subscription from the SNS topic account, then you can't see the subscription in the SQS console. This is because the SNS account owns the subscription, so you can see the subscription only in the SNS console.

If you created the SNS subscription from the Amazon SQS console, then the SQS topic account owns the subscription. You can then see the SNS subscription from the SNS console and the SQS console.

The SQS subscription isn't visible under the SNS topic subscription

This issue occurs when you delete and recreate the SNS topic with the same name, and you don't delete the SQS subscription. You can see the SQS subscription on the SNS console in the Subscriptions list or from the SNS ListSubscriptions API call. To check the timestamp when the operations were performed, review the AWS CloudTrail events for the DeleteTopic and CreateTopic SNS API calls.

Related information

How do I configure my cross-account Amazon SQS endpoint to the Amazon SNS topic?