Message Filtering

What is Amazon SNS Message Filtering?

Architecture

By default, an Amazon SNS topic subscriber receives every message published to the topic. To receive a subset of the messages, a subscriber must assign a filter policy to the topic subscription. A filter policy is a simple JSON object containing attributes that define which messages the subscriber receives.

When you publish a message to a topic, Amazon SNS compares the message attributes to the attributes in the filter policy for each of the topic’s subscriptions. If any of the attributes match, Amazon SNS sends the message to the subscriber. Otherwise, Amazon SNS skips the subscriber without sending the message.

You can simplify your use of Amazon SNS by consolidating your message filtering criteria into your topic subscriptions. This allows you to offload the message filtering logic from subscribers and the message routing logic from publishers, eliminating the need to filter messages by creating a separate topic for each condition. You can use a single topic, differentiating your messages using attributes. Each subscriber receives and processes only the messages accepted by its filter policy.

In this section, you will create a new SQS queue for EU Orders, and create a subscription to the Orders SNS topic with a filter policy to route each message to the EU Orders SQS queue. To indicate EU Orders, you will assign an attribute, location, with a value eu-west to the message.

First, create a new SQS queue for EU Orders.

Step 1: Create the Orders-EU SQS queue

Create an Amazon SQS queue, called Orders-EU, to durably store EU Order messages.

  1. Open the AWS Management Console for SQS in a new tab or window, so you can keep this step-by-step guide open.

  2. Choose Create queue at the top of your queue list.

  3. Select Standard for the type of queue you want to configure.

  4. Enter Orders-EU as the queue Name.

  5. Use the default values for the other configurations and choose Create queue

    Create Queue

Step 2: Subscribe the Orders-EU SQS Queue to the Orders SNS Topic

  1. On the Queue detail page, select Subscribe to Amazon SNS topic.

    Subscribe Queue

  2. Select the topic that ends with Orders and choose Save.

    Subscribe Queue

  3. Verify that the queue is subscribed to the topic.

    Subscribed Queue

Step 3: Create EU Orders subscription filter policy

  1. Open the AWS Management Console for SNS in a new tab or window, so you can keep this step-by-step guide open.

  2. From the list of topics, choose the Orders topic.

    Select Topic

  3. In the Subscriptions search box, enter Orders-EU to find the Orders-EU SQS queue subscription.

    Search Subscriptions

  4. Select the Orders-EU subscription and then choose Edit.

    Select Subscription

  5. On the Edit subscription page, expand the Subscription filter policy section.

  6. In the JSON editor field, paste the following filter policy into the JSON editor. Remember to use the Copy to clipboard icon in the upper-right corner.

    {
      "location": [
        "eu-west"
      ]
    }
  7. Choose Save changes.

    Edit Subscription Filter Policy

    Amazon SNS applies your filter policy to the subscription.

    SNS Subscription Filter changes have a max 5 minute TTL If you follow the steps below and do not see your event, note the time and continue to try periodically for up to 6 minutes. If you still have troubles verifying the event, please notify the workshop staff.


(Recap)

Architecture

You have configured one SNS Topic, Orders, and two SQS queue subscriptions, Orders and Orders-EU:

  1. The Orders subscription does not have a filter policy, so the Orders SQS queue will receive all messages sent to the Orders SNS topic.

  2. The Orders-EU subscription has a filter policy, so the Orders-EU SQS queue will only receive messages with a message attribute, location, with a value of eu-west.

Next, using the Event Generator, you will generate test messages and verify their delivery to the appropriate queues.


Step 4: Publish test messages using the Event Generator

  1. Open the Event Generator for SNS in a new tab or window, so you can keep this step-by-step guide open.

  2. The Orders topic is pre-selected from the list of Topics.

    Event Generator - Topic

Message 1: No message attributes

  1. Choose Publish to publish the message to the Orders SNS topic.

    Event Generator - No Attributes

Message 2: Non-present message attributes

  1. Choose Add attribute to add a message attributes.

    Event Generator - Non-Present Attributes

  2. For the attribute Type, leave the selection as String. For the attribute Name, enter category. For the attribute Value, enter books.

    Event Generator - Non-Present Attributes

  3. Choose Publish to publish the message to the Orders SNS topic.

    Event Generator - Non-present Attributes Publish

Message 3: Non-matching message attributes

  1. For the attribute Type, leave the selection as String. For the attribute Name, enter location. For the attribute Value, enter us-west.

    Event Generator - Non-Matching Attributes

  2. Choose Publish to publish the message to the Orders SNS topic.

    Event Generator - Non-matching Attributes Publish

Message 4: Matching message attributes

  1. For the attribute Type, leave the selection as String. For the attribute Name, enter location. For the attribute Value, enter eu-west.

    Event Generator - Matching Attributes

  2. Choose Publish to publish the message to the Orders SNS topic.

    Event Generator - Matching Attributes Publish

Step 5: Verify message delivery

  1. Open the AWS Management Console for SQS in a new tab or window, so you can keep this step-by-step guide open.

  2. Note that the Orders queue shows 4 messages delivered and the Orders-EU queue only shows 1 message delivered.

    Verify Messages

    This is because only Message 4 had the matching message attribute key and value (location = eu-west) to satisfy the subscription filter.

Step 6: Cleanup

Cleanup the messages by purging the Orders and Orders-EU SQS queues.

  1. From the queue list, select the Orders queue and click the Purge button.

    Select Queue

  2. Enter purge in the confirmation text box and click Purge.

    Purge Queues Confirmation Dialog

  3. Refresh the queue list to verify that the Orders queue has been purged.

    Purge Queue Verify

  4. Repeat the steps above to purge the Orders-EU queue.

CONGRATULATIONS! You have successfully used subscription filters to route messages published to an SNS topic subscription and verified that it was sent to the correct SQS queue. In the next section, you will review several more advanced monitoring examples.