Cloudformation example of an S3 bucket with attached SQS notifications

Creating an s3 bucket with an SQS queue attached is a simple and powerful configuration. Cloudformation allows one to express such a configuration as code and commit it to a git repository. I was not able to find a complete example of how to express such a configuration using Cloudformation. What follows is written using the Troposhere library. Please do not take this post to be an endorsement of using Troposhere.

t = self.template
 
# The queue which will handle the S3 event messages
t.add_resource(Queue(
    "MyQueue",
    VisibilityTimeout=30,
    MessageRetentionPeriod=60,
    QueueName=Sub("my-${AWS::Region}-${AWS::AccountId}")
))
 
# The bucket that will generate the s3 events. The NotificationConfiguration
# also supports SNS and Lambda. Notifications can also be filtered according
# the S3 key of the object to which the event relates.
t.add_resource(Bucket(
    "MyBucket",
    BucketName=Sub("my-${AWS::Region}-${AWS::AccountId}"),
    # Note that the queue policy must be created first
    DependsOn="MyQueuePolicy",
    NotificationConfiguration=NotificationConfiguration(
        QueueConfigurations=[
            QueueConfigurations(
                Event="s3:ObjectCreated:*",
                Queue=GetAtt("MyQueue", "Arn"),
            )
        ]
    )
))
 
# The queue policy will give access to the S3 bucket to send on the queue
# The queue policy can also be used to give permission to the message receiver
t.add_resource(QueuePolicy(
    "MyQueuePolicy",
    Queues=[Ref("MyQueue")],
    PolicyDocument={
        "Version": "2012-10-17",
        "Statement": [
            # Allow the S3 bucket to publish to the queue
            # https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html#grant
            # -destinations-permissions-to-s3
            {
                "Effect": "Allow",
                "Principal": Principal("Service", ["s3.amazonaws.com"]),
                "Action": [
                    "SQS:SendMessage"
                ],
                "Resource": GetAtt("MyQueue", "Arn"),
                "Condition": {
                    "ArnLike": {
                        # have to construct the ARN from the static bucket name to avoid
                        # the circular dependency
                        # https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-destination-s3/
                        "aws:SourceArn": Join("", [
                            "arn:aws:s3:::",
                            Sub("my-${AWS::Region}-${AWS::AccountId}")
                        ])
                    }
                }
            },
            # Allow some user to read from the queue. This is just and example,
            # please change this to match the permissions your use case requires.
            {
                "Effect": "Allow",
                "Principal": AWSPrincipal(GetAtt("MyUser", "Arn")),
                "Action": [
                    "sqs:ReceiveMessage"
                ],
                "Resource": GetAtt("MyQueue", "Arn"),
            }
        ]
    }
))
 
# Allow some user to manipulate the S3 bucket. This is just and example,
# please change this to match the permissions your use case requires.
t.add_resource(BucketPolicy(
    "MyBucketPolicy",
    Bucket=Ref("MyBucket"),
    PolicyDocument={
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": AWSPrincipal(GetAtt("MyUser", "Arn")),
                "Action": [
                    "s3:GetObject",
                    "s3:PutObject",
                    "s3:DeleteObject"
                ],
                "Resource": Join("", [GetAtt("MyBucket", "Arn"), "/*"])
            }
        ]
    }
))

Leave a Reply