{"id":11107,"date":"2018-11-09T12:07:33","date_gmt":"2018-11-09T12:07:33","guid":{"rendered":"http:\/\/www.lexicalscope.com\/blog\/?p=11107"},"modified":"2018-11-09T12:07:33","modified_gmt":"2018-11-09T12:07:33","slug":"cloudformation-example-of-an-s3-bucket-with-attached-sqs-notifications","status":"publish","type":"post","link":"https:\/\/www.lexicalscope.com\/blog\/2018\/11\/09\/cloudformation-example-of-an-s3-bucket-with-attached-sqs-notifications\/","title":{"rendered":"Cloudformation example of an S3 bucket with attached SQS notifications"},"content":{"rendered":"<p>Creating an s3 bucket with an SQS queue attached is a simple and powerful configuration. Cloudformation allows one to express such a configuration as code and commit it to a git repository. I was not able to find a complete example of how to express such a configuration using Cloudformation. What follows is written using the <a href=\"https:\/\/github.com\/cloudtools\/troposphere\">Troposhere library<\/a>. Please do not take this post to be an endorsement of using Troposhere.<\/p>\n<pre lang=\"python\">\r\nt = self.template\r\n\r\n# The queue which will handle the S3 event messages\r\nt.add_resource(Queue(\r\n    \"MyQueue\",\r\n    VisibilityTimeout=30,\r\n    MessageRetentionPeriod=60,\r\n    QueueName=Sub(\"my-${AWS::Region}-${AWS::AccountId}\")\r\n))\r\n\r\n# The bucket that will generate the s3 events. The NotificationConfiguration\r\n# also supports SNS and Lambda. Notifications can also be filtered according\r\n# the S3 key of the object to which the event relates.\r\nt.add_resource(Bucket(\r\n    \"MyBucket\",\r\n    BucketName=Sub(\"my-${AWS::Region}-${AWS::AccountId}\"),\r\n    # Note that the queue policy must be created first\r\n    DependsOn=\"MyQueuePolicy\",\r\n    NotificationConfiguration=NotificationConfiguration(\r\n        QueueConfigurations=[\r\n            QueueConfigurations(\r\n                Event=\"s3:ObjectCreated:*\",\r\n                Queue=GetAtt(\"MyQueue\", \"Arn\"),\r\n            )\r\n        ]\r\n    )\r\n))\r\n\r\n# The queue policy will give access to the S3 bucket to send on the queue\r\n# The queue policy can also be used to give permission to the message receiver\r\nt.add_resource(QueuePolicy(\r\n    \"MyQueuePolicy\",\r\n    Queues=[Ref(\"MyQueue\")],\r\n    PolicyDocument={\r\n        \"Version\": \"2012-10-17\",\r\n        \"Statement\": [\r\n            # Allow the S3 bucket to publish to the queue\r\n            # https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/dev\/NotificationHowTo.html#grant\r\n            # -destinations-permissions-to-s3\r\n            {\r\n                \"Effect\": \"Allow\",\r\n                \"Principal\": Principal(\"Service\", [\"s3.amazonaws.com\"]),\r\n                \"Action\": [\r\n                    \"SQS:SendMessage\"\r\n                ],\r\n                \"Resource\": GetAtt(\"MyQueue\", \"Arn\"),\r\n                \"Condition\": {\r\n                    \"ArnLike\": {\r\n                        # have to construct the ARN from the static bucket name to avoid\r\n                        # the circular dependency\r\n                        # https:\/\/aws.amazon.com\/premiumsupport\/knowledge-center\/unable-validate-destination-s3\/\r\n                        \"aws:SourceArn\": Join(\"\", [\r\n                            \"arn:aws:s3:::\",\r\n                            Sub(\"my-${AWS::Region}-${AWS::AccountId}\")\r\n                        ])\r\n                    }\r\n                }\r\n            },\r\n            # Allow some user to read from the queue. This is just and example,\r\n            # please change this to match the permissions your use case requires.\r\n            {\r\n                \"Effect\": \"Allow\",\r\n                \"Principal\": AWSPrincipal(GetAtt(\"MyUser\", \"Arn\")),\r\n                \"Action\": [\r\n                    \"sqs:ReceiveMessage\"\r\n                ],\r\n                \"Resource\": GetAtt(\"MyQueue\", \"Arn\"),\r\n            }\r\n        ]\r\n    }\r\n))\r\n\r\n# Allow some user to manipulate the S3 bucket. This is just and example,\r\n# please change this to match the permissions your use case requires.\r\nt.add_resource(BucketPolicy(\r\n    \"MyBucketPolicy\",\r\n    Bucket=Ref(\"MyBucket\"),\r\n    PolicyDocument={\r\n        \"Version\": \"2012-10-17\",\r\n        \"Statement\": [\r\n            {\r\n                \"Effect\": \"Allow\",\r\n                \"Principal\": AWSPrincipal(GetAtt(\"MyUser\", \"Arn\")),\r\n                \"Action\": [\r\n                    \"s3:GetObject\",\r\n                    \"s3:PutObject\",\r\n                    \"s3:DeleteObject\"\r\n                ],\r\n                \"Resource\": Join(\"\", [GetAtt(\"MyBucket\", \"Arn\"), \"\/*\"])\r\n            }\r\n        ]\r\n    }\r\n))\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Creating an s3 bucket with an SQS queue attached is a simple and powerful configuration. Cloudformation allows one to express such a configuration as code and commit it to a git repository. I was not able to find a complete &hellip; <a href=\"https:\/\/www.lexicalscope.com\/blog\/2018\/11\/09\/cloudformation-example-of-an-s3-bucket-with-attached-sqs-notifications\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[17,18],"tags":[],"class_list":["post-11107","post","type-post","status-publish","format-standard","hentry","category-cloud","category-cloudformation"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2e3P7-2T9","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/posts\/11107","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/comments?post=11107"}],"version-history":[{"count":2,"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/posts\/11107\/revisions"}],"predecessor-version":[{"id":11109,"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/posts\/11107\/revisions\/11109"}],"wp:attachment":[{"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/media?parent=11107"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/categories?post=11107"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lexicalscope.com\/blog\/wp-json\/wp\/v2\/tags?post=11107"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}