A few weeks ago I found myself in need of a a place where I could share public encryption keys with others for a side project of mine. As the adjective public implies, there is nothing secret about public keys: they can be shared in the open safely, so that was not a concern. The problem was to find a convenient way to do that. More precisely, I needed a place where I could share certain public keys with everyone, and where anyone could put their public keys to share them with me, and with me only.

In the end, I turned to AWS S3 as it is a natural place to look at when it comes to file storage and sharing. But it took a lot of trial and error before I was actually able to find an appropriate configuration for the bucket. I also put some automation with terraform into the mix, both because I prefer to automate things that I may have to do several times, and because it turned out that I’ll have to bring this inbox of mine up and down at need. The outcome is a terraform module that I have just published on github.

Sounds interesting? Read on!

If you are old enough… ahem… experienced enough to remember when Anonymous FTP servers were the way to share and download files and programs, you may also remember that there was often an incoming area, where people could submit their software packages for public distribution. That directory was special though: you were allowed to upload files into it, but fetching them was usually denied, to prevent the server from being abused for illegal exchange of files.

The idea behind the public inbox in S3 was the same: hosting a web site with a special directory: anything outside that directory is publicly accessible in read-only mode; the special directory is a write-only place where only those AWS “entities” designed as the owners of the inbox will have full powers.

Creating an S3 bucket is nothing to be sweating about, nor it is configuring it to host a web site. The difficult part was, of course, setting the ACLs and policies of the bucket right, so that the bucket works as intended. Managing access to S3 buckets is a complex topic because there are many ways to do it, and some of them make more sense than the others depending on what you are trying to do. In that sense, this article on the AWS blog is quite informative, but it’s just a starting point. Once you choose your strategy, you must dive deeper into the documentation, and that’s not a walk in the park. My terraform module packs it all together, providing sensible defaults for ACLs, bucket policies and convenient web site routing rules.

Watch out for vandalism

You bet! Let’s go back for a second to the Anonymous FTP and it’s incoming area. As we said, such a service was prone to abuse if appropriate measures were not put in place; but with those measures in place it was still prone to vandalism. In fact, one could start pushing files and files into the incoming area until the server would be in trouble (exhaustion of the storage quota for the incoming area, or of the storage resources of the FTP server if quotas were not in place).

In S3, we may have bigger problems than that. S3 storage is theoretically unlimited: anyone could push objects in our inbox at will, just for the fun of it, and make you pay for it. The storage occupation cost can be circumvented, or at least mitigated, with a lambda-based clean-up process, but you would still have to pay for the requests (you can read the details about S3 pricing here).

Solutions? Workarounds?

The only workaround that is 100% effective is to build the inbox when it’s needed, and take it down when it’s not. Terraform goes a long way to make that process easy and painless: using the module and some additional terraform code I am able to create the S3 bucket and push the files in it (index.html, error.html and the public keys) in a few seconds. Destroying the whole thing is as fast as creating it.

If creating and destroying the inbox is inconvenient, there is another approach that is not implemented in the module (or not yet): the bucket policy could be altered to open the inbox or lock it down when it’s not needed for public uploads. If you prefer that alley, you’ll have to do lock it down by hand for now, and use terraform to reopen it.

Update: this is now implemented in the release of v2.0.0 of the module; see the CHANGELOG for details.

Look, I just need the ACL/policy/routing rules

Fair enough, here you go. These are the bucket settings for public access:

This is the bucket policy (replace the owner’s ARN and the bucket name with yours):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowOwnerAll",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:user/foo"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::bar/inbox/*",
                "arn:aws:s3:::bar/*",
                "arn:aws:s3:::bar"
            ]
        },
        {
            "Sid": "AllowAnonymousGet",
            "Effect": "Allow",
            "NotPrincipal": {
                "AWS": "arn:aws:iam::123456789012:user/foo"
            },
            "Action": "s3:GetObject",
            "NotResource": "arn:aws:s3:::bar/inbox/*"
        },
        {
            "Sid": "AllowAnonymousPutInbox",
            "Effect": "Allow",
            "NotPrincipal": {
                "AWS": "arn:aws:iam::123456789012:user/foo"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::bar/inbox/*"
        },
        {
            "Sid": "DenyAnonymousAllInbox",
            "Effect": "Deny",
            "NotPrincipal": {
                "AWS": "arn:aws:iam::123456789012:user/foo"
            },
            "NotAction": "s3:PutObject",
            "Resource": "arn:aws:s3:::bar/inbox/*"
        }
    ]
}

And these are the redirection rules for the static web site:

[
    {
        "Condition": {
            "KeyPrefixEquals": "/"
        },
        "Redirect": {
            "ReplaceKeyPrefixWith": "index.html"
        }
    }
]

And that’s all. Until next time, enjoy!