...

Understanding Amazon S3 Block Public Access

15 December 2019

Despite advances in Amazon Web Services (AWS) controls around S3 (Amazon Simple Storage Service), we continue to see data leaks and breaches centered around data stored on S3. In November 2018, Amazon released the Block Public Access feature to make it easier to secure access to S3. Newly created S3 buckets have always been private by default, but there is still confusion around the different ways data in an S3 bucket can become public.

S3 Background

S3 is extremely secure and private by default. The only way S3 buckets become public unintentionally is by misconfiguration. As Gartner points out, the vast majority of cloud data breaches are due to misconfiguration. When a company loses data that was stored in S3, it’s not Amazon’s fault, it’s the company’s fault.

While it’s true that an S3 leak isn’t necessarily Amazon’s fault, the design of the service itself makes it too easy to makes mistakes that lead to data leakage. S3 has a evolved since it’s launch in 2006 and is comprised of a complex set of features and functionality. It is this complexity that makes misconfiguration so easy. The documentation for this simple service is over 140 pages.

Let’s take a look at the ways organizations can identity public S3 buckets and prevent accidentally exposing sensitive data.

Before we do that, however, let’s acknowledge the fact that they are cases where you might actually want a bucket to be public. If you’re hosting a website or static content that you intend to be publicly accessible, then of course the bucket has to be public.

How are Bucket Made Public?

According to AWS documentation, customers may use four mechanisms for controlling access to Amazon S3 resources: IAM policies, bucket policies, ACLs, and Query String Authentication. Though technically true, we think this glosses over a few other common access paths. Namely, CloudFront distributions, Access Points, Object ACLs, and your own custom application code.

Access Control Lists

Access Control Lists (ACLs) are the easiest way to make buckets public. Their lack of granularity also makes them potentially dangerous as the effective access is broad.

Bucket Policies

Bucket policies are much more flexible (and potentially complex) than ACLs. Policies can be created with fine grained permissions and numerous conditional statements to give precise access.

CloudFront

CloudFront is a content delivery network (CDN) that can serve as a frontend to content stored in S3. Even if the underlying S3 bucket has no public access through ACLs or policy, CloudFront can be granted access through IAM policies allowing it to serve up otherwise protected content as if it was publicly accessible.

Access Points

S3 Access Points are unique host names that you can create to enforce distinct permissions and network controls for any request made through the access point. Access points can be configured to serve specific content within S3 buckets. If “Block Public Access” is applied to a bucket, it is also enabled for all Access Points associated with that bucket.

Your Applications

You applications may serve up access to S3 resources by using Query String Authentication or by interfacing directly with the AWS S3 API. With Query String Authentication, you can create a URL to an Amazon S3 object which is only valid for a limited time. However, the options for delivering access programmatically via the API are effectively limitless. Access granted within a custom application is typically much harder to detect and discover since the access mechanism is likely buried in code somewhere separate from the AWS infrastructure itself. This isn’t something we can query the AWS API for. You’ll need knowledge your application’s architecture to determine if it is brokering access to S3 data that is otherwise not publicly accessible.

Object Access Control Lists

Bucket ACLs and policies can be overridden on specific objects. When S3 receives a request for an object, it converts all the relevant permissions - resource-based permissions (object access control list (ACL), bucket policy, bucket ACL) and IAM user policies - into a set of policies to be evaluated at run time. It then evaluates the resulting set of policies in a series of steps. In each step, it evaluates a subset of policies in three specific contexts—user context, bucket context, and object context.

S3 Access Flow Fig. 1: S3 Access Evaluation

If access to an object is not explicitly denied by a bucket ACL or policy, it can still be public via the object ACL.

Discovering Public Buckets

Because of the myriad ways to make S3 buckets and objects public, AWS continues to iterate on services and features to aid customers in managing this data access complexity.

Trusted Advisor

AWS Trusted Advisor was released in 2013 to help customers surface and implement various AWS best practices. At the time, there were relatively few checks and even fewer security-related checks. In 2018, the S3 Bucket Permissions check was added as a free check for every account (previously it was available only to Business and Enterprise support customers). Trusted Advisor is an on-demand query tool that gives recommendations for Cost Optimization, Performance, Security, Fault Tolerance, and Service Limits.

Trusted Advisor Fig. 2: AWS Trusted Advisor Guidance

Access Analyzer for S3

Part of the IAM Access Analyzer, Access Analyzer for S3 will show which buckets have policies resulting in public access as well as the current Block public access setting at the account level. Note the published delay times for detecting changes to policies, ACLs and account-level block public access settings.

When a bucket policy or ACL is added or modified, Access analyzer generates and updates findings based on the change within 30 minutes. Findings related to account-level Block public access settings may not be generated or updated for up to 6 hours after you change the settings.

S3 Access Analyzer Fig. 3: AWS S3 Access Analyzer

AWS Config

AWS Config was released in 2015, but only allowed creating 25 config rules. Since then, Config has been expanded significantly to cover many more AWS resources as well as some third-party resources. Config has several managed S3 rules by default, including s3-account-level-public-access-blocks, s3-bucket-blacklisted-actions-prohibited, s3-bucket-logging-enabled and s3-bucket-public-read-prohibited. Be aware though, that enabling AWS Config rules does incur additional charges, both for storing recorded objects and for rule evaluations.

One of the most interesting features of AWS Config is the built-in remediation functionality.

AWS Config Remediation Fig. 4: AWS Config Remediation

AWS Config Remediation Fig. 5: AWS Config Remediation Action

AWS API

Last but not least, we can always leverage the powerful AWS API to check bucket status programmatically. If you have more complex remediation steps, API integration allows the most flexibility in terms of executing downstream actions.

To simply query whether or not a bucket is publicly accessible, use the get-bucket-policy-status call.

$ aws s3api get-bucket-policy-status --bucket my-public-bucket --output json

{
    "PolicyStatus": {
        "IsPublic": true
    }
}

A bucket with no public access (via ACL or policy) will return:

$ aws s3api get-bucket-policy-status --bucket my-private-bucket --output json

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicyStatus operation: The bucket policy does not exist

It’s important to note what the API is telling us in this case. Since the public status of a bucket involves a complex set of evaluations, get-bucket-policy-status is returning the result of that evaluation. See the meaning of public for a detailed breakdown.

Comparison

When considering the different methods for managing public S3 bucket access, take into account the complexity of your environment, the need for quick response/remediation, and the ability to development and implement custom coded solutions. While a custom development Lambda function, integrated with CloudWatch events can get you near real-time notifications to your DevOps Slack channel, you might be able to save considerable development resources and ongoing management cycles by implementing the pre-built AWS Config rules.

  Notification Frequency Customized Notifications Remediation Support
Trusted Advisor on-demand and/or weekly no no
Access Analyzer on-demand report no no
AWS Config within minutes and/or scheduled interval no yes
API real-time yes custom

Enabling “Block Public Access”

Now that we’ve come full circle and have a clear understanding of what public means, we can move on towards managing public access and restricting it wherever possible. Since AWS released Block Public Access in 2018, organizations have a much more effective method of controlling unintended public S3 access.

In order to enable Block Public Access, enable it at the Access Point, Bucket or Account level.

Amazon S3 Block Public Access provides four settings. These settings are independent and can be used in any combination. Each setting can be applied to an access point, a bucket, or an entire AWS account. If the block public access settings for the access point, bucket, or account differ, then Amazon S3 applies the most restrictive combination of the access point, bucket, and account settings.

What to Watch For

Because of the order in which access grants are evaluated, it is possible to inadvertently block necessary/intended access to S3 resource by enabling RestrictPublicBuckets.

As an example, suppose that a bucket owned by Account-1 has a policy that contains the following:

This policy qualifies as public because of the third statement. With this policy in place and RestrictPublicBuckets enabled, Amazon S3 allows access only by CloudTrail. Even though statement 2 isn’t public, Amazon S3 disables access by Account-2. This is because statement 3 renders the entire policy public, so RestrictPublicBuckets applies. As a result, Amazon S3 disables cross-account access, even though the policy delegates access to a specific account, Account-2. But if you remove statement 3 from the policy, then the policy doesn’t qualify as public, and RestrictPublicBuckets no longer applies. Thus, Account-2 regains access to the bucket, even if you leave RestrictPublicBuckets enabled.