ctx: prod
back to posts
$kubectl describepost/terragrunt-state-bucket-rootaccess

Terragrunt stopped writing RootAccess into your state bucket. Your bucket didn't notice

Terragrunt v1.2.0-rc1 shipped on September 24, and one of its breaking changes is a single paragraph about a bucket policy. Since December 2019, every time Terragrunt bootstrapped an S3 state backend it attached a statement with the Sid RootAccess, granting s3:* on the bucket and its objects to arn:aws:iam::<account-id>:root. In 1.2 that statement is gone.

Buckets that already have it keep it. Terragrunt will not touch them. Why that matters is not what most people will assume, and it takes a look at what the statement actually did to see it.

▸The 60-second version — flip through the deck9 slides · swipe →
Terragrunt 1.2 · AWS IAM

:root never meant root

Your state bucket's policy, decoded.

Since 2019

s3:* for the account ARN

Every S3 backend bootstrap added a RootAccess statement. Removed in 1.2.0-rc1.

the misread

It reads like a lock

"Only the root user." AWS docs: an account ARN "does not limit permissions to only the root user."

The real gate

IAM always decided

Same account: an identity policy alone opens S3. AdministratorAccess, PowerUserAccess, the CI role with s3:*.

the trap

1.1 puts it back

With --backend-bootstrap and --non-interactive, Terragrunt 1.1 calls the bucket out of date and re-adds the statement.

Find and strip

get-bucket-policy, jq, put-bucket-policy

Filter out Sid == "RootAccess". If it was the only statement, delete the policy.

The real lock

Deny with an allowlist

An explicit Deny on aws:PrincipalArn beats any identity policy. Keep a break-glass role in it.

takeaway

It meant "ask IAM"

:root in a Principal never meant root.

What Terragrunt wrote

This is the statement, as the bootstrap code built it (EnableRootAccesstoS3Bucket in the S3 backend client), with a made-up account and bucket:

{
  "Sid": "RootAccess",
  "Effect": "Allow",
  "Principal": {
    "AWS": ["arn:aws:iam::111122223333:root"]
  },
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::acme-tofu-state",
    "arn:aws:s3:::acme-tofu-state/*"
  ]
}

It sat next to a second statement, EnforcedTLS, which denies any request where aws:SecureTransport is false. That one is still there in 1.2 and it is a good default. Only RootAccess went away.

The option to avoid it existed all along, skip_bucket_root_access = true, added a few hours after the statement itself, in December 2019. In 1.2 it has nothing left to skip and is deprecated.

How it reads, and what it means

Almost everyone who opens that policy reads :root as the root user, the email-and-password identity that owns the account. Read that way, the statement looks harmless or even protective: only the most privileged identity has full access.

the same line, two readings
How it reads"Only the root user can touch this bucket." Looks like a lock, so nobody audits past it.the misread
What it means"The whole account. IAM decides which identities." The account ARN and the bare account ID behave the same way.the reality

The IAM documentation settles it in one line:

Using the account ARN in the Principal element does not limit permissions to only the root user of the account.

An account principal delegates. Access then goes to whichever users and roles in that account have an identity policy that allows it.

The part that surprised me

Here is where the Terragrunt release notes and the Terragrunt docs say slightly different things. The release notes say the statement "widened reach to state files that routinely hold secrets." The state backend docs say the account already owns the bucket, "so the statement gave the account no reach it did not have."

The docs are the precise version. Inside a single account, S3 grants a request if either the identity policy or the bucket policy allows it and nothing denies it. A role with s3:GetObject on * could read your state before Terragrunt added RootAccess, and it can read it after you remove it. Delegating to the account just hands the decision back to IAM, which already had it.

So removing the statement does not shrink who can read your state. What the statement did, for almost seven years, was make the bucket policy look like it was doing the protecting.

what actually decides access to state
requests3:GetObject on the state key
→
identity policyallows it? that's enough in-account
→
bucket policyonly an explicit Deny can stop it

The list of identities that can open your state file is therefore the list of identities whose IAM policies allow S3 reads on that bucket. In a typical account that includes more than people expect:

IdentityReads state?Why
AdministratorAccessyes* on *
PowerUserAccessyeseverything except IAM, Organizations, Account
AmazonS3FullAccessyess3:* on *
CI role with s3:* on *yesthe shortcut someone took to make the pipeline green
Any of the above, after RootAccess is removedstill yesthe bucket policy was never the gate

State is where the secrets you could not keep out of it end up. Our default is that secrets never land in state: they come from an encrypted store at apply time, and where the provider supports it they go through write-only arguments. That is the half you fix in code, and it only reaches as far as provider support for write-only arguments does. Generated passwords, credentials a provider returns as attributes, and resources written before any of this existed still put secrets into state, which is why the other half, who can open the file, matters.

The trap: Terragrunt 1.1 puts it back

This is the detail that will bite teams who clean up quickly. Before 1.2, Terragrunt did not only add RootAccess at creation. On every bootstrap it checked whether the statement was present, and if it was missing it marked the bucket as needing an update. The check lives in the 1.1.5 source:

if !client.SkipBucketRootAccess {
    enabled, err := client.checkIfBucketRootAccess(ctx, l, bucketName)
    if err != nil {
        return false, toUpdate, err
    }

    if !enabled {
        toUpdate.RootAccess = true

        updates = append(updates, "Bucket Root Access")
    }
}

Then it asks before changing anything:

Remote state S3 bucket acme-tofu-state is out of date. Would you like Terragrunt to update it? (y/n)

In CI you run with --non-interactive, and in that mode every prompt resolves to yes:

The non-interactive flag is set to true, so assuming 'yes' for all prompts

So if you strip the statement while one pipeline somewhere still runs Terragrunt 1.1 with --backend-bootstrap, the next run of that pipeline writes it straight back. Order matters:

the safe order
Move every runner and laptop to 1.2, or set skip_bucket_root_access = true in the root config for the ones you cannot move yet.
Find every state bucket that carries a RootAccess statement.
Confirm the roles that need the bucket can reach it through their own IAM policies.
Strip the statement, then decide whether you want a real lock on top.

Find it and strip it

For a single bucket, the Terragrunt docs give the recipe. Fetch the policy, drop the statement by Sid, put it back:

aws s3api get-bucket-policy --bucket acme-tofu-state --query Policy --output text \
  | jq '.Statement |= map(select(.Sid != "RootAccess"))' > policy.json

aws s3api put-bucket-policy --bucket acme-tofu-state --policy file://policy.json

If you bootstrapped with skip_bucket_enforced_tls, RootAccess may be the only statement in the policy, and the filter leaves an empty list. S3 will not take a policy with no statements. Delete the policy instead:

aws s3api delete-bucket-policy --bucket acme-tofu-state

To find every affected bucket in an account, loop over them and look for the Sid:

for b in $(aws s3api list-buckets --query 'Buckets[].Name' --output text); do
  aws s3api get-bucket-policy --bucket "$b" --query Policy --output text 2>/dev/null \
    | jq -e '.Statement[] | select(.Sid == "RootAccess")' > /dev/null \
    && echo "$b"
done

If you still want the old behaviour, 1.2 lets you opt back in explicitly with enable_bucket_root_access = true in the remote_state config. And if you want to make sure nobody keeps the deprecated option around, the skip-bucket-root-access strict control turns its warning into an error:

terragrunt run --all --strict-control skip-bucket-root-access -- plan

The lock you thought you had

If the goal was "only these identities can read state," the tool for that is an explicit Deny with a principal allowlist. AWS documents this pattern as the replacement for NotPrincipal:

{
  "Sid": "StateReadersOnly",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::acme-tofu-state",
    "arn:aws:s3:::acme-tofu-state/*"
  ],
  "Condition": {
    "ArnNotLike": {
      "aws:PrincipalArn": [
        "arn:aws:iam::111122223333:role/terragrunt-ci",
        "arn:aws:iam::111122223333:role/terragrunt-plan-readonly",
        "arn:aws:iam::111122223333:role/break-glass"
      ]
    }
  }
}

An explicit deny beats any identity policy, so AdministratorAccess on some unrelated role no longer opens your state. Two cautions. Keep a break-glass role in the list, because the same deny applies to whoever tries to fix a mistake in it. And roll it out on a non-production state bucket first, because every principal that is not on the list, including ones you forgot about, stops working the moment you apply it.

What to actually do

  • Upgrade every Terragrunt that runs --backend-bootstrap to 1.2, or pin skip_bucket_root_access = true where you can't yet
  • Run the loop above in each account and list the buckets that still carry RootAccess
  • Strip the statement with the docs recipe, or delete the policy if it was the only statement
  • List the roles that can read each state bucket through IAM, starting with the CI role
  • Add a Deny with an aws:PrincipalArn allowlist if that list is longer than it should be

The honest summary of this change is that Terragrunt removed a line that looked like security and was not. Your exposure is exactly what it was yesterday. The difference is that the bucket policy no longer implies otherwise.


When did you last list who can read your state bucket, and was the answer shorter or longer than you expected?