Mastering AWS IAM: Troubleshooting Access Denied Errors Effectively
Troubleshoot AWS IAM AccessDenied errors with CloudTrail, policy evaluation logic, simulators, SCPs, boundaries, and conditions.
Mastering AWS IAM: Troubleshooting Access Denied Errors Effectively
AWS IAM AccessDenied errors mean AWS evaluated your request and did not find an effective permission path. The fastest fix is not guessing at a broader policy. It is identifying the exact action, resource, principal, and condition context that AWS evaluated.
Most IAM failures come from one of four places: no matching Allow, an explicit Deny, a permissions boundary or service control policy limiting the identity, or a condition that does not match the request.
Start with IAM Evaluation Logic
AWS starts from a default deny. A request is allowed only when an applicable policy allows it and no applicable policy denies it.
An explicit Deny wins over every Allow. That deny can come from an identity policy, resource policy, permissions boundary, session policy, service control policy, or another applicable policy type.
For same-account access, identity policies and resource policies often work together. For cross-account access, you usually need permission on both sides: the caller's identity must be allowed to make the request, and the target resource or target account must trust or allow that caller.
Capture the Exact Failed Request
Before editing policies, capture these details:
- Principal: user, role, assumed-role session, or AWS service principal.
- Action: the API operation, such as
s3:GetObjectorec2:RunInstances. - Resource: the ARN or resource ID.
- Region and account.
- Conditions: source IP, VPC endpoint, MFA, tags, encryption context, or requested region.
CloudTrail is usually the best source. Search for the failed event around the time of the error and inspect eventName, userIdentity, resources, errorCode, and errorMessage.
aws cloudtrail lookup-events \
--lookup-attributes AttributeKey=EventName,AttributeValue=RunInstances \
--max-results 10
CloudTrail does not explain every authorization decision in perfect detail, but it often gives you the missing action and the real principal. That alone catches many assumed-role and session-name mix-ups.
Use Simulation and Access Analysis Tools
The IAM Policy Simulator can test many identity-based policy questions before you change production permissions. Select the user or role, choose the service action, provide the resource ARN when possible, and include relevant condition keys.
For recent AWS accounts and services, also check the access-denied message itself. Some services return an encoded authorization failure message. If you have permission, decode it with STS:
aws sts decode-authorization-message \
--encoded-message '<encoded-message-from-error>'
That decoded message can show which policy type contributed to the deny.
IAM Access Analyzer is useful for resource-policy and cross-account questions. It can help you find unintended external access and validate some policies before deployment.
Check Each Policy Layer
Identity-based policies are attached to users, groups, and roles. They answer what the identity can do.
Resource-based policies are attached to resources such as S3 buckets, KMS keys, SQS queues, Lambda functions, and role trust policies. They answer who can use that resource or assume that role.
Permissions boundaries set the maximum permissions for a user or role. They do not grant access by themselves. The effective permissions are the intersection of the identity policy and the boundary.
Service control policies in AWS Organizations set maximum permissions for accounts or organizational units. SCPs do not grant access by themselves either. They can block an action even when an IAM policy allows it.
Session policies and permission tags can also reduce access for assumed-role sessions. If a role works in one path but fails when assumed by a CI job, compare the session policy, session tags, and trust policy conditions.
Common AccessDenied Patterns
Missing Dependent Actions
Some AWS operations require more than the obvious action. For example, launching an encrypted EC2 instance may require EC2 permissions plus KMS permissions for the key. CloudTrail and service documentation are your best references for dependent actions.
Wrong Resource ARN
S3 bucket-level and object-level ARNs are different:
{
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/*"
}
arn:aws:s3:::my-bucket covers the bucket. arn:aws:s3:::my-bucket/* covers objects in the bucket. Many S3 errors come from granting one when the API call needs the other.
Condition Mismatch
Conditions must match the request context exactly. A policy that allows access only through one VPC endpoint will deny requests from another endpoint, from the public AWS endpoint, or from a different region if the condition checks that region.
{
"Effect": "Deny",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::my-sensitive-data",
"arn:aws:s3:::my-sensitive-data/*"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
That statement denies HTTP requests and allows HTTPS requests to continue through the rest of policy evaluation. It does not grant access by itself.
KMS Key Policy Gaps
KMS is a common source of confusing access errors. An IAM policy may allow kms:Decrypt, but the key policy must also allow the principal directly or allow the account to delegate access through IAM policies.
Check the key policy, grants, encryption context conditions, and the service using the key.
A Practical Troubleshooting Flow
First, reproduce or capture the failing call. Get the exact API action and principal from CloudTrail.
Next, search for explicit denies in SCPs, resource policies, permissions boundaries, and identity policies. Explicit denies save time because they override everything else.
Then confirm there is a matching allow for the exact action and resource. Include dependent actions and related resources such as KMS keys, IAM roles passed to services, network interfaces, or log groups.
Finally, test with the narrowest policy change possible. Avoid fixing an unknown deny with Action: "*" or Resource: "*"; that hides the problem and creates a new security risk.
The useful habit is to treat every AccessDenied as an evidence problem. Once you know the principal, action, resource, and policy layer, the fix is usually small and clear.