How do I troubleshoot permissions issues with Lambda?

Short description

Lambda function permissions issues arise due to the following reasons:

• The Lambda function doesn’t have permission to run the actions in the code.
• The AWS service responsible for invoking the Lambda function doesn’t have sufficient permission to invoke the function.
• A user account doesn’t have proper permissions to create, update, or delete Lambda resources.
• Lambda doesn’t have permission to create an elastic network interface due to missing Amazon Elastic Compute Cloud (Amazon EC2) permissions.

Resolution

Permissions considerations

• A Lambda function's execution role is an AWS Identity and Access Management (IAM) role that grants the function permissions to access AWS services and resources. You provide this role when you create a function, and Lambda assumes the role when your function is invoked. You can create an execution role that has permissions to send logs to Amazon CloudWatch and to upload trace data to AWS X-Ray.
• You can add or remove permissions from a Lambda function's execution role at any time. You can also configure your Lambda function to use a different role. Add permissions for any AWS services that your function calls with an AWS SDK. Then, add permissions for AWS services that Lambda uses to turn on optional features.
• You provide an execution role when you create a Lambda function. When you invoke your new Lambda function, Lambda automatically provides your function with temporary credentials by assuming the execution role. You don't have to call sts:AssumeRole in your function code. When you add permissions to your Lambda function, also update the function's code or configuration. By updating the function's code or configuration, your function's running instances are forced to stop so that they might be replaced. For more information, see Using resource-based policies for Lambda.
• Use resource-based policies to grant usage permission to other AWS accounts or organizations for each resource. A resource-based policy can give permission to an AWS service to invoke your Lambda function on your behalf.
• To invoke or manage a Lambda function, grant an account permission. Create a single resource-based policy to grant permissions to an entire organization in AWS Organizations. Use resource-based policies to grant invoke permission to an AWS service so that a function can be invoked in response to specified account activities.
• Resource-based policies apply to a single function, version, alias, or layer version. Resource-based policies grant permission to one or more AWS services and accounts. For trusted accounts that need access to multiple resources, or to use API actions that resource-based policies don't support, use cross-account roles. For more information, see Lambda permissions.
• Use IAM identity-based policies to grant users in your account access to Lambda. Identity-based policies can apply to users directly, or to groups and roles that are associated with a user. Grant users in another account permissions to assume a role in your account and to access your Lambda resources.
• Lambda provides AWS managed policies that grant access to Lambda API actions. Managed policies might grant access to other AWS services used to develop and manage Lambda resources. Lambda updates these managed policies as needed to ensure that your users have access to new features when they're released. For more information, see Identity-based IAM policies for Lambda.

Troubleshooting steps

1.    If the Lambda function is supposed to invoke another function or another AWS service and it’s failing, check the Lambda execution role.
For more information, see Lambda execution role.

2.    Verify that the AWS service or user has permission set for the lambda:InvokeFunction action.
For more information, see Actions, resources, and condition keys for AWS Lambda.

3.    In a cross-account scenario, access the source and destination accounts to check the permissions needed to assume an IAM role to invoke the Lambda function.
For more information, see How do I configure a Lambda function to assume an IAM role in another AWS account?

4.    To connect to a VPC, your Lambda function's execution role must have the following permissions:
ec2:CreateNetworkInterface
ec2:DescribeNetworkInterfaces
ec2:DeleteNetworkInterface

5.    Resolve permissions issues by comparing the non-working permissions set with any of your working setup. You might also refer to AWS documentation to verify permissions.

6.    The source account needs to have permission to invoke the Lambda function. Review the event source mapping permissions required for the Lambda function. Event source mapping uses permissions in the function's execution role to read and manage items in the event source. Permissions, event structure, settings, and polling behavior vary by event source. For more information, see Troubleshooting AWS Lambda identity and access.

7.    Check your AWS CloudTrail logs to trace API calls made to Lambda. Review the information collected by CloudTrail to determine:
The request that was made to Lambda.
The IP address from which the request was made.
Who made the request.
When the request was made.
For more information, see Logging Lambda API calls with CloudTrail.

8.    If you still can’t resolve the issue, open a case with AWS Support. Provide the following information in the case:

• The Lambda function ARN.
• The workflow on the Lambda function setup with all included AWS services.
• Details about whether the issue is intermittent or continuous.
• Details about whether the issue is a cross-account scenario.
• The permissions and/or IAM role names involved.
• Complete CloudWatch logs in .txt format from when the issue occurred. These CloudWatch logs are used to identify Lambda function errors that include timeout issues, init durations, and permissions issues.
• The exact timestamp of the issue with the time zone or timestamp in UTC.

Note: AWS Support representatives don’t have access to customer CloudWatch logs due to security and privacy reasons.

Related information

How can I use resource-based policies with AWS Lambda to grant permission to AWS services?

How do I configure a Lambda function to assume an IAM role in another AWS account?