How do I invoke a Lambda function through a cross-account Kinesis stream?

3 minute read
0

I want to invoke an AWS Lambda function through an Amazon Kinesis stream that's in another AWS account.

Resolution

Amazon Kinesis Data Streams supports resource-based policies. This allows you to process data ingested into a stream in one account with an AWS Lambda function in another account.

Follow the steps below to create the Lambda function and then configure it to access your Kinesis Data Stream.

Create a Lambda execution role

  1. Access the IAM console and choose the Roles tab.
  2. Choose Create Role.
  3. For AWS Service, choose Trusted Entity Type. For Service, choose Lambda.
  4. In the Permissions policies table, select the managed IAM policy AWSLambdaKinesisExecutionRole.
  5. Provide a name for the role and choose Create Role.

Note: The managed IAM policy AWSLambdaKinesisExecutionRole has the required Kinesis Data Streams and Lambda invocation permissions. It also grants access to all potential Kinesis Data Streams resources that you might be able to access.

Create the Lambda function

Create an AWS Lambda function that can process records in an Amazon Kinesis data stream.

For Execution role, choose the role that you created.

Note: This policy has the required Kinesis Data Streams and Lambda invocation permissions. It also grants access to all potential Kinesis Data Streams resources that you may be able to access.
Add the execution role to the Kinesis Data Streams resource owner to configure the resource policy.

Configure the Kinesis Data Stream

  1. Access the Amazon Kinesis Data Streams console, and choose your data stream.
  2. Choose the Data stream sharing tab.
  3. Choose the Create policy button to start the visual policy editor.
  4. Choose the consumer in the Enhanced fan-out tab to share a registered consumer within a data stream.
  5. Choose Create policy. You can also write the JSON policy directly.
  6. For Principal, enter the cross-account Lambda execution role that you created. Be sure to include the action kinesis:DescribeStream.
    For more information on example resource policies for Kinesis Data Streams, see Example resource-based policies for Kinesis Data Streams.
  7. Choose Create policy. Or, use the PutResourcePolicy to attach the policy to your resource.

If you use server-side encryption with an AWS managed key, and want to share access through a resource policy, then you must use a customer managed key (AWS KMS key). For more information, see What is server-side Encryption for Kinesis Data Streams? Also, you must allow your sharing principal entities to have access to your KMS key through KMS cross-account access. For more information, see Allowing users in other accounts to use a KMS key.

AWS OFFICIAL
AWS OFFICIALUpdated a month ago
3 Comments

On 11/22/2023, Kinesis Data Streams launched support for cross-account access with AWS Lambda using resource-based policies. With resource-based policy, customers can specify AWS accounts, IAM users, or IAM roles and the exact Kinesis Data Streams actions for which they want to grant access. Once access is granted, customers can configure a Lambda function in another account to start processing the data stream. To get started and learn more click here.

replied 4 months ago

Hi AWS-User-2408 after adding the resource-based policies. How it will work? I have assumed some working scenario for the below following cases for the above policies: Case 1: Can a kinesis stream from A account with invoke the lambda function from B account automatically when event is published to kinesis streams? Case 2: Or we can have Event bridge on top B account lambda which will read the data from A account kinesis on certain intervals?

My goal is to achieve this use cases : Some events will get published to kinesis stream in A account, once the event got published kinesis must trigger the lambda from B account. Also please confirm whether I can achieve my use case or not?

Bhuvi
replied 4 months ago

Hi Bhuvi, Once you add a resource-based policy on Kinesis Data Stream Sharing Tab, you are required to add Kinesis as Lambda Trigger in the lambda account(You'll need to add the ARN of kinesis stream. If you're adding in the console, paste the ARN into the search box and continue to save it. Since it is in a different account, the drop down list won't have the Kinesis Stream Name. You can validate your integration by sending new messages into the Kinesis Stream via CLI commands. You. should notice invocations on the lambda given the CW logs are appropriately configured.

Mukesh
replied a month ago