How can I troubleshoot VPC endpoints in my private Elastic Beanstalk environment network configuration?

Short description

Completely private environments have the following setup andallow access to your web application only from within the same Amazon Virtual Private Cloud (Amazon VPC).

• Load balancer visibility: Private
• Load balancer subnets: Both private subnets
• Instance public IP: Disabled
• Instance subnets: Both private subnets

All communication across the VPC takes place through configured VPC endpoints. Also, all communication within the VPC takes place through the local route in the route table. The following are common errors that you might see in the environment because of networking configurations:

• The Amazon Elastic Compute Cloud (Amazon EC2) instances failed to communicate with Elastic Beanstalk. Communication failure is due to configuration problems with the VPC or a failed EC2 instance. Check your VPC configuration, and try launching the environment again.
• The instances didn't respond in the allowed command timeout time.

Resolution

When creating a completely private Elastic Beanstalk environment, make sure that you have the following configurations set up:

• You must have a VPC with at least two private subnets in different Availability Zones.
• The VPC must have DNS hostnames and DNS resolutions turned on. Having these turned on adds a DNS entry in your VPC that maps the public service endpoint to the interface VPC endpoint.
• You must create VPC endpoints so that AWS services outside of the VPC and the instance inside of the VPC can communicate through these endpoints.

Check VPC endpoints

A standard Elastic Beanstalk environment must have the following VPC endpoints for successful creation:

• com.amazonaws.your-region.elasticbeanstalk (Elastic Beanstalk interface endpoint)
• com.amazonaws.your-region.elasticbeanstalk-health (Elastic Beanstalk health interface endpoint)
• com.amazonaws.your-region.cloudformation (AWS CloudFormation interface endpoint)
• com.amazonaws.your-region.logs (logs interface endpoint)
• com.amazonaws.your-region.sqs (Amazon Simple Queue Service (Amazon SQS) interface endpoint)
• com.amazonaws.your-region.s3 (Amazon Simple Storage Service (Amazon S3) gateway endpoint)

Environments running on the Docker platform that runs on 64 bit Amazon Linux 2, must have the following additional VPC endpoints:

• com.amazonaws.your-region.ecr.dkr
• com.amazonaws.your-region.ecr.api

Environments running on the Amazon Elastic Container Service (Amazon ECS) platform that runs on 64 bit Amazon Linux 2, must have the following additional VPC endpoints:

• com.amazonaws.your-region.ecs
• com.amazonaws.your-region.ecr.api
• com.amazonaws.your-region.ecr.dkr

Note: To make sure that your Docker and Multi-Docker environment work properly, provide a private Docker image.

Check the security group

When creating the interface VPC endpoint, you must choose the subnets and the security group for your VPC endpoint. Make sure that you choose private subnets and the security group with the appropriate inbound rules.

The security group that's attached to the interface VPC endpoints must have an inbound rule for port 443 (HTTPS):

Inbound rule:

Type	Protocol	Port range	Source
HTTPS	TCP	443	IP address of client (refer note)

Note: To protect incoming traffic through the VPC endpoint, consider specifying only the IP address or network of the client in the Source.

Also, verify that the security group that's associated to the instance allows HTTP traffic from the load balancer’s security group:

Inbound rule:

Type	Protocol	Port range	Source
HTTP	TCP	80	Load balancer’s security group

Verify that the security group that's associated with the load balancer allows all HTTP traffic in the inbound rule:

Inbound rule:

Type	Protocol	Port range	Source
HTTP	TCP	80	0.0.0.0/0

Note: You don't need to restrict the egress traffic in any of the preceding security groups.

Check for route table entries

When creating gateway endpoints, you have the option to choose a route table. Make sure that you choose the route table that's associated with the private subnet of the VPC. The route table that's associated with the private subnet must have a route for each gateway VPC endpoint. For example, you must have a route for each gateway VPC endpoint that's associated with Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB.

Route table entries:

Destination	Target
172.xx.x.0/16	local
pl-63a5xxx	vpce-04xxx

Check the VPC endpoint policies

The VPC endpoint policies must allow your Elastic Beanstalk environment to communicate privately to the service. When you create an interface or gateway endpoint, you can attach an endpoint policy. The endpoint policy controls which AWS principals can use the VPC endpoint to access the endpoint service. AWS principals include AWS accounts, AWS Identity and Access Management (IAM) users, and IAM roles. If you don't specify an endpoint policy when you create an endpoint, then by default the endpoint has full access.

The following is a sample S3 Gateway VPC endpoint minimum policy that's required for an S3 endpoint when working with Elastic Beanstalk.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "EBBucketAccess",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:DeleteObject",
                "s3:PutBucketPolicy",
                "s3:PutObjectVersionAcl"
            ],
            "Resource": [
                "arn:aws:s3:::elasticbeanstalk-*",
                "arn:aws:s3:::elasticbeanstalk-*/*",
                "arn:aws:s3:::cloudformation*/*"
            ]
        }
    ]
}

---

Related information

Access an AWS service using an interface VPC endpoint