How do I troubleshoot issues when I create an AWS Fargate profile?

6 minute read
0

I have issues when I create an AWS Fargate profile.

Short description

A Fargate profile is a mechanism to specify which pods are scheduled on Fargate nodes in an Amazon Elastic Kubernetes Service (Amazon EKS) cluster.

A Fargate profile has selectors that are matched with every incoming pod specification YAML file. The match must be successful and AWS Fargate considerations must be met before the pod is scheduled on Fargate nodes. The pod is scheduled with subnets and the AWS Identity and Access Management (IAM) role specified in the Fargate profile.

Some of the pod placement rules are as follows:

  • If a pod specification has namespace and a match label configuration, then the pod is placed on the Fargate profile that matches the namespace. The pod specification must also match the pod selector's labels
  • If the Fargate profile has multiple pod selector configurations, then the scheduled pod that matches either of the pod selector configurations is used.
  • If a pod specification matches multiple Fargate profiles, then the pod is scheduled according to a random Fargate profile. This occurs unless the following annotation is specified within the pod specification: eks.amazonaws.com/fargate-profile.
  • Kubernetes affinity and anti-affinity rules don't apply and aren't needed with Amazon EKS Fargate Pods.

Resolution

The following are common issues encountered when you create a Fargate profile:

How can I create a Fargate profile to schedule pods on Fargate nodes?

You can use the Amazon EKS console, AWS Command Line Interface (AWS CLI), SDK, or an API to create a Fargate profile.

How can I create a Fargate profile with AWS CloudFormation?

Use the AWS::EKS::FargateProfile CloudFormation resource type to create a Fargate profile.

How can I run the CoreDNS pods on the Fargate nodes only?

By default, CoreDNS is configured to run on Amazon EC2 infrastructure on Amazon EKS clusters. If you want to run the CoreDNS Pods on Fargate serverless compute in your cluster, you must perform the rollout restart of the CoreDNS deployments.

If you created your cluster with eksctl and the --fargate option, then follow the actions in Next steps.

Note: It's a best practice to use eksctl to create or update EKS clusters because it simplifies cluster resource administration. For more information, see EKS Fargate support on the eksctl website.

What are the default limits for a Fargate profile? 

The default limits when you create a Fargate profile are:

  • An EKS cluster can have up to ten Fargate profiles.
  • The Fargate profile can have up to five selectors.
  • The Fargate profile selector can have up to five label pairs.

What pod execution role must be included with the Fargate profile?

The pod execution role is an IAM role that's used by the Fargate node to make AWS API calls. The AmazonEKSFargatePodExecutionRolePolicy managed policy must be attached to this role.

Kubelet on the Fargate node uses this IAM role to communicate with the API server. This role must be included in the aws-auth ConfigMap so that Kubelet can authenticate with the API server. When you create a Fargate profile, the Fargate workflow automatically adds this role to the cluster's aws-auth ConfigMap.

If your Fargate nodes show as 'Not Ready', then make sure that the pod execution role is included in aws-auth ConfigMap.

The following is a sample aws-auth ConfigMap mapRoles section snippet after you create a Fargate profile with a pod execution role:

mapRoles: |       - groups:
      - system:bootstrappers
      - system:nodes
      - system:node-proxier
      rolearn: <Pod_execution_role_ARN>
      username: system:node:{{SessionName}}

If the aws-auth ConfigMap is altered after you create the Fargate profile, then you might receive this warning when pods on Fargate nodes are scheduled:

Pod provisioning timed out (will retry) for pod: <pod_nginx>

I'm want to migrate workloads to EKS Fargate. How do I create subnets and security groups for usage?

EKS Fargate supports only private subnets. This means that there isn't a default route to the internet gateway within the route tables attached to the subnets specified within your Fargate profile. So, you can have either a NAT gateway or VPC endpoints configured for the subnets that you intend to use for the Fargate profile.

The cluster security group is by default attached to Fargate nodes. You don't need to provision a security group specifically for this purpose.

If you use VPC endpoints for your subnets, make sure that the cluster has private endpoint access activated. The security group attached to the VPC endpoints must have an inbound rule that allows HTTPS port 443 traffic from the cluster's VPC CIDR.

I created Fargate profiles with an API-based provisioner such as Terraform or AWS CloudFormation. Why are my Fargate profiles in the CREATE_FAILED state?

Only one Fargate profile can be created or deleted at a time. If you delete a Fargate profile, then no other Fargate profiles can be created or deleted at the same time.

If you use an API-based provisioner, the creation or deletion of a Fargate profile must start after all other Fargate profiles are created or deleted successfully.

Can I specify the resources (CPU, memory) to be provisioned for Fargate nodes within the Fargate profile?

You can't directly specify the amount of resources to be provisioned within the Fargate profile. It's a best practice to specify resource requests within your Fargate pod specification YAML file. Doing so helps the Fargate workflow assign at least that amount of resources for the pod. For more information, see How Kubernetes applies resource requests and limits on the Kubernetes website.

The amount of vCPU or memory that you see after you run the kubectl describe node command might not be the same as the amount that you requested for the pod. The amount of memory and CPU the node has depends on the available capacity in the Fargate resource allocation pool. You're billed based on the amount that you requested within your pod specification. You aren't billed for the amount of resources visible with kubectl.

Note: If you don't specify a vCPU and memory combination, then the smallest available combination is used (0.25 vCPU and 0.5 GB memory).

AWS OFFICIAL
AWS OFFICIALUpdated 2 months ago