How can I allocate memory to tasks in Amazon ECS?

9 minute read
0

I want to use Amazon Elastic Container Service (Amazon ECS) to allocate memory to tasks.

Short description

In Amazon ECS, memory can be defined at both the task level and at each container level. Memory defined at the task level is the hard limit of memory for the task. At the container level, there are two parameters for allocating memory to tasks: memoryReservation (a soft limit) and memory (a hard limit). For tasks that are hosted on Amazon EC2 instances, the task level memory field is optional, and any value can be used. If a task-level memory value is specified, then the container-level memory value is optional. The value for each parameter is subtracted from the available memory units of an Amazon ECS container instance when a task is running. The calculation is based on the soft limit, hard limit, or task-level memory of a task definition. For more information, see Cluster reservation.

Note: The memory and memoryReservation parameters are set as the container definition parameters of an Amazon ECS task definition. If you specify a value for both container-level memory and memoryReservation, then memory must be greater than memoryReservation. If you specify memoryReservation, then that value is subtracted from the available memory resources for the container instance where the container is placed. Otherwise, the value of memory is used. For more information, see Memory.

Resolution

Before you start, check to see that you have an Amazon ECS cluster that includes an Amazon Elastic Compute Cloud (Amazon EC2) instance. For more information on creating a cluster, see Creating a cluster using the classic console. For more information on configuring the cluster and container instance, see Container Instance Memory Management.

View the memory allocations of a container instance

  1. Open the Amazon ECS console.
  2. In the navigation pane, choose Clusters.
  3. Choose the cluster that you created.
  4. Choose the ECS Instances view, and then choose the container instance included with the cluster that you created from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is equal to the memory in the Registered column.
  5. For statistics on the resource usage of the instance, connect to the instance using SSH, and then run the docker stats command.

Create a task definition with a soft limit

  1. From the Amazon ECS console, in the navigation pane, choose Task Definitions.
  2. Choose Create new Task Definition.
  3. For the launch type, choose EC2, and then choose Next step.
  4. For Task Definition Name, enter a name.
  5. In the Container Definitions section, choose Add container.
  6. For Container name, enter a name.
  7. For Image, enter nginx or the appropriate Docker image for your environment.
  8. For Memory Limits (MiB), choose Soft limit, and then enter 700.
  9. Choose Add, and then choose Create.

Run the task definition with a soft limit

  1. From the Amazon ECS console, in the navigation pane, choose Clusters, and then choose the cluster that you created.
  2. Choose the Tasks view, and then choose Run new Task.
  3. Choose Launch Type as EC2, choose the task definition that you created with soft limit, and then choose Run Task. Note: Task Definition and Cluster can be prepopulated with the name of the task definition and cluster that you created earlier if you are using ECS for the first time.
  4. When the Last status column of the task with a soft limit shows as RUNNING, move to next step.
    Note: To update the status of the task to RUNNING, refresh the page.
  5. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is less than the memory in the Registered column.
  6. For statistics on the resource usage of the instance, connect to the instance using SSH, and then run the docker stats command.
  7. Choose Clusters from the navigation pane, and then choose the cluster.
  8. Choose the Tasks view, select the task with a soft limit, and then choose Stop.
  9. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is equal to the memory in the Registered column.

Create a new revision of a task definition with a hard limit

  1. From the Amazon ECS console, in the navigation pane, choose Task Definitions.
  2. Select the task definition that you created with a soft limit, and then choose Create new revision.
  3. In the Container Definitions section, in the Container Name column, choose the container that you added for the task definition with a soft limit.
  4. For Memory Limits (MiB), choose Hard limit, and then enter 1000.
  5. Choose Update, and then choose Create.

Run the revised task definition with a hard limit

  1. From the Amazon ECS console, in the navigation pane, choose Clusters, and then choose the cluster that you created.
  2. Choose the Tasks view, and then choose Run new Task.
  3. Choose Launch Type as EC2
  4. For Task Definition, choose the task definition with a hard limit that you created, and then choose Run Task.
  5. When the Last status column of the revised task with a hard limit shows as RUNNING, move to next step.
    Note: To update the status of the task to RUNNING, refresh the page.
  6. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that available memory in the Available column is less than the memory in the Registered column.
  7. For statistics on the resource usage of the instance, connect to the instance using SSH, and then run the docker stats command.
  8. Choose Clusters from the navigation pane, and then choose the cluster.
  9. Choose the Tasks view, select the task with a hard limit, and then choose Stop.
  10. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is equal to the memory in the Registered column.

Create a new revision of a task definition with both a soft limit and a hard limit

  1. From the Amazon ECS console, in the navigation pane, choose Task Definitions.
  2. Select the task definition that you created with a hard limit, and then choose Create new revision.
  3. In the Container Definitions section, in the Container Name column, choose the container that you added for the task definition with a hard limit.
  4. For Memory Limits (MiB), choose Soft limit, and then enter 700.
  5. Choose Add Hard limit, and then enter 1200.
  6. Choose Update, and then choose Create.

Run the revised task definition with both a soft limit and a hard limit

  1. From the Amazon ECS console, in the navigation pane, choose Clusters, and then choose the cluster that you created.
  2. Choose the Tasks view, and then choose Run new Task.
  3. Choose Launch Type as EC2
  4. For Task Definition, choose the task definition that you created with both a soft limit and a hard limit, and then choose Run Task.
  5. When the Last status column of the task shows as RUNNING, move to next step.
    Note: To update the status of the task to RUNNING, refresh the page.
  6. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is less than the memory in the Registered column.
  7. For statistics on the resource usage of the instance, connect to the instance using SSH, and then run the docker stats command.
  8. Choose Clusters from the navigation pane, and then choose the cluster.
  9. Choose the Tasks view, select the task with both a soft limit and a hard limit, and then choose Stop.
  10. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is equal to the memory in the Registered column.

Create a new revision of a task definition with Task Level Memory limit

  1. From the Amazon ECS console, in the navigation pane, choose Task Definitions.
  2. Select the task definition that you created with a hard and soft limit, and then choose Create new revision.
  3. Under the Task Size section, for Task memory (MiB) enter 1000
  4. In the Container Definitions section, in the Container Name column, choose the container that you added for the task definition with a hard and soft limit.
  5. For Memory Limits (MiB), remove Soft limit, by choosing the x icon on the right side
  6. Next for Hard limit, remove the value 1200 by selecting it and deleting it.
  7. Choose Update, and then choose Create.

Run the revised task definition with Task Level Memory limit

  1. From the Amazon ECS console, in the navigation pane, choose Clusters, and then choose the cluster that you created.
  2. Choose the Tasks view, and then choose Run new Task.
  3. Choose Launch Type as EC2
  4. For Task Definition, choose the task definition that you created with Task Level memory, and then choose Run Task.
  5. When the Last status column of the task shows as RUNNING, move to next step.
    Note: To update the status of the task to RUNNING, refresh the page.
  6. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is less than the memory in the Registered column.
  7. For statistics on the resource usage of the instance, connect to the instance using SSH, and then run the docker stats command.
    Note: You might observe that the memory limit shown by the docker stats command might not be 1000 MiB for the container. This is because the Task Level memory is managed by ECS Agent and not by docker daemon.
  8. Choose Clusters from the navigation pane, and then choose the cluster.
  9. Choose the Tasks view, select the task with Task level memory, and then choose Stop.
  10. Choose the ECS Instances view, and then choose the instance from the Container Instance column.
    Note: The Details pane shows that the memory in the Available column is equal to the memory in the Registered column.

Related information

Amazon ECS CloudWatch metrics

Amazon EC2 Instance types

Limit a container's access to memory

Task size

AWS OFFICIAL
AWS OFFICIALUpdated a year ago