How do I see a list of my Amazon EC2 instances that are connected to Amazon EFS?

6 minute read
0

I want to see a list of my Amazon Elastic Compute Cloud (Amazon EC2) instances that have mounted an Amazon Elastic File System (Amazon EFS). How do I do that?

Short description

The VPC flow logs are used to track the traffic on the elastic network interface of each Amazon EFS mount target. The flow logs can be pushed to Amazon CloudWatch logs. Using CloudWatch logs insights, the traffic flow on the mount target's elastic network interface is filtered to provide the list of Amazon EC2 instances that have mounted an Amazon EFS in a specific timestamp.

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

Perform the following steps once. After completing these steps, each time you want to list the IP addresses of the clients mounting the Amazon EFS, run a query to create a current list.

Create a log group

  1. Open the CloudWatch console.
  2. In the navigation pane, choose Logs, and then choose Log groups.
  3. Choose Create log group.
  4. Enter a Log group name, Retention setting and an optional KMS key ARN. You can also add Tags here.
  5. Choose Create.

Create an Identity and Access Management (IAM) role with permission for publishing flow logs to CloudWatch Logs

  1. Open the IAM console.
  2. In the navigation pane, under Access management, choose Roles.
  3. Choose Create role and create a new IAM role.
  4. The IAM policy that's attached to your IAM role must include the permissions to publish the VPC flow logs to CloudWatch. Similarly, it must have a trust relationship that allows the flow logs service to assume the role.

Get the list of elastic network interfaces used by the mount target of your Amazon EFS

Note: Amazon EFS has a different mount target for each Availability Zone.

  1. Open the Amazon EFS console.
  2. Under File systems, choose the specific Amazon EFS, and then choose View details.
  3. Click on Network, and note the Network Interface ID for each mount target.

Create the flow logs

  1. Open the Amazon EC2 console.
  2. Choose Network & Security, and then choose Network Interfaces.
  3. Choose all the elastic network interfaces that are being used by the mount target.
  4. From the Actions menu, choose Create flow log. Use the following values when creating the flow log:
  • Name: Optional
  • Filter: Select All
  • Maximum aggregation interval: Choose from default 10 mins or 1 min
  • Destination: Select Send to CloudWatch logs
  • Destination log group: Choose the log group you created
  • IAM role: Choose the IAM Role you created
  • Log record format: Chose from AWS default format or Custom format.
  • Tag: Optional
  1. Choose Create.
  2. Monitor the flow log status by choosing the specific elastic network interface that you created a flow log for. At the bottom of the screen, choose Flow logs. Verify that the Status is Active.
  3. The first flow log are pushed to CloudWatch Logs after about 10 minutes.

Verify that the flow logs are in CloudWatch Logs

  1. Open the CloudWatch console, and then choose Logs.
  2. Choose the Log groups created in step 1.
  3. Verify that all the log streams you created now appear. Each elastic network interface has a different log stream.

Run a query

To run a query in CloudWatch Logs Insights:

  1. In the CloudWatch console, choose Logs, and then choose Logs Insights.
  2. Choose the log groups that you created from the drop-down menu.
  3. Choose the duration that you want to review the flow logs for (5m, 30m, 1h, 3h, 12, or Custom).
  4. Enter the query below:
fields @timestamp, @message | filter dstPort="2049" | stats count(*) as FlowLogEntries by srcAddr | sort FlowLogEntries desc

Note: This query reviews all flow logs generated for all mount targets. It filters the logs that have a destination port set to Port=2049 (Amazon EFS clients connect to mount targets on NFS port 2049). It retrieves all unique source IPs (Amazon EFS client IPs), and sorts them by the most active client connections. Activity is determined by the number of entries in the flow log.

  • Choose Run query. The output contains the list of private IPs of all the Amazon EC2 instances where you mounted Amazon EFS.

The following is an example of the query output:

#          srcAddr              FlowLogEntries
1      111.22.33.44                 78
2      111.55.66.77                36
3      111.88.99.000                33

Run a query using the AWS CLI <br>

To run a query from the AWS CLI, follow these steps:

  1. After the VPC flow log is set up, you can use an AWS CLI command to run the query.
  2. Check that the AWS CLI is updated to the latest version:
$ pip install --upgrade awscli
  • Check that jq is installed:
yum install -y jq
  • Use the following AWS CLI query using these query parameters:
  • log-group-name: Enter the log group name you created.
  • start-time / end-time: These values are in Unix/Epoch time. Use the epoch converter to convert human-readable timestamps to Unix/Epoch time.
  • test.json: You can optionally change the json file name each time you run this command. Changing the name makes sure that the previous output isn't merged with the new output.
  • sleep: This value (in seconds) is used as a delay while the CloudWatch Logs insights query is carried out. The value entered depends on how long you want to review the flow logs. If you want to review the logs for a longer duration, such as weeks, then increase the sleep time.
aws logs start-query --log-group-name EFS-ENI-Flowlogs --start-time 1643127618 --end-time 1643128901 --query-string 'filter dstPort="2049" | stats count(*) as FlowLogEntries by srcAddr | sort FlowLogEntries desc' > test.json && sleep 10 && jq .queryId test.json | xargs aws logs get-query-results --query-id

AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago