How do I resolve intermittent connection issues when using a NAT instance?

4 minute read
0

I'm using a NAT instance to connect instances to the internet in an Amazon Virtual Private Cloud (Amazon VPC) subnet. But, the instances have intermittent connection issues.

Short description

The intermittent connection issues might be related to the following:

  • Port exhaustion on the source
  • Network access control list (network ACL) rules
  • Network issues

Resolution

Reduce port exhaustion on the source

Verify that the instances in the private subnet reached their operating system-level connection limits. To get the number of active connections, run the netstat command.

Linux:

netstat -ano | grep ESTABLISHED | wc --l
netstat -ano | grep TIME_WAIT | wc --l

Windows:

netstat -ano | find /i "estab" /c
netstat -ano | find /i "TIME_WAIT" /c

Note: If you configured the TCP parameter to reuse ports during the TIME_WAIT state, then remove TIME_WAIT from the preceding commands.

If the command returns a value that's near the allowed local port range (source port for client connections), then you might experience port exhaustion. To reduce port exhaustion, try one of the following solutions.

Increase the operating system's ephemeral port range by running the following command:

For Windows, run as administrator:

netsh int ipv4 set dynamicport tcp start=1025 num=61000

For Linux, run as root:

$echo 1024 65535 > /proc/sys/net/ipv4/ip_local_port_range

Note: Depending on the type of distribution, set the preceding parameter permanently during reboots.

-or-

Resolve any application-level issues that drain the available connections.

Verify that the network ACL rules allow traffic for the ephemeral port range

The network ACL of the public and private subnets must allow traffic for the ephemeral port range (1024-65535).

For example, to allow your Amazon Elastic Compute Cloud (Amazon EC2) instances to access an HTTPS website, the network ACL associated with the private subnet must have the following rules:

Inbound rules:

SourceProtocolPort RangeAllow / Deny
PUBLIC_IPTCP1024-65535ALLOW

Outbound rules:

DestinationProtocolPort RangeAllow / Deny
PUBLIC_IPTCP443ALLOW

The network ACL that's associated with the NAT instance subnet must have the following rules:

Inbound rules:

SourceProtocolPort RangeAllow / Deny
VPC CIDRTCP443ALLOW
PUBLIC_IPTCP1024-65535ALLOW

Outbound rules:

DestinationProtocolPort RangeAllow / Deny
PUBLIC_IPTCP443ALLOW
VPC CIDRTCP1024-65535ALLOW

Traffic is dropped if the following occurs:

  • The network ACL allows only a subset of the ephemeral port range.
  • The instances in the private subnet or NAT instance use a source port that's outside of the ephemeral port range.

Check for network issues

If the network performance is degraded on the source, NAT, network medium, or destination, then you can experience connection issues. To resolve this issue, see How do I troubleshoot network performance issues between Amazon EC2 Linux instances in a VPC and an on-premises host over the internet gateway?

You can also experience network issues when the network exceeds its maximums at the instance level. For supported instance types, you can use the network performance metrics from the Elastic Network Adaptor (ENA) to monitor when traffic exceeds the maximums. To learn more, see Monitor network performance for your EC2 instance.

Related information

Ephemeral ports


AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago