Enhanced Networking in EC2

Enhanced Networking in Amazon EC2 (Elastic Compute Cloud) is a feature that enables higher network performance by providing better throughput, lower latency, and reduced jitter compared to traditional networking techniques. It's particularly useful for networking-heavy tasks, including those that require real-time or near-real-time performance.

Here's how Enhanced Networking works in EC2:

  1. Use of Specialized Hardware: Enhanced Networking uses network interface cards (NICs) with Single Root I/O Virtualization (SR-IOV) to bypass the host OS and virtualize the network adapter directly. This reduces overhead by bypassing the hypervisor, resulting in better network performance.

  2. Elastic Network Adapters (ENA): ENA is a custom network interface optimized to deliver high throughput and packet per second (PPS) performance and consistently low latencies. It supports popular operating systems and works with various instance types.

  3. Driver Support: Enhanced Networking requires the right drivers to be installed on the instance. Some AMIs (Amazon Machine Images) come with these drivers pre-installed, while others require manual installation.

  4. Instance Type Support: Not all EC2 instance types support Enhanced Networking. It's mainly available on newer and more compute-optimized instances.

  5. Scalability: Enhanced Networking supports more Packets Per Second (PPS), lower inter-instance latencies, and very low network jitter, allowing for the support of clustered and high-performance computing applications.

  6. Increased Bandwidth: With Enhanced Networking, instances can utilize higher bandwidth, allowing more data to be transferred per unit of time.

  7. Compatibility: While Enhanced Networking offers many benefits, it's essential to verify its compatibility with your operating system, applications, and instance types to ensure it functions as expected.

  8. Configuration: Depending on the instance type and operating system, specific configuration might be required to enable Enhanced Networking. Some instances may have it enabled by default.

  9. Cost: Enhanced Networking features usually come at no additional cost, but associated instances may have higher costs due to their enhanced capabilities.

Enhanced Networking can be a powerful tool for many applications, particularly those that require high network performance. It's essential to understand your application's needs and carefully choose the right instance types, network interfaces, and configurations to make the most of this feature.

Enhanced networking support

All current generation instance types support enhanced networking, except for T2 instances.

You can enable enhanced networking using one of the following mechanisms:

Elastic Network Adapter (ENA)

The Elastic Network Adapter (ENA) supports network speeds of up to 100 Gbps for supported instance types.

The current generation instances use ENA for enhanced networking, except for C4, D2, and M4 instances smaller than m4.16xlarge.

Intel 82599 Virtual Function (VF) interface

The Intel 82599 Virtual Function interface supports network speeds of up to 10 Gbps for supported instance types.

The following instance types use the Intel 82599 VF interface for enhanced networking: C3, C4, D2, I2, M4 (excluding m4.16xlarge), and R3.

For a summary of the enhanced networking mechanisms by instance type, see Summary of networking and storage features.

Test whether enhanced networking is enabled

To deploy test machines use the following guide:

How to launch a single EC2 instance via AWS CLI

The following AMIs include the required ENA module and have ENA support enabled:

  • Amazon Linux 2023

  • Amazon Linux 2

  • Amazon Linux AMI 2018.03 and later

  • Ubuntu 14.04 or later with linux-aws kernel

  • Red Hat Enterprise Linux 7.4 or later

  • SUSE Linux Enterprise Server 12 SP2 or later

  • CentOS 7.4.1708 or later

  • FreeBSD 11.1 or later

  • Debian GNU/Linux 9 or later

To test whether enhanced networking is already enabled, verify that the ena module is installed on your instance and that the enaSupport attribute is set. If your instance satisfies these two conditions, then the ethtool -i eth<n> command should show that the module is in use on the network interface.

Kernel module (ena)

To verify that the ena module is installed, use the modinfo command as shown in the following example.

[ec2-user ~]$ modinfo ena
filename:       /lib/modules/4.14.33-59.37.amzn2.x86_64/kernel/drivers/amazon/net/ena/ena.ko
version:        1.5.0g
license:        GPL
description:    Elastic Network Adapter (ENA)
author:         Amazon.com, Inc. or its affiliates
srcversion:     692C7C68B8A9001CB3F31D0
alias:          pci:v00001D0Fd0000EC21sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd0000EC20sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00001EC2sv*sd*bc*sc*i*
alias:          pci:v00001D0Fd00000EC2sv*sd*bc*sc*i*
depends:
retpoline:      Y
intree:         Y
name:           ena
...

In the above Amazon Linux case, the ena module is installed.

ubuntu:~$ modinfo ena
ERROR: modinfo: could not find module ena

In the above Ubuntu instance, the module is not installed, so you must first install it. For more information, see Enable enhanced networking on Ubuntu.

Instance attribute (enaSupport)

To check whether an instance has the enhanced networking enaSupport attribute set, use one of the following commands. If the attribute is set, the response is true.

  • describe-instances (AWS CLI/AWS CloudShell)

      aws ec2 describe-instances --instance-ids instance_id --query "Reservations[].Instances[].EnaSupport"
    
Network interface driver

Use the following command to verify that the ena module is being used on a particular interface, substituting the interface name that you want to check. If you are using a single interface (default), it this is eth0. If the operating system supports predictable network names, this could be a name like ens5.

In the following example, the ena module is not loaded, because the listed driver is vif.

[ec2-user ~]$ ethtool -i eth0
driver: vif
version:
firmware-version:
bus-info: vif-0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

In this example, the ena module is loaded and at the minimum recommended version. This instance has enhanced networking properly configured.

[ec2-user ~]$ ethtool -i eth0
driver: ena
version: 1.5.0g
firmware-version:
expansion-rom-version:
bus-info: 0000:00:05.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

References

  1. Enhanced networking on Linux

  2. Enable enhanced networking with the Elastic Network Adapter (ENA) on Linux instances

  3. How to launch a single EC2 instance via AWS CLI