DL1 instances deliver up to 40% better price performance for training deep learning models compared to our latest GPU-based EC2 instances. These instances feature Gaudi accelerators that are purpose-built for training deep learning models. You can also get further cost savings by using EC2 Savings Plan to significantly reduce the cost of training your deep learning models.

Developers across all levels of expertise can get started easily on DL1 instances. They can continue to use their own workflow management services by using AWS DL AMIs and DL Containers to get started on DL1 instances. Advanced users can also build custom kernels to optimize their model performance using Gaudi’s programmable Tensor Processing Cores. (TPCs). Using Habana SynapseAI® tools, they can seamlessly migrate their existing models running on GPU- or CPU-based instances onto DL1 instances with minimal code changes.

DL1 instances support leading ML frameworks such as TensorFlow and PyTorch, enabling you to continue using your preferred ML workflows. You can access optimized models such as Mask R-CNN for object detection and BERT for natural language processing on Habana’s GitHub repository to quickly build, train, and deploy your models. SynapseAI’s rich Tensor Processing Core (TPC) kernel library supports a wide variety of operators and multiple data types for a range of model and performance needs.

DL1 instances are powered by Gaudi accelerators from Habana Labs (an Intel company), that feature eight fully programmable TPCs and 32 GiB of high bandwidth memory per accelerator. They have a heterogeneous compute architecture to maximize training efficiency and a configurable centralized engine for matrix-math operations. They also have the industry’s only native integration of ten 100 Gigabit Ethernet ports on every Gaudi accelerator for low latency communication between accelerators.

The SynapseAI® SDK comprises of a graph compiler and runtime, TPC kernel library, firmware, drivers, and tools. It is integrated with leading frameworks such as TensorFlow and PyTorch. Its communication libraries help in rapidly scaling up to multiple accelerators using the same operations you use for GPU-based instances today. This deterministic scaling results in higher utilization and increased efficiency across a variety of neural network topologies. Using SynapseAI® tools, you can seamlessly migrate and run your existing models onto DL1 instances with minimal code changes.

DL1 instances offer 400 Gbps of networking throughput and connectivity to Amazon Elastic Fabric Adapter (EFA) and Amazon Elastic Network Adapter (ENA) for applications that need access to high-speed networking. For fast access to large datasets, DL1 instances also include 4 TB of local NVMe storage and deliver 8 GB/sec read throughput.

DL1 instances are built on the AWS Nitro System, which is a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware and software to deliver high performance, high availability, and high security while also reducing virtualization overhead.