Atlassian Scales to Millions of Databases Using Amazon RDS and Amazon Aurora

Atlassian provides workplace products designed for software developers and project managers to help teams of all sizes succeed. Prior to 2017, Atlassian ran the software products Jira, which facilitates bug tracking and agile project management, and Confluence, which helps teams collaborate and share knowledge efficiently, in its own self-managed data centers. Strong customer growth meant that Atlassian wanted to focus more on supporting its customers and less on managing physical infrastructure. By migrating to AWS, Atlassian established a foundation that would support future growth and innovation. A key component of this foundation was the data store that supported Atlassian’s customers. Atlassian selected Amazon RDS for PostgreSQL, which developers can use to deploy scalable PostgreSQL deployments in minutes with cost-efficient and resizable hardware capacity, as the database for Jira and Confluence. When Atlassian completed its migration to AWS in 2017, the company had 350,000 relational databases running on Amazon RDS.

Atlassian was able to achieve an efficient Amazon RDS environment using a combination of approaches. These approaches included optimizing its database infrastructure and using Amazon RDS Reserved Instances, which provide AWS customers the option to reserve a database instance for a 1- or 3-year term and receive a significant discount compared to the Amazon RDS on-demand database instance pricing. These savings fueled further customer growth and helped Atlassian to focus on innovations for its customers, including the introduction of a free pricing tier for customers wanting to use Atlassian’s cloud offerings. “We knew that we would need to add something bigger as we continued to grow,” says Arul Shaji Arulappan, senior principal engineer at Atlassian. After rigorous testing, Atlassian chose to adopt Amazon Aurora PostgreSQL-Compatible Edition—a relational database service that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases—for its larger “tenants,” or groups of users. The company continues to use Amazon RDS for its smaller tenants, providing flexibility depending on customer need. Initially, Atlassian’s cloud services supported tenants for customers with up to 5,000 users. However, innovation from the Atlassian team soon made it possible for Atlassian’s cloud customers to host up to 25,000 users per tenant on Aurora.

Because Atlassian’s team was familiar with using Amazon RDS for PostgreSQL, developers already had much of the knowledge needed to operate the new Aurora database. “It’s not a new system,” says Arulappan. “It’s the same as Amazon RDS for PostgreSQL, just architected differently.”

Amazon RDS and Aurora offer automations that make daily operations simpler, such as the use of high-level APIs for security groups. Using these, Atlassian can better meet compliance requirements and maintain a standard and consistent security posture for all Amazon RDS and Aurora instances on its solution. Additionally, Atlassian relies on Amazon RDS Multi-AZ Deployments, which provide enhanced availability and durability for Amazon RDS database instances. “Not having to manually configure any of those pieces is a big win for the business,” says Philip O’Sullivan, engineering manager at Atlassian.

Using Amazon RDS and Aurora, which can both automatically scale, the company can improve performance and reduce costs. Atlassian uses both Amazon Aurora Read Replicas and Amazon RDS Read Replicas—which make it easy to elastically scale out beyond the capacity constraints of a single database instance for read-heavy database workloads—during times of peak traffic. The company can then downsize read replicas when they aren’t needed, resulting in significant cost savings.

Atlassian’s Amazon RDS and Aurora databases are managed by two teams—one focused on Amazon RDS provisioning and security and compliance concerns and the other focused on Amazon RDS fleet management, such as tenant rebalancing and performance monitoring. These teams use a variety of AWS solutions to manage the company’s more than 2.8 million databases. For monitoring and alerts, Atlassian’s teams use Amazon CloudWatch, a monitoring and observability service built for DevOps engineers, developers, site reliability engineers, and IT managers. “Using Amazon CloudWatch helps us gain insights into our operational health,” says Arulappan. And by using AWS Key Management Service (AWS KMS)—which makes it easy for customers to create and manage cryptographic keys—Atlassian is able to encrypt the content in its databases with ease.

Another AWS offering that Atlassian has taken advantage of is Amazon RDS Performance Insights, a database performance tuning and monitoring feature that helps users quickly assess and visualize the load on their databases. Its dashboard provides insights such as recent performance times and performance load. If there is a spike in activity, the Atlassian team uses the dashboard to identify which tenant might be causing it. “Amazon RDS Performance Insights is the first thing that the operational team looks to when there is an issue,” says Arulappan. “Most of the time, that’s enough to find out where the problem is.”

Atlassian will continue to use Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL-Compatible Edition to support its growth as it focuses on its business objectives. Instead of devoting time and resources to daily database management, Atlassian can now focus on implementing changes that will be relevant for future enterprise customers. “Using Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL-Compatible Edition reduces the complexity of the scaling process,” says Arulappan. “It’s definitely been a positive experience for Atlassian.” Now, Atlassian is working toward making it possible for its largest customers to host as many as 50,000 users per tenant—making the most of Atlassian’s solution on AWS.