DOSM Helps Nine Million Users Access Malaysia Census Data on AWS

The Department of Statistics Malaysia (DOSM), which collects national economic and social statistics, became the first Malaysian federal government agency to migrate to Amazon Web Services (AWS). The agency needed to conduct Malaysia’s national census for the year 2020, a process that only takes about 3 months, but it needed to maintain its costly on-premises infrastructure year-round. By switching to cloud services, DOSM could use cloud resources only as needed, achieving cost savings for the government and improving scalability. On the cloud, the agency would also gain access to a wide range of AWS services for everything from managing user traffic to strengthening security.

One particular goal was to migrate Malaysia Statistics Ubiquitous Learning (MySUL)—DOSM’s e-learning solution for training census workers—to the cloud to address access and performance issues. To that end, DOSM reached out to AWS Public Sector Partner IPSB Technology (IPSB), which, since 2008, has provided technology solutions that help its customers meet their business needs. IPSB uses the latest innovative technologies to implement sustainable solutions and partners with local and global information technology leaders to provide its clients with future-ready solutions. DOSM worked with IPSB to use Amazon CloudFront, a content delivery network, alongside DOSM’s on-premises hosting solution, reducing the hosting workload when MySUL reaches peak demand in advance of the census period.

DOSM also struggled to handle spikes in traffic to its on-premises database. It overcame these challenges and worked with Uberfusion Sdn Bhd (Uberfusion), an AWS Advanced Partner, to address traffic spikes using Amazon Aurora Serverless, an on-demand automatic scaling configuration for Amazon Aurora, a relational database built for the cloud. “Our existing on-premises server was too small to serve the nation,” says Razale Ibrahim, principal assistant director at DOSM. “Using AWS, we were able to upgrade our infrastructure, improve the user experience, and increase access to our portal so everyone can get the information they need.”

DOSM has been providing statistics on Malaysia’s economic performance, population, and environment since 1949. In 2019, with the 2020 census on the horizon, DOSM seized the opportunity to address performance issues with its on-premises server, which could only process 50 percent of the nine million users who need access to the census portal, by migrating to the cloud. The agency was confident AWS could handle its census traffic needs. “After I learned about the stability and features AWS can offer, I was eager to test its services,” says Ibrahim.

One primary reason DOSM chose to migrate to AWS was to meet and surpass government information technology security governance guidelines. DOSM worked with Uberfusion to design its cloud architecture using various services such as Amazon GuardDuty, a threat detection service, and AWS Security Hub, which provides a comprehensive view of security alerts from multiple AWS services. To prevent brute-force attacks, DOSM uses AWS Web Application Firewall (AWS WAF), a service that protects web applications from common web exploits. Developers and operators are able to securely access the AWS environment through AWS Client VPN, a fully managed remote-access VPN solution.

DOSM also needed to run its e-learning solution, MySUL, as efficiently and securely as possible. Working with IPSB, DOSM began using automatic scaling in Amazon CloudFront to reduce its hosting workload and operational costs. DOSM uses Amazon CloudFront to cache the hosting for its on-premises MySUL web server. It also uses AWS Certificate Manager, a service for provisioning, managing, and deploying public and private Secure Socket Layer / Transport Layer Security certificates to manage the secure socket layer for MySUL.

Migrating to AWS unlocked a range of services that address DOSM’s diverse needs and workloads, equipping the agency for the census period. Setting up MySUL in the cloud took just 2 weeks, and DOSM used Amazon CloudFront to reduce previous issues that census workers experienced, such as login problems and difficulty accessing multimedia resources because of the limitations of the agency’s on-premises infrastructure. “MySUL includes a lot of multimedia content to help census workers improve their skills and knowledge, but we received a lot of user complaints about access,” says Ibrahim. “Amazon CloudFront is very helpful because we can cache the content for users to access smoothly on the cloud instead of on our server.”

DOSM also upgraded its bandwidth and infrastructure—and achieved the scalability and elasticity it needed—in time to support traffic spikes as users accessed the 2020 census data. Each time DOSM’s server use reached 70 percent, AWS automatically upgraded the server’s RAM, enabling the census portal to handle 100 percent of traffic at any time. Users could access the portal 24/7, and DOSM experienced no downtime or scaling issues, even at a peak of 200,000 simultaneous users. DOSM has saved an estimated 25–50 percent on resource costs because it no longer needs to invest in or maintain hardware and only pays for the bandwidth and services it uses. Because of the flexibility of AWS, the agency has also reduced time and complexity when introducing changes. “In the past, to add an extra server, we had to go through the information and communications technology committee within DOSM and then the ministry, which took about 18 months,” says Ibrahim. “Now that we’re using AWS, we can implement solutions quickly.”

Because of its success, DOSM’s five-member infrastructure team has the complete trust and support of top management to move forward with its efforts to save costs and streamline the agency’s information and communications technology infrastructure. “On AWS we’re able to stay two or three steps ahead and are ready to grab any opportunity that will help DOSM,” says Ibrahim.

Now that DOSM has the flexibility and scalability to handle access needs and traffic spikes before, during, and after census periods, it plans to migrate its remaining workloads to the cloud. The agency plans to phase out all its old servers, achieving additional cost savings for the government and reducing staff workloads. DOSM has already created a plan to fully migrate to the cloud and expects to have 50–70 percent of its workloads on the cloud by 2025.

“Using AWS is very simple and focused, and the AWS teams understand what we need,” says Ibrahim. “We’re quickly moving forward with our project to migrate to the cloud, and our momentum is the result of using AWS.”