Achieving 99.9% data availability to deliver carbon-free electricity

A division of General Electric, GERE provides wind, solar, and hydroelectric power-generation solutions to customers around the world. Each asset contains Internet of Things (IoT) sensors that continually collect status, performance, and event data. Using this data, GERE’s Digital Services team built cloud-native and on-premises asset management solutions used by its contractual services fleet and external customers. “Each sensor emits critical data from the asset. We harness the data and deliver insights to our customers around the globe,” says Prahalad Tarigopula, chief architect of digital engineering at GERE. “We use our system to detect anomalies, predict failures, quickly deploy maintenance, and drive value to our customers.”

Managing thousands of assets to maximize power generation while controlling cost is a concern for all renewable energy companies, and GERE is no exception. Although it had a system in place to manage assets, its expanding fleet and growing customer demand led to operating cost, scalability, and reliability challenges. Siloed applications led to inconsistent user experiences and overlapping information across GERE’s enterprise.

GERE chose to create a new, fully managed, and user-centric Digital Services platform that collects, consumes, and analyzes asset data. It also wanted to present data to customers on a new persona-driven interface. To achieve these goals, GERE turned to AWS. “It was very clear that AWS had all the building blocks that we needed to solve the challenges that we were facing,” says Tarigopula. “For example, we could scale our solution quickly on AWS to serve a large number of users without having to provision, manage, or set up infrastructure.”

In 2020, GERE built its Digital Services platform on AWS. The solution uses a containerized architecture on Amazon Elastic Kubernetes Service (Amazon EKS), a managed container service for running and scaling Kubernetes applications in the cloud or on premises. “Because we have a container-based architecture on Amazon EKS, it is simpler for us to add more applications and digital capabilities to support our solution,” says Tarigopula. GERE used AWS Enterprise Support, which provides companies with a concierge-like service to help them achieve their outcomes and find success in the cloud, to help design its architecture and better understand AWS service capabilities.

GERE’s solution also uses Amazon Managed Streaming for Apache Kafka (Amazon MSK), a fully managed Apache Kafka service that makes it simple to ingest and process logs and other forms of streaming data in near real time. Using Amazon MSK, GERE processes terabytes of data from over 40,000 assets each day, a 500 percent higher volume compared to the previous solution. The platform’s availability also improved on AWS, increasing from 89–92 percent to 99.9 percent. “By migrating to AWS, our team improved our deployment frequency by over five times,” says Brian Case, chief digital officer of Digital Services at GERE. “We can deploy capabilities at a pace that matches the growth of our industry and the needs of our customers.”

In GERE’s new system, after data is processed, it is sent to AWS Glue—a serverless data-integration service that makes it simple to discover, prepare, and combine data for analytics, machine learning (ML), and application development—where it is processed through extract, transform, load jobs. Next, the data is sent to an analytics solution powered by Amazon EMR, which is used to rapidly process, analyze, and apply ML to big data using open-source frameworks. GERE uses Amazon EMR to run analytics on its wind turbine data, which helps it accurately detect and troubleshoot anomalies and deploy preventative maintenance before a turbine’s performance degrades.

After being analyzed, the data can be consumed by more than 300 microservices running on Amazon EKS, helping serve valuable, actionable insights to GERE’s customers. “Using Amazon EKS has fundamentally changed the way we build our services and applications,” says Tarigopula. “We used to have multiple big applications that ran on their own monolithic architecture. Now that we use a container-based architecture, we have a single unified solution where we can bring lots of different applications together in a cohesive way.” GERE has also improved its upgrade and application deployment cycles on AWS. Under its previous solution, deployments were limited to once or twice in 2 weeks. Now, GERE has multiple deployments per day and can deploy new changes on demand.

GERE also improved its ability to scale on AWS. Previously, the company scaled manually when it needed more capacity, which was labor and resource intensive. GERE now automatically scales on demand, removing the need to provision infrastructure. GERE also migrated from a single-tenant to a multitenant solution. Now, it can quickly scale to support new users without the high infrastructure overhead or upfront cost previously required to create a new tenancy on its previous solution. “By using cloud-native, fully managed AWS services, GERE can focus on building the capabilities at scale that deliver direct value to customers while maintaining agility and speed,” says Anwar Ahmed, chief technology officer - Digital Engineering at GERE.

On AWS, GERE is in a strong position to expand its digital capabilities and take on new challenges. Having completed the solution’s initial development, GERE plans to advance its solution using advanced artificial intelligence powered by Amazon EMR and Amazon Redshift, a fully managed, petabyte-scale data warehouse service in the cloud.

GERE also plans to use Amazon SageMaker—which data scientists and developers can use to prepare, build, train, and deploy high-quality ML models quickly—to strengthen its ML capabilities. “We have a natural cadence when working with the AWS team,” says Tarigopula. “Using AWS Enterprise Support, we received the help we needed to build this solution and unlock some of the challenges we faced. It has been a great experience using AWS services during this journey.”