National Renewable Energy Laboratory’s OpenEI.org Case Study

The National Renewable Energy Laboratory (NREL), based in Golden, Colorado, is the U.S. Department of Energy’s primary national laboratory for renewable energy and energy-efficiency research and development. The laboratory includes national research centers and collaborative research facilities, which focus on studying wind , bioenergy, and photovoltaics, a method of converting solar energy into direct current electricity.

Since the White House Office of Management and Budget launched the Open Government Directive in 2009, NREL’s OpenEI.org Open Energy Information portal, has focused on publishing and sharing research data from a variety of sources. The organization currently hosts dozens of public-facing websites, which are updated frequently with the latest energy information. Sharing information and fostering collaboration among scientific communities are long standing goals for NREL. “A lot of our researchers want to share data and collaborate with other scientists in the field, and for us, using cloud-based solutions makes external data access and collaboration easy,” says Chris Webber, senior cloud engineer at NREL.

Based on a program need to build a collaborative data repository for the Marine Hydrokinetic Program, NREL wanted to build a secure, yet collaborative, platform to collect, curate, store, and share moderately sensitive data, which focuses on water power research. As part of this effort, NREL built an environment with a Moderate Authority to Operate (ATO) accreditation from the Federal Information Security Management Act (FISMA). With a FISMA Moderate ATO, NREL maintains all mandated cyber security requirements, while gaining the ability to manage and share moderately sensitive data with other government agencies and research entities.

As it prepared to design the new infrastructure, NREL knew it needed agility and flexibility. “Our goal was to make it easy for analysts and scientists to access and publish data, but we didn’t want to spend our time managing infrastructure to facilitate that. We want to focus on the product—the data itself,” says Webber. For example, NREL uses a dev-ops team approach focused on the needs of the client and ensures that the research metadata is optimized for accessibility. “We need to make sure the right descriptors and keywords are there so we can easily connect our users to all the other research sites,” says Jon Weers, senior web strategist at NREL. “If the data isn’t discoverable, it’s not useful to researchers.”

NREL also needed its new moderate environment to be scalable. “We wanted to make it possible for anyone to use the site, so it had to be scalable enough to respond to that kind of wide-open demand,” says Weers. Additionally, the new environment had to be secure, due to the sensitivity of the data involved.

To meet all these needs, NREL realized the cloud was the right technology choice. “We’re a small team, working with clients that have tight budgets, so we need to be able to leverage scalable resources without investing lots on infrastructure,” says Webber. “It just makes sense to build our data repositories and many other websites on the cloud.”

Prior to creating its moderate ATO environment, NREL had moved the open-source data platform for its Open Energy Information Initiative (OpenEI.org) to Amazon Web Services (AWS). “AWS gave us high availability, flexibility, and scalability for OpenEI, and we were confident it would work for our new environment as well,” says Webber. Additionally, Amazon has its own FISMA Moderate ATO, which appealed to NREL.

After deciding to expand its use of AWS, NREL designed and launched a moderate cloud environment within AWS. In addition to OpenEI, the new AWS environment hosts the new Marine Hydrokinetic Data Repository (MHKDR) application. The Department of Energy uses MHKDR to collect program data, hold the data securely for a certain time frame, and enable restricted access to that data by both the department and national laboratories. Once the data can be released, the public gets access to the data for research purposes. The AWS moderate cloud also hosts a geothermal energy research database and several other sites that focus on collaboration and data access.

NREL supports the moderate cloud with more than 300 Amazon Elastic Compute Cloud (Amazon EC2) instances, and it relies on Elastic Load Balancing to distribute incoming network requests for the environment. To optimize performance of the web applications within the environment, NREL uses Amazon DynamoDB, a NoSQL database service for applications that need single-digit millisecond latency, and the Amazon ElastiCache web service to deploy and scale an in-memory cache. With ElastiCache, NREL can improve application performance by retrieving information from fast in-memory caches instead of depending on slower disk-based databases.

NREL uses Amazon Virtual Private Cloud (Amazon VPC) to build secure private networks within the environment. For data storage, the organization uses Amazon Simple Storage Service (Amazon S3) buckets to store multiple terabytes of research data, and Amazon Elastic Block Storage (Amazon EBS) to provide block storage for the MHKDR, OpenEI, and geothermal energy databases.

To build the cloud environment, the Cloud Team in NREL’s Strategic Energy Analysis Center worked with the Office of the CIO to meet more than 400 ATO-related controls, documenting the controls and validating the overall system security. Open data and all metadata associated with the data sets are shared through an interface that shows the types of data being submitted, descriptions of the data, and time frames for release to the public. Close to 6,000 people are expected to visit the new MHKDR site each month. Metadata from the MHKDR is shared with open data partners, allowing the data on MHKDR to be discoverable on sites like science.gov, in the Thompson Reuters data citation index, and on Data.gov. Users can access MHKDR data from any of these high-profile sites, increasing the number of downloads from the MHKDR by orders of magnitude.

The new AWS moderate cloud environment gives the small Strategic Analysis Cloud Team at NREL an easy way to manage its MHKDR, OpenEI, and geothermal energy websites. “Using AWS, I can manage everything from one API, including looking at performance, monitoring costs, and creating new virtual servers,” says Webber. His team can also use that API to efficiently manage the environment. “We have the agility to rapidly respond to our customers by using AWS,” says Weers. For example, NREL can quickly launch new web servers to support new websites for internal analysis, prototyping, or to meet urgent external demand. “We can launch new websites in a day, as opposed to the several weeks it would take to procure storage and purchase and install hardware. We can accomplish it all with a few API calls.”

NREL also has the scalability it needs to quickly launch virtual machines as more researchers share data on the organization’s websites. “Our MHKDR and geothermal data repositories are built to share data, and with AWS we can support that in a very scalable way,” says Weers. “By being on AWS, we can have all these open data connections available, so anyone can come to our websites and we can be scalable enough to respond to that sort of wide-open demand. If 10 more websites connect to the MHKDR tomorrow, we can very easily accommodate them.”

The organization also saves money by using AWS. “Our data curators have to work less,” Weers says. “The previous environment was more traditional, with in-house virtual machines, and the curators had to push a lot more buttons to launch and manage those machines. Because they no longer have to do that, we’re seeing up to 30 percent cost savings related to that task.”

By using AWS to manage the MHKDR environment, NREL can concentrate more on data quality. “Using AWS, we worry less and spend less time and money on infrastructure. As a result, we can focus more of our attention on content and data quality and on making sure our renewable energy data is discoverable by researchers around the world. Our developers spend less time coding connections and security protocols and more time coding great data and data curation measures,” says Weers.

AWS security controls coupled with the controls implemented by NREL make the ATO environment as secure as internal laboratory servers. “Security is paramount for us. The Department of Energy signed off on this cloud environment because they were comfortable with our security protocols,” says Webber. “Our cybersecurity group helped us manage each of the 400-plus ATO controls we needed, and so many of those controls were already handled by AWS through Amazon’s own ATO accreditation. That really helped us accelerate the adoption of this cloud.”

With its new moderate cloud up and running, NREL has migrated additional web applications to the environment and will continue to do so. “Now that we have this environment, I’m getting several calls a week from researchers and potential collaborators interested in taking advantage of it,” says Weers. “We’re going to see exponential growth because of this new environment, and AWS is a big reason for that.”