How researchers at The University of Manchester explore magnetic properties of molecules with the AWS Cloud

Dr. Nicholas Chilton and his research group at The University of Manchester’s Department of Chemistry in the School of Natural Sciences.

Dr. Nicholas Chilton and his research group investigate the magnetic properties of molecules for high-density storage, quantum computing, and applications like magnetic resonance imaging (MRI) contrast agents. He is a senior lecturer and Royal Society University research fellow in The University of Manchester’s Department of Chemistry in the School of Natural Sciences. He turned to the cloud when the university’s onsite high-performance computing (HPC) cluster couldn’t provide the high-throughput compute power needed to answer his research questions.

Understanding and measuring how a molecule’s magnetic properties interact with different environments involves tens of thousands of calculations with many variables and degrees of freedom. Dr. Chilton says, “We wanted to know how the shape and flexibility of a molecule are connected to its magnetic properties. This requires calculation of every possible vibration along with its impact to the magnetic properties.”

Dr. Chilton’s research spurred The University of Manchester’s journey to Amazon Web Services (AWS). “We couldn’t have done this before. Right now, we’re working with tens of thousands of calculations. Eventually, as we look at molecules in real-world environments on a surface, in a solid, or in a solution, we will need to perform hundreds of thousands of calculations at once. We needed the cloud,” said Dr. Chilton.

When picking a cloud service provider (CSP), Dr. Simon Hood, head of research platforms and information technology (IT) at The University of Manchester, says the evidence showed that AWS was the best choice for this work. “We spoke to many different CSPs. The difference was that AWS was willing to send tech support to sit down and spend time with us to help us develop resources in the cloud. This was invaluable. The other CSPs were happy to talk high-level, but they were not as forthcoming with technical time and expertise.”

Speeding the time to science from months to weeks with AWS

Dr. Hood says, “Dr. Chilton’s research is comprised of thousands and thousands of relatively small computational jobs. We did some development work on our HTCondor pool and pushed the jobs into Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances.”

This solution uses AWS CloudFormation, an infrastructure-as-code service, with the AWS HTCondor Annex to allow jobs to burst into the cloud. From an end-user perspective, a single line was changed in an HTCondor job submission to make this solution available.

“It worked beautifully, and we were up and running in about two weeks,” recalls Dr. Hood. Switching from their on-premises HPC to Amazon EC2 Spot Instances allowed Dr. Chilton’s research to evolve. He says, “In the past, we relied on empirical models to interpret our results — what we knew experimentally versus what we thought would be happening. The results we’ve obtained with AWS have given us insight at the molecular scale that we couldn’t achieve before.”

Switching from on-premises HPC to AWS also saved time for Dr. Chilton’s team. To complete Dr. Chilton’s calculations onsite with the necessary throughput, The University of Manchester would have to push all users off the platform for a few months. However, using Amazon EC2 Spot Instances, the calculations took six days to complete, a job that would have taken 130 days on premises.

Dr. Chilton says, “We can’t wait for these calculations to take months or years. The field will have moved on if we wait that long to arrive at an answer. Moving to the cloud is about scale. These questions require computational throughput at a new order of magnitude, and we had to do it with the cloud.”

Dr. Chilton says saved time is the biggest benefit of working with AWS. “The possibilities now of what we can do allows us to ask questions that we couldn’t dream of asking before because it was infeasible to get the jobs done. AWS is allowing us to do bigger, grander science.”

As of January 2020, The University of Manchester is continuing to use this solution across their research community. They recently submitted a research project with two million jobs on HTCondor on AWS.

Read more stories from universities around the globe about how they use AWS to further their research, enrich their campuses, and more including stories from The University of Nottingham, University of British Columbia, and the University of Nicosia.

Listen to Fix This podcast episodes to hear how other organizations like Fred Hutch and Emory University use AWS to speed the time to science. Episodes are available on Apple Podcasts, Google Play, Spotify, Stitcher, TuneIn, Overcast, iHeartRadio, and via RSS.