Protecting data with Amazon S3 Object Lock

Update (12/11/2023): As of November 20, 2023, Amazon S3 supports enabling S3 Object Lock on existing buckets.

Amazon S3 Object Lock is an Amazon S3 feature that allows you to store objects using a write once, read many (WORM) model. You can use WORM protection for scenarios where it is imperative that data is not changed or deleted after it has been written. Whether your business has a requirement to satisfy compliance regulations in the financial or healthcare sector, or you simply want to capture a golden copy of business records for later auditing and reconciliation, S3 Object Lock is the right tool for you.

S3 Object Lock has been assessed for SEC Rule 17a-4(f), FINRA Rule 4511, and CFTC Regulation 1.31 by Cohasset Associates. You can download a copy of the Cohasset Associates Assessment report. You can then provide the assessment report to your regulator when you notify them of your decision to use Amazon S3 for your regulated data.

There is no additional charge for using this feature – so go on and add some retention dates to your objects in S3.

Customers using S3 Object Lock

Many AWS customers use AWS’ WORM storage capabilities (S3 Glacier Vault Lock and S3 Object Lock) today. A regulated broker-dealer uses the WORM features of AWS for compliance with U.S. Securities and Exchange Commission (SEC) and Financial Industry Regulatory Authority, Inc. (FINRA) rules.

Many AWS customers recognize that information is an essential company asset and prefer immutability and strong controls for their valuable data. S3 Object Lock provides such customers with immutable storage and protection from deletion.

Amazon S3 Object Lock is unique because…

Amazon S3 Object Lock is the only cloud object storage WORM capability in the market today that gives you the flexibility to apply retention settings on individual objects in addition to default retention settings for all objects within an S3 bucket. Our financial services customers who store immutable trade records are often required to match the retention duration of their trade records to the duration of the trade. For example, if a customer in the financial services industry needs to store a set of trading records for 7 years and another set of records for 5 years, they can put 7-year and 5-year retention dates on the two sets of records. These customers find this flexibility to be especially useful in matching the retention duration of different objects within the same bucket in S3 to the length of these trades. Other customers who store a large number of objects in the same storage resource and want to apply locks to only a subset of them also benefit from this flexibility.

Let’s start with the basics

Amazon S3 Object Lock provides two ways to manage object retention. The first is retention periods and the second is legal holds.

• A retention period specifies a fixed period of time during which an object remains locked. During this period, your object is WORM-protected and can’t be overwritten or deleted. You apply a retention period either in number of days or number of years with the minimum being 1-day and no maximum limit.
• A legal hold provides the same protection as a retention period, but it has no expiration date. Instead, a legal hold remains in place until you explicitly remove it.

Using Amazon S3 Object Lock, you can prevent an object from being deleted or overwritten for a fixed amount of time, or until the legal hold is removed. An object version can have either a combination or both a retention period and a legal hold. For example, you may have an object with a 1-year retention date plus a legal hold.

Governance mode or Compliance mode?

When setting a retention period for your objects or buckets, you can choose the retention mode you wish to apply to your objects. You can choose either the Governance mode or the Compliance mode for your objects.

You should use the Governance mode if you want to protect objects from being deleted by most users during a pre-defined retention period, but at the same time want some users with special permissions to have the flexibility to alter the retention settings or delete the objects. Users with the s3:BypassGovernanceRetention permission can override or remove governance-mode retention settings. Most customers will use the Governance mode since they don’t have compliant storage requirements.

You should use the Compliance mode if you have a requirement to store compliant data. You should only use the Compliance mode if you never want any user, including the root user in your AWS account, to be able to delete the objects during a pre-defined retention period.

Note: The only way to delete an object under the Compliance mode before its retention date expires is to delete the associated AWS account.

You should use a legal hold when…

We picked the name legal hold, but it is really for any situation where you are not sure for how long you want your objects to stay immutable. This may because you have an active litigation or an upcoming external audit of your data or any other reason and want to keep objects in a WORM state till the audit is complete. You may have an ongoing project utilizing a dataset that you want to keep in a WORM state until the project is complete.

Legal Hold works as an infinite retention period. Once applied it is not possible to delete any object until the hold is released manually. The hold can only be removed by users with special permissions.

What else can I do with S3 Object Lock?

In April 2019, S3 Object Lock added support for Cross-Region Replication (CRR). This means that in addition to locking objects, you can now configure your S3 buckets to enable automatic, asynchronous copying of locked objects, and associated metadata, to an S3 bucket in a different AWS Region. Often, if your data is important enough to use a retention date, it is important enough to replicate it to another AWS region.

Ok, I’m convinced I need S3 Object Lock. Walk me through the set-up process

Before you can lock any objects, you must configure a bucket to use Amazon S3 Object Lock. Let’s create a bucket ‘s3objectlockexample’ and turn on versioning for the bucket. Amazon S3 Object Lock only works for buckets that have versioning enabled.

Note: You can only enable Amazon S3 Object Lock for new buckets. If you want to turn on S3 Object Lock for an existing bucket, contact AWS Support.

Next, expand the ‘Advanced settings’ tab and turn on S3 Object Lock.

Note: We also checked Default encryption and CloudTrail object logging as those are recommended settings.

Now we have a bucket with S3 Object lock turned on. What we’ve done so far doesn’t automatically lock objects that you put into the bucket. For that, we will set up a default retention mode and period on the bucket. To do this, navigate to the S3 Object Lock settings under Advanced settings in the Properties tab of the bucket.

Click on Object Lock, select Enable governance mode and enter a retention period in days.

On the next screen, you will be prompted to confirm your choice.

Note: The default settings we just created will apply to all new objects that are placed in the bucket, unless you explicitly specify a different retention mode and period for an object when you upload it.

Let’s upload a few objects and verify that everything works as expected. We’ll use the upload function on the S3 Console to add a few text files.

Let’s look at the Object lock properties, under the Properties tab, of one of the objects ‘text-file1’ we just uploaded.

The object has been locked with a Retain until date of 2022-05-01, which  is 1000 days from the date the object was uploaded.

You can do the same programmatically as well. In addition, you can modify retention periods, add/remove legal holds for objects already in your bucket.

Additional Information

Amazon S3 Object Lock has been assessed by Cohasset Associates for use in environments that are subject to SEC 17a-4, CTCC, and FINRA regulations. For more information about how Amazon S3 Object Lock relates to these regulations, see the Cohasset Associates Compliance Assessment.

Learn More

Read more about S3 Object Lock and Managing Object Locks.