Moderate content with machine learning services to protect users
This Guidance helps you implement a serverless architecture to efficiently moderate the increasing influx of user-contributed content and sensitive information. This content and information can come from a broad range of industries including gaming, social media, e-commerce, and regulated environments, such as healthcare and financial services.
Architecture Diagram
Step 1
Customers upload their content into the AWS Cloud.
Step 2
Content securely persists into an Amazon Simple Storage Service (Amazon S3) bucket or another data store.
Step 3
Activate workflows, publisher/subscription patterns, and custom code to moderate the content.
Step 4
Process the audio streams within video streams using Amazon Transcribe and Amazon Rekognition, and extract content moderation categories using simple APIs.
Step 5
Use Amazon Transcribe to convert audio into text and leverage natural language processing (NLP) with Amazon Comprehend.
Step 6
Using Amazon Textract and using extract content Amazon Comprehend natural language processing moderate content.
Step 7
Integrate human workforces to customize model vocabularies and image labels using Amazon SageMaker Ground Truth.
Step 8
Bring humans into the loop for scenarios that aren’t fully automatable using Amazon Augmented AI (Amazon A2I).
Step 1
Customers upload their content into the AWS Cloud.
Step 2
Content securely persists into an Amazon Simple Storage Service (Amazon S3) bucket or another data store.
Step 3
Activate workflows, publisher/subscription patterns, and custom code to moderate the content.
Step 4
Process the audio streams within video streams using Amazon Transcribe and Amazon Rekognition, and extract content moderation categories using simple APIs.
Step 5
Use Amazon Transcribe to convert audio into text and leverage natural language processing (NLP) with Amazon Comprehend.
Step 6
Using Amazon Textract and using extract content Amazon Comprehend natural language processing moderate content.
Step 7
Integrate human workforces to customize model vocabularies and image labels using Amazon SageMaker Ground Truth.
Step 8
Bring humans into the loop for scenarios that aren’t fully automatable using Amazon Augmented AI (Amazon A2I).
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
By choosing Lambda and Amazon S3 you can implement shared design standards. Providing you with the ability to share created assets across accounts, environments and teams.
-
SecurityRead the Security whitepaper
This Guidance makes use of managed services to help you reduce the security maintenance tasks as part of the shared responsibility model. While out of scope for this Guidance, you can also validate the integrity of the software that runs in your Lambda functions.
By choosing managed services you ensure AWS API calls are all done through HTTPS endpoints using TLS communication, thus protecting data in transit.
-
ReliabilityRead the Reliability whitepaper
By using Amazon API Gateway you ensure highly available network connectivity for this Guidance public endpoint as well as providing automatic protection against Distributed Denial of Service (DDoS) attacks through AWS Shield at no extra cost.
By using a combination of Amazon EventBridge and Amazon Simple Queue Service (Amazon SQS) you are able to implement loosely couple dependencies, allowing us to isolate the behavior of a component from other components that depend on it, increasing resiliency and agility.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
By choosing managed serverless services, you can offload the need to manage scaling requirements. With Lambda, simply upload your code, and the service will manage everything required to run and scale that code. And Amazon API Gateway handles all the tasks involved in accepting and processing up to hundreds of thousands of concurrent API calls.
-
Cost OptimizationRead the Cost Optimization whitepaper
This Guidance makes use of serverless or application-level services Lambda and Amazon SQS to remove the need to manage resources.
By choosing both managed and serverless services you have the ability to set attributes that can ensure sufficient capacity. You must set and monitor these attributes so that your excess capacity is kept to a minimum and performance is maximized.
-
SustainabilityRead the Sustainability whitepaper
By choosing managed services, you remove the need to identify periods of low or no utilization in your resources.
This Guidance optimizes software and architecture for asynchronous and scheduled jobs by using queue-driven architectures. Various user-contributed content do not require immediate action and as such they can be scheduled to avoid load spikes and resource contention from simultaneous execution.
Implementation Resources
A detailed guide is provided to experiment and use within your AWS account. Each stage of building the Guidance, including deployment, usage, and cleanup, is examined to prepare it for deployment.
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
Content moderation design patterns with AWS managed AI services
Utilize AWS AI services to automate content moderation and compliance
Protect your users, brand, and budget with AI-powered content moderation
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.