What is Data Governance?

Analytics governance is both governing data for use in analytic applications, as well as governing usage of analytics systems. Your analytics governance team can establish governance mechanisms, such as analytics report versioning and documentation. As always, keep track of regulatory requirements, establish company policy, and provide guardrails to the broader organization.

ML governance applies many of the same data governance practices to ML. Data quality and data integration need to provide the data required for model training and production deployment (feature stores are one important aspect of this). Responsible artificial intelligence (AI) is paying special attention to using sensitive data for building models. Additional ML governance capabilities include enabling people to participate in model building, deployment, and monitoring; documenting model training, versioning, supported use cases, and guiding ethical model use; and monitoring the model in production for accuracy, drift, overfitting, and underfitting.

Generative AI requires additional data governance capabilities, like quality and integrity of data to support adaptation of foundation models for training and for inference, governance of Generative AI toxicity and bias, and foundation model (FM) operations: FMOps.

You can support AI/ML with the same data governance program. Data preparation is necessary to transform data into a form that AI/ML models can use for training and production inference—but the most efficient data preparation is the preparation you don’t have to do. Data scientists spend too much time preparing data for each use case—your data governance team can help alleviate this undifferentiated heavy lifting. In addition, data governance can oversee the creation of shaped feature stores to be used across AI and ML use cases.

Finally, sensitive data needs to be protected appropriately, so your team can mitigate the risks of sensitive data being used to train the foundation models.

Much like analytics in general, you have to govern the use of AI/ML models that you build or customize. Ideally, this should be closely associated with analytics governance, because that function will know how to support various business areas.

Your data governance program should balance centralization and decentralization (including self-service). Throughout your organization, you’ll have a mix of centralized, federated, and decentralized governance—again, depending on the business requirements. You should empower domain teams as much as possible while maintaining coherence across domains (such as the ability to link data together).  

• Centralized data governance: Central organizations are ultimately responsible for mission statements, policies, tool choices, and more. The day-to-day actions are many times pushed into lines of business (LOB).
• Federated data governance: Federated data governance empowers individual business units or business initiatives to operate in the way that best matches their needs. With federated data governance, there is still a smaller centralized team which focuses their work on solving problems that repeat the most frequently, including enterprise-wide data quality tools, for example.
• Self-serve or decentralized data governance: Each LOB does what they need for their specific project. Each project uses any tools or processes from other projects where there is a fit-for-use. As topics like data mesh (itself decentralized) increase in popularity, so does self-service data governance. 

Data governance requires people, process, and technology solutions across a range of capabilities.

Curate data at scale to limit data sprawl. Curating your data at scale means identifying and managing your most valuable data sources, including databases, data lakes, and data warehouses, so you can limit the proliferation and transformation of critical data assets. Curating data also means ensuring that the right data is accurate, fresh, and free of sensitive information so users can have confidence in data-driven decisions and in the data feeding applications.

Capabilities:  Data quality management, data integration, and master data management

Discover and understand your data in context to accelerate data-driven decisions. Understanding your data in context means that all users can discover and comprehend the meaning of their data so they can so they can use it confidently to drive business value. With a centralized data catalog, data can be found easily, access can be requested, and data can be used to make business decisions.

Capabilities: data profiling, data lineage, and data catalogs

Protect and securely share your data with control and confidence. Protecting your data means being able to strike the right balance between data privacy, security, and access. It’s essential to be able to govern data access across organizational boundaries, with tools that are intuitive for both business and engineering users.

Capabilities: Data lifecycle, data compliance, and data security

Reduce business risk and improve regulatory compliance. Reducing risk means understanding how that data is being used and by whom. AWS services help you monitor and audit data access—including access through ML models-- to help ensure data security and regulatory compliance. Machine learning also requires auditing transparency to ensure responsible use and simplified reporting.

Capabilities: usage auditing for data and ML

With end-to-end data governance on AWS, organizations have control over where their data sits, who has access to it, and what can be done with it at every step of the data workflow. Data governance with AWS helps organizations accelerate data-driven decisions by making it easy for the right people and applications to securely and safely find, access, and share the right data when they need it. You can curate data by automating data integration and data quality to limit the proliferation of data. You can discover and understand your data with centralized catalogs that boost data literacy. You can protect your data with precise permissions that let you share data with confidence. You can reduce risk and improve regulatory compliance by monitoring and auditing data access.

• Amazon DataZone – unlock data across organizational boundaries with built-in governance
• AWS Glue – discover, prepare, and integrate all your data at any scale
• AWS Lake Formation – build, manage, and secure data lakes in days
• Amazon QuickSight unified business intelligence at hyperscale
• Amazon SageMaker – build, train, and deploy machine learning models for use cases with fully managed infrastructure, tools, and workflows
• ML governance web page
• Amazon Bedrock – build and scale generative AI applications with foundation models (FMs)
• Amazon Macie - discover and protect sensitive data at scale
• Amazon Simple Storage Service (Amazon S3) access points – object storage built to retrieve any amount of data from anywhere
• AWS Data Exchange – easily find, subscribe to, and use third-party data in the cloud
• AWS Clean Rooms – create clean rooms in minutes to collaborate with your partners without sharing raw data

Get started with Data Governance on AWS by creating a free account today.