- A McKinsey survey shows that data-driven enterprises are 23% more profitable than their peers, yet fewer than 25% of companies consider themselves to have a mature data governance framework[6]
- DAMA-DMBOK defines 11 knowledge areas of data management, with Data Governance positioned as the core supervisory function that spans all other areas[1]
- Google's research on production ML systems found that over 80% of time in machine learning projects is spent on data collection, cleaning, and feature engineering, with data quality directly determining model success or failure[7]
- Data platform architecture, through a three-tier design of "Data Lake, Data Warehouse, and Feature Platform," elevates data from siloed management to a strategic asset shared across the entire enterprise[5]
1. What Is Data Governance? Why the AI Era Demands It Even More
Data Governance is an organization-level set of strategies, processes, standards, and role definitions designed to ensure the availability, integrity, security, and compliance of enterprise data. It is not a tool, a system, or a single department's responsibility -- it is an institutionalized data management capability.
In its authoritative work DAMA-DMBOK[1], DAMA International positions data governance as the "core" of data management -- surrounding 10 knowledge areas including data architecture, data quality, master data management, metadata management, data security, and data integration. In other words, data governance is not just "one piece" of data management; it is the governance layer that oversees all data management activities.
In the AI era, the importance of data governance has been dramatically amplified. Traditional BI reports have a relatively high tolerance for data quality issues -- a monthly sales report with 2% missing data typically does not affect decision-making. But machine learning models are far more sensitive to data quality than humans: biases in training data get amplified by models, improperly handled missing values cause feature engineering failures, and inconsistent data definitions prevent cross-departmental feature interoperability. Polyzotis et al.'s research published in ACM SIGMOD[7] clearly states that the greatest challenge facing production ML systems lies not in algorithms, but in data lifecycle management.
McKinsey's research[6] corroborates this view from a business value perspective: enterprises that truly extract value from data have, without exception, established mature data governance mechanisms. Data governance is not a cost center -- it is a foundational infrastructure investment for AI transformation.
2. Data Governance Frameworks: DAMA-DMBOK and DCAM
Building a data governance system requires methodological guidance. The two most widely adopted frameworks in the industry are DAMA-DMBOK and DCAM, which define "what to do" and "how well to do it" from different perspectives.
2.1 DAMA-DMBOK: The Data Management Body of Knowledge
DAMA-DMBOK (Data Management Body of Knowledge)[1], published by DAMA International, is the "textbook" of the data management field. The second edition defines 11 knowledge areas:
- Data Governance -- The core supervisory function
- Data Architecture -- Overall data blueprint design
- Data Modeling & Design -- Logical and physical models
- Data StoRAG (Retrieval-Augmented Generation)e & Operations -- Database management
- Data Security -- Access control and encryption
- Data Integration & Interoperability -- ETL/ELT pipelines
- Document & Content Management -- Unstructured data
- Reference & Master Data -- MDM
- Data Warehousing & BI -- Analytics infrastructure
- Metadata Management -- Data about data
- Data Quality Management -- The six dimensions of quality
2.2 DCAM: Data Management Capability Assessment Model
The DCAM (Data Management Capability Assessment Model)[2], published by EDM Council, approaches from the angle of "maturity assessment," helping enterprises answer a critical question: How mature is our data governance?
DCAM divides data management capabilities into six dimensions, each with multiple sub-items scored on a 1-5 scale:
| DCAM Dimension | Assessment Focus | Maturity Level 1 | Maturity Level 5 |
|---|---|---|---|
| Strategy & Business Case | Whether data governance has executive support and budget | No formal strategy | Data strategy deeply integrated with enterprise strategy |
| Organization & Governance Structure | Whether roles such as CDO and Data Steward exist | No dedicated roles | Cross-departmental governance committee operating at maturity |
| Technology Architecture | Whether the data platform supports governance needs | Scattered Excel files | Automated data platform and quality engine |
| Data Quality | Mechanisms for quantifying and improving data quality | No quantified metrics | Real-time quality dashboards with automated remediation |
| Data Control Environment | Whether policies, standards, and processes are complete | Verbal agreements | Automated policy enforcement and compliance auditing |
| Data Management Lifecycle | Full lifecycle management from creation to destruction | No lifecycle awareness | Automated archiving and compliant destruction |
DAMA-DMBOK tells you "what to do," and DCAM tells you "how well you're doing it" -- using both together is the best practice for planning a data governance roadmap.
3. Data Platform Architecture: Data Lake, Data Warehouse, and Feature Platform
The data platform (sometimes called "data middle platform") is an architectural concept widely discussed in Asian enterprises in recent years. Its core idea is to aggregate data scattered across various business systems through a unified technology platform for governance, processing, and service delivery, upgrading data from a "departmental asset" to an "enterprise asset."
The data engineering architecture proposed by Reis and Housley in Fundamentals of Data Engineering[5] is highly aligned with this concept. We can decompose the data platform into three core layers:
3.1 Data Lake -- The Raw Data Aggregation Layer
The data lake is the "entry point" of the data platform, responsible for storing raw data from various business systems in a low-cost, highly scalable manner. Its defining characteristic is Schema-on-Read: data is written in its original format (JSON, CSV, Parquet, images, logs) and structure is defined only at read time.
Key technology choices:
- Storage layer: AWS S3 / Azure Data Lake Storage / GCS
- Table format: Apache Iceberg, Delta Lake, Apache Hudi (supporting ACID transactions and time travel)
- Data ingestion: Apache Kafka (streaming), Airbyte / Fivetran (batch ELT)
3.2 Data Warehouse -- The Structured Analytics Layer
The data warehouse is the "processing plant" of the data platform, producing structured datasets ready for analytics and reporting after raw data undergoes cleaning, transformation, and modeling. Modern data warehouses have evolved from traditional Kimball / Inmon architectures to cloud-native solutions.
Key technology choices:
- Cloud-native warehouses: Snowflake, Google BigQuery, AWS Redshift Serverless
- Transformation tools: dbt (Data Build Tool) -- a SQL-first data transformation framework
- Modeling methodologies: Dimensional Modeling, OBT (One Big Table), Data Vault 2.0
3.3 Feature Platform -- The AI Service Layer
The feature platform is the critical bridge connecting the data platform to AI/ML. The core problem it solves is: how to enable data scientists to efficiently access governed, consistent, and reusable feature data.
Key technology choices:
- Feature Store: Feast (open source), Tecton, SageMaker Feature Store
- Feature computation: Apache Spark / Flink (batch + streaming feature computation)
- Feature serving: Low-latency online Feature Serving (backed by Redis / DynamoDB)
| Architecture Layer | Core Function | Representative Tools | Data Format |
|---|---|---|---|
| Data Lake | Raw data aggregation and long-term storage | S3 + Iceberg + Kafka | Raw / Semi-structured |
| Data Warehouse | Structured modeling and analytics | Snowflake + dbt | Structured / Star Schema |
| Feature Platform | ML feature management and serving | Feast + Redis | Feature Vectors |
4. The Six Dimensions of Data Quality
Data quality is the core deliverable of data governance. Both DAMA-DMBOK[1] and Gartner's research[3] indicate that data quality can be systematically quantified and managed across six dimensions:
| Dimension | Definition | Quantitative Metric | Common Issue Example |
|---|---|---|---|
| Completeness | Whether required data fields are present and not missing | Non-null ratio >= 99.5% | 15% of customer address fields are empty |
| Consistency | Whether the same data is consistent across different systems | Cross-system comparison consistency rate | Same customer has different name formats in ERP and CRM |
| Timeliness | Whether data is updated within the timeframe required by the business | Data latency <= SLA definition | Inventory data updates daily, but the business needs real-time inventory |
| Accuracy | Whether data correctly reflects the real world | Match rate against authoritative sources | Product price becomes negative due to ETL error |
| Uniqueness | Whether data records are free from inappropriate duplicates | Duplication rate <= 0.1% | Same customer created as two master records due to spelling variations |
| Validity | Whether data conforms to predefined formats and rules | Rate of passing validation rules | Alphabetic characters appearing in a phone number field |
Practical recommendation: The first step in data quality management is not deploying tools, but defining "quality rules." Every critical data field should have a clearly defined quality SLA (Service Level Agreement), and an automated quality monitoring dashboard should be established. Common data quality tools include Great Expectations (open source), Soda Core, Monte Carlo, and Atlan.
5. Master Data Management (MDM)
Master data is the most critical, most shared core entity data in an enterprise -- customers, products, suppliers, employees, organizational structures, and geographic regions. The goal of MDM is to establish a "Single Source of Truth" for these core entities, ensuring data consistency across systems and departments.
5.1 Four MDM Implementation Styles
DAMA-DMBOK[1] defines four MDM implementation styles, and enterprises should choose based on their IT architecture and business requirements:
- Consolidation: Each system retains its own master data, while the MDM system periodically aggregates, matches, and cleanses it to produce a "Golden Record" for analytics purposes. This is the least intrusive starting approach.
- Registry: The MDM system does not copy data but instead builds a cross-system master data index. When querying a customer, MDM tells you which systems contain this record and which version is most authoritative.
- Centralized: The MDM system becomes the sole center for creating and maintaining master data, with all downstream systems sourcing master data from MDM. This provides the highest consistency but is also the most difficult to implement.
- Coexistence: A combination of consolidation and centralized approaches -- certain scenarios are centrally managed by MDM, while others allow systems to maintain their own data with periodic synchronization. This is the most common choice for large enterprises.
5.2 Core MDM Processes
Regardless of the chosen style, MDM involves the following core processes:
- Data Profiling: Inventory master data across all systems to understand its distribution, quality, and degree of duplication
- Matching & Merging: Use fuzzy matching algorithms (such as Jaro-Winkler distance, probabilistic matching) to identify different records of the same entity and merge them into a Golden Record
- Survivorship Rules: When the same field has different values across systems, define which system's data prevails (for example: customer name defers to CRM, credit limit defers to ERP)
- Ongoing Stewardship: Assign Data Stewards responsible for day-to-day master data maintenance, exception handling, and quality monitoring
6. Metadata Management
Metadata is "data about data" -- it tells you: what this data is, where it comes from, when it was created, who is responsible for it, how it's calculated, and where it can be used. Within a data governance framework, metadata management is the critical bridge connecting the "technical layer" to the "business layer."
6.1 Three Types of Metadata
- Technical Metadata: Table structures, column types, indexes, partitioning strategies, ETL schedules -- targeting engineering teams
- Business Metadata: Business definitions, calculation logic, data owners, usage contexts -- targeting business users
- Operational Metadata: Data update frequency, last update time, record counts, quality scores -- targeting operations teams
6.2 Why the AI Era Especially Needs Metadata Management
When enterprise data scientists need to find suitable training data for a new ML project, without proper metadata management they face a series of questions: Does the "revenue" column in this table include tax or not? Which source was this feature computed from? When was this data last updated? Can I use this PII-containing data for model training?
The goal of metadata management is to ensure all these questions have clear answers -- and that these answers are automatically maintained, not dependent on the memory of a senior engineer.
7. Data Catalog and Data Lineage
Data Catalog and Data Lineage are the two core deliverables of metadata management and the most important capabilities of modern data governance platforms.
7.1 Data Catalog
A data catalog is the "search engine" for enterprise data assets -- it enables anyone to quickly find the data they need, understand its definition, quality status, and access permissions. A mature data catalog should have the following capabilities:
- Full-text search and tag classification: Enter "customer lifetime value" to find all related tables, columns, and reports
- Automated data inventory: Use crawlers to automatically scan databases and build and maintain a data registry
- Business Glossary: Unify definitions of business metrics such as "revenue," "active users," and "churn rate" to prevent each department from interpreting them differently
- Data quality metric integration: Display the quality score for each table and column directly within the catalog
- Access request workflow: After discovering needed data in the catalog, users can directly initiate an access permission request
Representative tools: DataHub (open-sourced by LinkedIn), Apache Atlas, Atlan, Alation, Collibra.
7.2 Data Lineage
Data lineage tracks the complete path of data from source to final use -- which system this data came from, which ETL transformations it went through, which reports reference it, and which ML model uses it. The value of data lineage is most prominent in three scenarios:
- Impact Analysis: When upstream table structures change, automatically identify all affected downstream reports and models
- Root Cause Analysis: When a report shows anomalous numbers, trace back along the lineage to quickly pinpoint which ETL step caused the problem
- Regulatory Traceability: When regulators require enterprises to prove the source and calculation process behind a decision metric, data lineage provides a complete audit trail
8. GDPR and Taiwan's Personal Data Protection Act: Requirements for Data Governance
Data governance is not just a technical issue -- it is also a compliance issue. As global data privacy regulations become increasingly stringent, enterprise data governance systems must be capable of meeting regulatory requirements.
8.1 Core Requirements of GDPR
The EU's GDPR imposes several specific technical and procedural requirements on data governance:
- Records of Processing Activities: Enterprises must maintain complete records of all personal data processing activities -- data catalogs and data lineage are the technical foundation for meeting this requirement
- Data Protection Impact Assessment (DPIA): High-risk data processing activities must undergo impact assessments; training and inference of AI/ML models typically fall into this category
- Right to Erasure: Data subjects have the right to request deletion of their personal data -- this requires enterprises to know which systems contain a given individual's data (a direct use case for MDM and data lineage)
- Data Portability: Data subjects have the right to obtain their personal data in a structured format
8.2 Taiwan's Personal Data Protection Act
Taiwan's Personal Data Protection Act[8], while not as strict as GDPR, similarly imposes clear requirements on enterprise data governance:
- Lawful basis for collection, processing, and use: Enterprises must have a clear legal basis or data subject consent
- Notification obligation: When collecting personal data, data subjects must be informed of the purpose, categories, duration, and methods of use
- Security measures: Enterprises must adopt appropriate technical and organizational measures to protect personal data
- Data subject rights: Right to inquiry, right to access, right to request copies, right to correction, right to request cessation of collection/processing/use, and right to request deletion
For enterprises, compliance requirements are a powerful driver for data governance. Without a robust data catalog, you cannot answer "where does this person's data reside?"; without data lineage, you cannot prove "how was this decision calculated?"; without MDM, you cannot ensure that a "deletion request" covers the corresponding records across all systems.
9. Data Governance Challenges for AI/ML
As enterprises begin to deploy AI/ML at scale, data governance faces a series of new challenges not fully addressed by traditional frameworks. Polyzotis et al.'s research[7], drawing from Google's internal practices, systematically identifies the data lifecycle challenges of production ML systems.
9.1 Training Data Bias
The output quality of ML models is directly constrained by the quality and representativeness of training data. Sources of training data bias include:
- Selection Bias: Training data does not represent the true distribution -- for example, a credit scoring model trained only on data from approved borrowers, ignoring rejected applicants
- Labeling Bias: Human-annotated labels reflect the subjective biases or cultural backgrounds of the annotators
- Historical Bias: Historical data itself contains structural societal inequities -- models trained on this data perpetuate and reinforce these biases
Data governance's response to training data bias is to establish metadata records for training data (data cards / datasheets), requiring every training dataset to have clear source documentation, known bias declarations, recommended usage scope, and limitation statements.
9.2 Feature Management
As the number of enterprise ML models grows, feature management becomes a critical challenge:
- Redundant feature computation: Different teams independently compute the same features, leading to logical inconsistencies and wasted compute resources
- Training-Serving Skew: Features computed in Python during training are re-implemented in Java for inference; logic discrepancies cause model performance degradation
- Lack of feature definition governance: When a feature's computation logic is modified, all models depending on that feature need re-evaluation -- but without feature lineage tracking, it is impossible to know which models are affected
Feature Store is the key technical component for addressing these challenges. It provides centralized feature definitions, version management, lineage tracking, and consistency in serving.
9.3 Model Provenance
Model provenance answers a seemingly simple but practically complex question: What data, what code, and what parameters was this model trained with?
This is not only a technical issue but also a compliance issue. When regulators require enterprises to explain the basis of an AI decision, the enterprise must be able to provide a complete provenance chain from data to model. This requires deep integration between data governance (data lineage + metadata) and MLOps (experiment tracking + model registry).
| AI Data Governance Challenge | Traditional Governance Approach | AI Era Additional Requirements | Recommended Tools / Practices |
|---|---|---|---|
| Training Data Quality | Six dimensions of data quality | Bias detection, representativeness assessment | Data Cards + Fairness Toolkit |
| Feature Management | Data dictionary | Feature Store, feature lineage | Feast + dbt |
| Model Provenance | Data lineage | Full-chain traceability: model to features to data | MLflow + DataHub |
| Privacy Compliance | Access control | Differential privacy, federated learning | PySyft + TensorFlow Privacy |
| Data Versioning | Database backups | Training data version management | DVC + LakeFS |
10. Data Mesh: From Centralized to Federated Governance
The Data Mesh concept proposed by Zhamak Dehghani in her book[4] poses a fundamental challenge to the traditional centralized data governance model.
Traditional data platforms adopt a centralized architecture: a central data team is responsible for all data aggregation, governance, and service delivery. This model works well in the early stages of an enterprise, but as scale increases, the central team becomes a bottleneck -- all requests must queue up, and all data modeling depends on the domain knowledge of a few individuals.
Data Mesh proposes four core principles:
- Domain-Oriented Ownership: Data is owned and governed by the business teams that understand it best, rather than being centralized in a single team
- Data as a Product: Each domain team treats its data as a "product" with clear SLAs, documentation, and quality guarantees
- Self-Serve Data Platform: The central team provides platform capabilities (rather than data capabilities), enabling domain teams to build data products in a self-service manner
- Federated Computational Governance: Governance standards are defined globally, but execution is the responsibility of each domain team, with governance rules embedded into the platform through automation
Data Mesh does not seek to replace data governance but rather to change the "execution model" of governance -- from manual review by a central team to automated policy enforcement embedded in the platform. This raises higher expectations for the level of automation in data governance.
11. Implementation Roadmap: From Data Inventory to Governance Maturity
Data governance is an endeavor that is "never truly finished," making a smart starting strategy critical. Below is our recommended four-phase roadmap:
Phase 1: Data Inventory and Current State Assessment (Months 1-3)
- Inventory all core business systems and their data assets
- Conduct a maturity self-assessment using the DCAM[2] framework
- Identify the top 10 most critical data entities (customers, products, orders, etc.)
- Establish a data quality baseline -- quantify the current state as a benchmark for future improvement
- Determine the governance organizational structure: Is a CDO needed? Who will serve as Data Stewards?
Phase 2: Building Core Governance Capabilities (Months 4-9)
- Establish a business glossary and unify definitions for the top 50 key business metrics
- Deploy a data catalog tool (DataHub is recommended as an open-source starting point)
- Build quality rules and automated monitoring for the top 10 critical data entities
- Begin consolidation-style MDM implementation -- start with customer master data
- Define a data classification and grading policy, identify sensitive data, and implement access controls
Phase 3: Expanding AI-Ready Capabilities (Months 10-15)
- Establish data lineage tracking, covering at least the core analytics pipelines
- Deploy a Feature Store to address redundant feature computation and training-serving skew
- Build a training data governance process -- Data Cards, bias detection, version management
- Integrate MLOps with data governance -- complete traceability from data to features to models
- Expand quality monitoring to all critical data pipelines
Phase 4: Continuous Optimization and Culture Building (Month 16 onward)
- Conduct periodic DCAM maturity re-assessments to track the evolution of governance capabilities
- Explore the feasibility of Data Mesh -- whether to transition from centralized to federated governance
- Build a data governance community -- promote data culture through training, knowledge-sharing sessions, and internal certifications
- Continuously respond to emerging regulatory and technological challenges (e.g., data governance requirements for generative AI)
12. Conclusion: Data Governance Is the Invisible Infrastructure of AI Transformation
Returning to the core proposition at the beginning of this article: Why does the AI era demand data governance even more?
The answer is clear: because the essence of AI is learning from data, and the quality of that learning can never exceed the quality of the data. An enterprise that adopts AI without data governance is like building a skyscraper on land without a foundation -- progress appears rapid on the surface, but a structural collapse is inevitable.
Data governance is not a "one-time project" but a continuously operating "organizational capability." It requires commitment from leadership (establishing and empowering a CDO), execution from middle management (building a Data Steward network), and participation from the frontline (data literacy training programs). Technology tools -- data catalogs, quality engines, Feature Stores -- are important enablers, but they cannot replace the transformation of organizational culture.
For enterprises planning AI transformation, our recommendation is: do not wait until AI projects fail to retroactively address data governance. Start now with data inventory, establish quality baselines, and deploy a data catalog. These investments may not appear to produce "AI outcomes" in the short term, but they are the invisible infrastructure that enables all AI outcomes to operate sustainably, reliably, and in compliance.
As DAMA-DMBOK[1] emphasizes: data is an organization's strategic asset, and assets must be managed. Data governance is the discipline and institutional framework for managing that asset.
Need professional consulting on data governance and data platforms?
Meta Intelligence has hands-on experience in data governance framework implementation, data platform architecture design, and AI readiness assessment. From data inventory to governance roadmap, we help enterprises build sustainably evolving data governance systems.
