Designing Data Architecture: If Ownership Isn’t Defined, You’re Building on Sand

Undefined data ownership is the most common cause of conflicting reports, fragile processing pipelines, and failed governance initiatives – an architecture that addresses this structurally looks very different from what most organizations operate today.

The Problem Has a Name – and It’s Not “Bad Data”

Data swamps, conflicting reports, fragile processing pipelines. In almost every company we support at CONVOTIS, the root cause is the same: data has no owner. Not because nobody wants responsibility – but because the architecture itself does not structurally account for ownership.

Every organization faces three questions that must be answered before selecting the first tool:

• Who is responsible for quality?
  In practice, the answer “everyone” is identical to “no one.” Domain-based ownership – where the producing team is responsible – sounds trivial, but requires organizational anchoring. A RACI matrix alone is not enough.
• Where is it authoritatively defined what an “active customer” means?
  Usually in a spreadsheet with three competing versions, none of which is considered official. A semantic layer in the repository solves this technically – but only if it is also clear who owns that definition.
• At which processing stage is a dataset considered ready for decision-making?
  Without explicit quality zones, everything ends up unfiltered in analytics tools. The result: reports that contradict each other, and discussions about which one is correct – instead of what the data is actually saying.

An architecture that leaves these three questions unanswered shifts trust toward gut feeling.

When Is the Transformation Worthwhile – and When Isn’t It?

Before diving into architecture, an honest assessment: the managed data product model is not a universal remedy.

Fewer than five teams, fewer than twenty datasets – lightweight conventions are entirely sufficient. The overhead becomes worthwhile when dependent teams block each other, when the cost of a data outage exceeds the effort of governance, or when a GDPR access request currently takes three weeks.

Introduced too early: complexity without value. Introduced too late: rebuilding on an unstable foundation, burdened by accumulated technical and organizational debt.

What a Data Product Is – and Why Traditional Pipelines Fail

The decisive shift is conceptual: away from the idea that data flows, toward the idea that data is delivered.

A data product is a clearly defined, versioned, documented, and monitored unit – comparable to an interface contract between teams. It has a stable access point, a defined service level, and a responsible team. Data Mesh describes this approach as domain-driven data ownership – not a tool, not a stack, but an organizational principle with technical consequences.

A mature data product fulfills six conditions:

• Discoverable – in the data catalog, not through word of mouth or internal email chains
• Documented – machine-readable field definitions, not buried in an abandoned wiki
• Quality commitment – measurable SLOs, not statements like “should probably be correct”
• Stable access point – no silent schema breaks, no migration chaos
• Clear ownership – one team, not a committee or blurred responsibilities
• Automated quality validation – during every processing step, not afterward

Anyone who leaves out even one of these conditions is not building data products. They are building well-intentioned tables.

The most expensive mistakes almost always occur around condition five – not because nobody checks, but because it is unclear who must respond to an alert. In one project we took over at CONVOTIS, monitoring alerts had remained unanswered for months because no team had formal responsibility. The data kept flowing. So did the decisions.

The Three Foundations: Lineage, Semantics, Quality

The data product is the unit. But it stands on three layers. If one is missing, the others collapse.

Lineage Tracking: Knowing Where a Value Comes From

Without lineage, troubleshooting becomes guesswork. With it, questions that otherwise take hours can be answered in minutes: where did the error occur – in the source system, during transformation, or only at delivery?

For getting started, table-level tracking is sufficient. OpenLineage defines the open exchange format, while Marquez provides a lightweight implementation. DataHub is the more comprehensive alternative: a complete catalog with lineage, search, and classification – requiring more operational effort, but also offering broader capabilities.

Column-level lineage becomes mandatory as soon as GDPR compliance must be demonstrable. Anyone implementing it only after the first access request quickly realizes how much work has accumulated – and how long a legally required request suddenly takes.

Semantic Layer: One Version of the Truth, Versioned

Two reports, one question, two answers – a classic symptom of missing semantic authority.

MetricFlow (part of dbt) stores term definitions versioned in the repository: a single, validated definition of active_customers, accessible from every analytics tool. No spreadsheets, no informal agreements, no competing versions in another department.

The familiar bottleneck: when departments compete, the semantic layer itself becomes the conflict point. Sales and Finance define “revenue” differently – and both have legitimate reasons. What helps is clear metric ownership and an escalation path involving business representatives before the definition enters the repository. Tools make these conflicts visible – people must resolve them.

Quality Monitoring in the Pipeline: Validate Early, Respond Explicitly

Quality checks belong in the processing pipeline – not in the report. Anyone discovering errors only in the dashboard has already enabled poor decisions.

A proven pattern divides the pipeline into three zones:

• Ingress zone – completeness, validity, format compliance
• Transformation zone – volume consistency, business rules, referential integrity
• Delivery zone – deviation from the previous day, quality score, release status

The critical factor is the response strategy when checks fail: hard stop for financial data, quarantine for high-volume data, warning labels for exploratory analytics. This decision must be explicit – it must not emerge implicitly because nobody addressed it. Great Expectations and dbt Tests implement this pattern.

Governance: Why Access Rules and Deletability Belong in the Architecture

GDPR access requests, deletion requests, audits – they all ask the same question: where is this person’s data stored? Anyone who first has to search for the answer has treated governance as an afterthought.

Four measures prevent this structurally:

• Access policies as code using Open Policy Agent. Without it: manual approvals, no auditability, no reproducibility.
• Sensitivity labeling at field level in schema metadata. Without it, the platform cannot automatically mask or block data – and therefore doesn’t.
• Retention periods as configuration in the versioned repository – otherwise email chains and manual processes take over, and deadlines are forgotten.
• Deletability as a design principle – embedded from day one.

Retrofitted deletability is one of the most expensive architectural problems we know. Multiple teams, multiple weeks, a single access request – because nobody can answer the question “where is this data actually stored?”

How to Start – Without Rebuilding the Entire System

The most common mistake during implementation: starting too broadly. The right starting point is focused.

Step 1: Identify the critical data path – where reports conflict, where teams blame each other, where an outage would have the greatest impact.

Step 2: For that exact path, implement lineage, integrate automated quality checks, and define response strategies.

Step 3: Clarify ownership explicitly – with names, escalation paths, and actual resources. Responsibility without resources and authority remains a statement of intent.

Only once this path runs stably and the team genuinely lives ownership does scaling become worthwhile.

The Point Where Technology Ends

Anyone treating data quality as a project goal will face a new problem after go-live. The tools for stable operations exist – mature, proven, well documented. What is almost always missing is someone who feels responsible when it matters. With authority. With resources. With an escalation path that works.