dbt lineage and schema graphs: the missing MCP integration

dbt (data build tool) is the standard for data transformation. It knows every model, every dependency, every SQL transformation in your pipeline. Its manifest.json is a machine-readable map of how data flows from raw sources to analytics-ready tables.

Pretensor knows your database schema — tables, columns, foreign keys, inferred joins, table classifications, community clusters. It builds a traversable graph on disk and exposes it to AI agents through MCP.

Put them together and the agent doesn’t just see static schema. It sees lineage. It knows how fact_orders was built from stg_orders and dim_customers, which dbt model produced it, and what downstream models depend on it. This is the bridge between “what tables exist?” and “where did this data come from?”

What dbt already knows

dbt’s manifest.json — generated every time you run dbt run or dbt compile — contains:

• Models: every .sql file in your models/ directory, with the raw SQL and compiled SQL.
• Dependencies: which models ref() other models (the DAG).
• Sources: tables that feed into dbt models (raw database tables).
• Tests: schema tests (unique, not_null), data tests, and their results.
• Documentation: .yml descriptions, column-level docs, tags.

This is rich metadata. But it lives in a JSON file that AI agents can’t query. An agent with access to raw manifest.json would need to parse it, find the right entries, understand the DAG structure, and correlate it with the live database schema — all from context-window reasoning. That’s fragile at scale.

What the schema graph adds

Pretensor’s Kuzu graph captures the database-side reality:

• Physical tables: their actual column names, types, and row counts (from information_schema).
• Relationships: both explicit FKs and inferred joins discovered from column-name/type matching.
• Classification: is this a fact table, dimension, bridge table, or reference table?
• Clusters: which tables are related (Louvain community detection).
• Join paths: precomputed shortest paths between any two tables in the graph.

Combined, dbt provides what transformations happened and the schema graph provides what the result looks like on disk. The agent sees both:

stg_orders (dbt model) → fact_orders (physical table, Fact)
  ├── FK → dim_customers (physical table, Dimension)
  │   └── upstream: stg_customers (dbt model) → raw_customers (dbt source)
  └── FK → dates (physical table, Dimension)
      └── upstream: dim_dates (dbt model) → seed_dates (dbt seed)

How Pretensor + dbt works

Pretensor has a --dbt-manifest flag on the index command. Point it at your manifest.json:

pretensor index postgresql://user:pass@host:5432/dbname \
  --dbt-manifest /path/to/dbt/project/target/manifest.json

Pretensor reads the manifest, extracts model-to-table mappings, and enriches the graph:

1. Model-to-table linking: Each dbt model is linked to its corresponding physical table in the schema graph. The agent sees dim_customers as both a node in the lineage graph (dbt model) and a node in the schema graph (PG table with columns, FKs, classification).

2. Lineage edges: The graph gets new edge types: CREATED_BY (table → model that produced it), DEPENDS_ON (model → upstream model), and SOURCED_FROM (model → source table). The agent can traverse lineage and schema in the same query.

3. Documentation enrichment: Column-level documentation from dbt .yml files is attached to graph nodes. The agent’s context call for a table includes both structural metadata (column type, FK) and documentation (“This is the customer dimension, sourced from the CRM, refreshed daily”).

4. Test results (future): dbt test results can flag stale or broken tables, giving the agent a data-quality signal alongside schema context.

What this enables for AI agents

With dbt enrichment, the agent’s questions get better answers:

Why this is the missing integration

MCP is still new. Most MCP database tools ship with a query tool and maybe a list_tables tool. The idea of combining transformation metadata (dbt) with schema metadata (information_schema) in an agent-queryable graph hasn’t been done yet.

This is partly because dbt and MCP live in different ecosystems. dbt is for analytics engineers writing SQL transformations. MCP is for AI tool builders. But the data engineer who uses dbt is increasingly the same person using Claude or Cursor for warehouse exploration. They shouldn’t have to bridge the two worlds manually.

The schema graph + manifest.json integration means:

• One source of truth for the agent. Not “here’s the schema” and “here’s the lineage file” — one graph with both.
• No manual DDL pasting. The agent discovers schema on its own, lineage included.
• Impact analysis that spans transformations. Changing a source table → downstream dbt models affected → physical tables affected → queries/dashboards affected.
• Documentation retrieval. The agent can answer “what does this column mean?” with the dbt docs, not just the column type.

Getting started

pip install pretensor

# Index with dbt manifest
pretensor index postgresql://user:pass@host:5432/dbname \
  --dbt-manifest ./target/manifest.json

# Start MCP server
pretensor serve --config-only

The context tool will now include model lineage and documentation in its response. The impact tool will traverse both FK edges (schema graph) and model-dependency edges (dbt DAG). The cypher tool can query lineage paths directly:

MATCH (m:Model)-[:CREATED_BY]->(t:Table)
WHERE t.name = 'fact_orders'
RETURN m.name, m.sql

Pretensor is open-source on GitHub. dbt manifest support is available in the current alpha. See the quickstart for full setup.