Connecting your database

Pretensor speaks to PostgreSQL, MySQL, Snowflake, and BigQuery — each driver ships as an optional extra. Every connection ends up as a ConnectionConfig regardless of how you supply it — there are two front doors: pass a DSN on the command line, or define named sources in .pretensor/config.yaml.

Install the connector you need

pip install 'pretensor[postgres]'           # PostgreSQL
pip install 'pretensor[mysql]'              # MySQL
pip install 'pretensor[snowflake]'          # Snowflake
pip install 'pretensor[bigquery]'           # BigQuery
pip install 'pretensor[all-connectors]'     # all four drivers in one

The base package installs no driver — a bare pip install pretensor can serve an already-indexed graph but cannot connect to a database.

Combine extras with comma separation: pip install 'pretensor[snowflake,clustering,embeddings]'.

Optional extras

Beyond connectors, Pretensor exposes three extras that turn on heavier intelligence features. Install only what you need — each one pulls in real binary deps.

Extra	Adds	What it does	When to install
pretensor[clustering]	leidenalg>=0.10	Leiden community detection during the intelligence pipeline. Without it, Pretensor logs a warning and falls back to igraph Louvain (no resolution tuning).	Anything beyond a toy database benefits — Leiden produces tighter, more stable clusters and is required for the clustering_resolution_override config knob to do anything.
pretensor[embeddings]	onnxruntime>=1.17, huggingface-hub>=0.20, transformers>=4.38, numpy>=1.26	Loads a local 384-dim ONNX text-embedding model (Snowflake/snowflake-arctic-embed-xs from the Hugging Face cache) to compute table embeddings at index time. Powers the semantic_search MCP tool, hybrid BM25+cosine reranking in query, and context’s similar-table lookup. No network calls at runtime once the model is cached.	You want meaning-based schema search — finding tables by what they contain rather than what they’re named.
pretensor[all-connectors]	psycopg2-binary, PyMySQL, snowflake-sqlalchemy, google-cloud-bigquery	All four database drivers in one extra.	You touch more than one engine.

Putting it together for a typical analytics-team install:

pip install 'pretensor[snowflake,clustering,embeddings]'

Clustering: what you get and what you lose

The clustering pass is what populates the Cluster nodes the context tool returns and that drive the pretensor://db/{name}/clusters resource. With leidenalg installed:

• Algorithm: Leiden with RBConfigurationVertexPartition, weighted by inferred-join confidence, deterministic seed.
• Tunable: set graph.clustering_resolution_override in your config to override the auto-resolution heuristic (which scales with table count). Higher values produce smaller, tighter clusters; lower values produce broader ones.
• Cluster size merge: graph.min_cluster_size_merge (default 3) absorbs single-table clusters into their nearest neighbor by edge weight.

Without leidenalg, the same step runs igraph’s multilevel (Louvain). It works, the graph still builds, and you’ll see a one-line warning the first time it runs:

leidenalg not installed — falling back to igraph Louvain (no resolution tuning).
Install the clustering extra for better community detection: pip install 'pretensor[clustering]'

Embeddings: what triggers the download

The embeddings extra is loaded lazily — Pretensor only imports onnxruntime and transformers when table embeddings are computed during index/reindex (on by default when the extra is installed; opt out with --no-embeddings or the PRETENSOR_EMBEDDINGS_DISABLED env var). The first time it runs, the ONNX model is fetched from Hugging Face into the standard HF cache (~/.cache/huggingface/). Subsequent runs are fully offline.

If you don’t install the extra, query still works as pure BM25 search and semantic_search degrades to a BM25 fallback — you lose the meaning-based ranking, not the tools themselves.

Pick a state location: per-project or global

Pretensor doesn’t care where its state lives — registry, encrypted DSNs, Kuzu graph files, search index — as long as --state-dir (or state_dir: in config) points at a writable directory. There are two common patterns:

• Per-project (default). State lives under ./.pretensor/ in whichever project you invoke pretensor from. Each project has its own registry, its own indexed graphs, its own visibility rules. Good when project-specific connection sets are the norm.
• Global, in ~/.pretensor/. One registry, one set of indexed graphs, available from any working directory. Good when the same handful of warehouses are queried from many repos and you don’t want to reindex per project.

For the global pattern, drop a config file at ~/.pretensor/config.yaml and tell it to keep state alongside itself:

~/.pretensor/config.yaml

state_dir: ~/.pretensor

sources:
  prod_pg:
    dialect: postgres
    host: db.internal.example.com
    database: analytics
    user: pretensor_ro
    password: ${PG_PASSWORD}

  warehouse_sf:
    dialect: snowflake
    account: xy12345.us-east-1.aws
    database: RAW_DB
    schema: STAGING
    warehouse: TRANSFORM
    role: ANALYST
    user: dbt_user
    password: ${SNOWFLAKE_PAT}

Then point any subcommand at it with the top-level --config flag:

pretensor --config ~/.pretensor/config.yaml index --all
pretensor --config ~/.pretensor/config.yaml serve --config-only

If you’ll always use the global config, alias the binary in your shell profile so the flag becomes implicit:

# ~/.zshrc or ~/.bashrc
alias pt='pretensor --config ~/.pretensor/config.yaml'

pt index --all
pt list
pt serve

Pretensor does not auto-discover ~/.pretensor/config.yaml — without --config, it only looks at ./.pretensor/config.yaml relative to the current directory. The alias is what makes “global” feel transparent.

The sidecar secrets file follows the same path: ~/.pretensor/sources.secrets.yaml is loaded automatically as long as it sits next to the config.yaml you point at.

Encrypted DSNs registered via pretensor add go into <state_dir>/registry.json and <state_dir>/keystore, so they too become global once state_dir points at ~/.pretensor.

One-shot: pass a DSN

The fastest path. Pretensor parses the DSN, encrypts it, and stores it under .pretensor/keystore so you don’t have to retype the password. pretensor init is the guided way to arrive at the same result: it walks you through building a DSN from parts for any of the four dialects and indexes it for you. See pretensor init and the Getting Started guide.

# index from a DSN
pretensor index 'postgresql://USER:PASS@HOST:5432/DBNAME'

# or register the connection first, then index it by name
pretensor add 'postgresql://USER:PASS@HOST:5432/DBNAME' --name prod
pretensor index --source prod

pretensor add writes to .pretensor/registry.json (or wherever --state-dir points) and encrypts the DSN with a key from .pretensor/keystore. pretensor list shows what’s registered, pretensor remove <name> drops it.

Config file: declarative sources:

For multiple databases, version-controlled connection definitions, or CI pipelines, drop a .pretensor/config.yaml in your repo:

state_dir: .pretensor

sources:
  prod_pg:
    dialect: postgres
    host: db.internal.example.com
    port: 5432
    database: analytics
    user: pretensor_ro
    password: ${PG_PASSWORD}        # resolved from env at connect time

  warehouse_sf:
    dialect: snowflake
    account: xy12345.us-east-1.aws
    database: RAW_DB
    schema: STAGING
    warehouse: TRANSFORM
    role: ANALYST
    user: dbt_user
    password: ${SNOWFLAKE_PAT}      # see "Snowflake authentication" below

  bq_warehouse:
    dialect: bigquery
    project: my-gcp-project
    dataset: analytics
    location: us

Then index a single source or all of them:

pretensor index --source prod_pg            # one
pretensor index --all                       # iterate every source in config

--all skips any source whose ${VAR} references aren’t set in the environment and prints a summary at the end. This is the recommended shape for CI: list missing-secret skips instead of crashing on the first one.

A source can also hold a whole-DSN reference instead of individual fields:

sources:
  prod_pg:
    url: ${DATABASE_URL}

pretensor init writes an entry in this shape automatically when the DSN it indexed came from a single environment variable, so the resulting config.yaml stays safe to commit: the resolved secret never lands in the file.

${VAR} substitution

Any string field in sources: may reference environment variables with ${VAR_NAME}. Substitution happens when the connection is built, not when YAML is parsed — so the file itself can be checked in, and the secret stays in your shell or CI secret store. If the variable is unset, you get an explicit error before Pretensor opens a socket.

This applies to the whole-DSN url form too: url: ${DATABASE_URL} resolves the full DSN from that variable at connect time, the same as any field-level ${VAR} reference.

The port field accepts either an integer or a ${VAR} string that resolves to an integer.

Sidecar secrets file

If you’d rather keep credentials out of config.yaml entirely (gitignored, separate rotation, separate filesystem permissions), drop a sources.secrets.yaml sibling next to it. Pretensor loads it automatically when present and merges each entry over the matching source in config.yaml. Per-source overrides win.

.pretensor/
├── config.yaml             # checked into git
└── sources.secrets.yaml    # gitignored

The sidecar is keyed by source name (the keys must match config.yaml’s sources: block) and accepts the same fields as a source. Typically you’d put just password (and maybe user) in the sidecar:

# .pretensor/config.yaml — safe to commit
sources:
  prod_pg:
    dialect: postgres
    host: db.internal.example.com
    database: analytics
    user: pretensor_ro
  warehouse_sf:
    dialect: snowflake
    account: xy12345.us-east-1.aws
    database: RAW_DB
    schema: STAGING
    warehouse: TRANSFORM
    role: ANALYST
    user: dbt_user

# .pretensor/sources.secrets.yaml — gitignored
prod_pg:
  password: "actual-prod-postgres-password"

warehouse_sf:
  password: "snowflake-pat-or-password"

Source names that exist only in the sidecar are ignored — Pretensor won’t materialize a source that has no main-config entry. Source names that exist only in config.yaml are loaded as-is. The sidecar’s value for any field overrides the main config’s value for the same source/field.

The sidecar is plain YAML; it does not auto-encrypt. Treat it like any other secrets file — gitignore it, restrict its filesystem permissions (chmod 600), and inject from your secret store in CI:

# in CI
echo "$PRETENSOR_SECRETS_YAML" > .pretensor/sources.secrets.yaml
chmod 600 .pretensor/sources.secrets.yaml
pretensor index --all

Choose one pattern per repo. ${VAR} substitution and the sidecar both work, but mixing them in the same source quickly gets confusing for the next person reading the config.

Allowed source keys

The YAML loader rejects unknown keys with a clear error, so typos surface immediately. The full set is:

dialect, url, host, port, user, password, database,
account, schema, warehouse, role,                 # Snowflake
private_key_path, private_key_passphrase,         # Snowflake key-pair auth
project, dataset, location                        # BigQuery

url is a whole-DSN reference and is mutually exclusive with the field-based keys (host, port, user, password, database, and the dialect-specific fields): a source uses either url or the individual fields, never both. dialect is the one exception, it’s a valid override alongside url (the URL scheme infers the dialect when dialect is omitted).

Repositories: repositories:

config.yaml can also list code repositories for pretensor analyze --all to iterate, so you don’t have to re-type --connection/--service per repository:

repositories:
  - path: /home/you/code/billing-api
    service: billing-api
    connection: prod_pg
  - path: /home/you/code/reporting-service
    service: reporting
    connection: warehouse_sf

Key	Required	Meaning
path	yes	Repository root to scan.
connection	yes	Connection whose tables the scan resolves against.
service	no	Consumer-service label. Defaults to the directory name in path.

pretensor init writes entries here automatically each time it links a repository (the first one, and any additional ones from the “link another?” loop); a repeated path updates the existing entry’s service/connection instead of duplicating it. See pretensor analyze [path] for the full flag reference.

PostgreSQL

DSN form (standard libpq):

postgresql://USER:PASS@HOST:5432/DBNAME
postgres://USER:PASS@HOST:5432/DBNAME             # alias
postgresql+psycopg2://USER:PASS@HOST/DBNAME       # driver variant

A host is required; port defaults to 5432, database to the path segment, and user/password are URL-decoded (URL-encode special characters with % escapes if your password has them).

For password handling, use env-var substitution rather than embedding secrets in CLI args (which leak into shell history):

export PG_PASSWORD='…'
pretensor index "postgresql://pretensor_ro:${PG_PASSWORD}@db.internal:5432/analytics"

SSL parameters (?sslmode=require, ?sslrootcert=…) pass through unchanged to the driver. The connector reads schema metadata, foreign-key constraints, view definitions, and — for tables under ~10M rows — aggregate column statistics. No row contents are read.

MySQL

DSN form:

mysql://USER:PASS@HOST:3306/DBNAME
mysql+pymysql://USER:PASS@HOST/DBNAME             # driver variant

A host is required; the port defaults to the driver’s standard 3306 when omitted, and the path segment names the database (which also scopes introspection to that schema). Credentials are URL-decoded, so %-escape special characters. Requires the mysql extra (pip install 'pretensor[mysql]'). In a config file, use dialect: mysql with the same host/port/user/password/database keys as Postgres.

Snowflake

DSN form:

snowflake://USER:PASS@ACCOUNT/DATABASE/SCHEMA?warehouse=WH&role=ROLE

The hostname slot holds your account identifier (e.g. xy12345.us-east-1.aws or myorg-myaccount). Database is required as the first path segment; schema is optional as the second segment. warehouse and role are query parameters — most production accounts need both.

Snowflake authentication

The DSN parser accepts user:password in the URL. Snowflake supports several credentials in the password slot, but Pretensor itself only sees a string — it’s whatever your Snowflake account is configured to accept:

• Plain password — works, fine for local development.
• Programmatic Access Token (PAT) — recommended for service accounts. Generate a PAT in Snowsight (Admin → Users & Roles → your user → Programmatic Access Tokens), then pass it where the password would go. The Snowflake driver treats it identically to a password.
• Key-pair — pass the PEM key location as DSN query parameters (?private_key_path=/path/to/rsa_key.p8&private_key_passphrase=…, passphrase optional for unencrypted keys) or as private_key_path / private_key_passphrase source keys in the config file. When a key is supplied, the password slot can be omitted.

Inject either via env var so the secret never lands in the config file or shell history:

export SNOWFLAKE_PAT='ey…'        # or SNOWFLAKE_PASSWORD
pretensor index "snowflake://dbt_user:${SNOWFLAKE_PAT}@xy12345.us-east-1.aws/RAW_DB/STAGING?warehouse=TRANSFORM&role=ANALYST"

Or in .pretensor/config.yaml:

sources:
  warehouse:
    dialect: snowflake
    account: xy12345.us-east-1.aws
    database: RAW_DB
    schema: STAGING
    warehouse: TRANSFORM
    role: ANALYST
    user: dbt_user
    password: ${SNOWFLAKE_PAT}

Pretensor uses Snowflake-specific catalogue views to pick up grants and last-accessed timestamps, which feed the classification heuristics — a never-queried table is downweighted in search results. If you want role-keyed visibility profiles synced from your warehouse, pretensor sync-grants --dsn <snowflake-dsn> --output .pretensor/visibility.yml writes them out.

OAuth and external-browser auth are not yet wired through the DSN parser. Password, PAT, or key-pair are the supported paths today.

BigQuery

DSN form:

bigquery://PROJECT/DATASET

Authentication uses Application Default Credentials — there is no password in the DSN. Run gcloud auth application-default login for local development, or set GOOGLE_APPLICATION_CREDENTIALS to a service-account JSON for CI:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/sa-key.json
pretensor index 'bigquery://my-gcp-project/analytics'

In a config file:

sources:
  bq:
    dialect: bigquery
    project: my-gcp-project
    dataset: analytics
    location: us            # optional, controls regionality of introspection jobs

Verifying a connection

After indexing, pretensor list shows the registered connection, its dialect, table count, and staleness. Then pretensor serve --config-only exposes the indexed graph over MCP. From an agent, calling the list_databases tool returns the same view the CLI shows.

What Pretensor does not read

In every connector, the introspection layer reads schema metadata, view definitions, foreign keys, and aggregate column statistics — never row contents. See the Privacy Model for the full data-handling boundary.