Databases

A project can connect to one or more databases. You list them under databases in pda_config.yaml. pda sync then writes a set of docs for each table, and the agent runs SQL against the database when it answers.

Supported databases

type	Database
athena	Amazon Athena
bigquery	Google BigQuery
clickhouse	ClickHouse
databricks	Databricks
duckdb	DuckDB
ducklake	DuckLake
fabric	Microsoft Fabric
motherduck	MotherDuck
mssql	Microsoft SQL Server
mysql	MySQL
postgres	PostgreSQL
redshift	Amazon Redshift
snowflake	Snowflake
starrocks	StarRocks
trino	Trino

Each type needs its CLI extra, for example ./cli[snowflake]. See Install the CLI.

Examples

DuckDB

databases:
    - name: epigenomics
      type: duckdb
      path: "{{ env('POLARIS_DATA_DIR') }}/epigenomics.duckdb"

Snowflake

databases:
    - name: covid19
      type: snowflake
      account_id: myorg-myaccount
      username: ANALYST
      authenticator: programmatic_access_token
      token: "{{ env('SNOWFLAKE_PAT') }}"
      database: COVID19_EPIDEMIOLOGICAL_DATA
      warehouse: COMPUTE_WH
      include:
          - PUBLIC.*

Snowflake also supports passwords, key pairs (private_key_path or private_key, with passphrase) and browser sign-in (authenticator: externalbrowser).

PostgreSQL

databases:
    - name: warehouse
      type: postgres
      host: db.internal.example.com
      port: 5432
      database: analytics
      user: polaris_reader
      password: "{{ env('PG_PASSWORD') }}"
      schema_name: public

Amazon Athena

databases:
    - name: lake
      type: athena
      region_name: us-west-2
      profile_name: my-profile
      schema_name: analytics
      work_group: primary

Give Polaris a read-only database user. Polaris runs queries read-only by default, but a read-only user is the safest setup.

Options for every database

Option	Default	What it does
name		A name for the connection, used in pda sync -p databases:<name> and in tests.
include	everything	Glob patterns of schema.table to document, such as PUBLIC.*.
exclude	nothing	Glob patterns of schema.table to skip.
exclude_columns	nothing	Glob patterns of schema.table.column to leave out of the docs.
allow_listed_only	false	When true, the agent may only query tables that were synced.
templates	columns, preview	Which docs to write for each table. See below.
preview_max_value_length		Cuts long values in previews. 0 turns cutting off.

Keep the scope small. Polaris answers more reliably with 10 to 20 well-documented tables than with 100 thinly documented ones.

Table docs

pda sync writes one folder per table:

databases/type=duckdb/database=epigenomics/schema=main/table=tcga_patients/
├── columns.md        # column names, types and comments
├── preview.md        # sample rows
├── profiling.md      # distinct counts, ranges and top values
└── annotations.md    # your own notes, never overwritten

Choose which files to write with templates:

Template	Writes	Notes
columns	columns.md	Column names, types and descriptions.
preview	preview.md	A few sample rows.
profiling	profiling.md	Counts, ranges and the most common values of each column. Helps the agent filter on real values.
ai_summary	ai_summary.md	A written summary of the table by a model. Needs llm.annotation_model.
query_history	query_history.md	Common queries from the database’s query history.

Profiling and AI summaries can be slow on large tables. Control how often they refresh:

profiling:
    refresh_policy: interval # always, interval or once
    interval_days: 7

Your own notes

Each table folder gets an annotations.md file the first time it is synced. Write your own notes there, such as what a column really means or which rows to leave out. pda sync never overwrites it.

Check the connection

Run pda debug to test every database and model in the config. See the CLI reference.