AI Data Analyst

Ask your data questions in plain English — free, private, and offline.

A self-hosted data analyst: upload a CSV, Excel or PDF file and ask business questions like "Which products lost the most revenue?", "Show me monthly sales." or "Which customers are most valuable?" — and get the SQL, a table, a chart and a short explanation. Natural language to SQL, without any LLM or API key, running on your own machine with one command.

How it's different from ChatGPT-based data tools: your data never leaves your machine and nothing is generated by a language model. A rule-based engine (app/nlp.py) understands the question and writes the SQL; a strict validator and a read-only database connection make execution safe. Deterministic, free, and auditable. See Trade-offs for what that means.

Quickstart

docker compose up --build

Open http://localhost:8000 → log in with the shared demo account 1234 / 1234 → upload tests/sample_sales.csv (120 rows, 5 categories, 2023–2024, includes negative/return rows) → ask questions.

Stop with Ctrl+C; wipe data with docker compose down -v.

Pipeline

question ──▶ nlp.build()            rule-based intent + column matching
         ──▶ validate()             sqlglot AST whitelist (app/validate.py)
         ──▶ run_sql()              read-only connection, statement_timeout, row cap
         ──▶ explain()              template narrative from the real numbers
         ──▶ chart.js               line for time series, bar for rankings

Example — "Which products lost the most revenue?" on sample_sales.csv:

SELECT "product", SUM("revenue") AS "revenue"
FROM "ds1_sales"
GROUP BY 1 ORDER BY 2 ASC LIMIT 10

Understood question shapes: top/bottom-N by metric, metric by dimension, monthly / weekly / daily / yearly trends, totals / averages / max / min, row counts, year filters ("revenue in 2023"), preview. Anything else gets example questions generated from the dataset's own columns.

Security model

Defense in depth — three independent layers (see threat model):

1. AST validation (app/validate.py, sqlglot): single statement, SELECT-only (incl. CTEs/unions, but data-modifying CTEs rejected), table allowlist (no pg_catalog/information_schema/other users' tables), no schema-qualified reads, SELECT INTO/FOR UPDATE/DML/DDL blocked anywhere in the tree, LIMIT clamped to 1000.
2. Read-only connection (app/db.py): second engine with default_transaction_read_only = on + statement_timeout = 5s — even a validator miss (e.g. pg_sleep) cannot mutate or hang.
3. Row cap at fetch time regardless of the query's LIMIT.

Plus: JWT auth (PyJWT + bcrypt), owner-scoped datasets, every attempt logged to query_log (user, SQL, status, rows, duration, error), 100 MB upload cap.

Ingestion

pandas read_csv / read_excel / pdfplumber (PDF tables) → sanitized column names → dtype-mapped CREATE TABLE → COPY FROM STDIN (fast bulk load). Naive date inference for text columns via 100-row sample.

Tests

python tests/test_validate.py      # security self-check (DROP, DML-in-CTE, stacked
                                   # statements, catalog reads, LIMIT clamp, ...)

Trade-offs

• Rule-based NL is rigid: it nails the shapes above and refuses anything else with suggestions built from the dataset's own columns. The upgrade path is an LLM behind the same nlp.build() interface — validator and read-only execution stay regardless.
• PDF upload takes the largest table in the file.
• One shared demo account for now (1234/1234); per-user isolation already enforced by owner checks on datasets.

Stack

Python 3.12, FastAPI, SQLAlchemy 2 + psycopg3, PostgreSQL 16, pandas, sqlglot, PyJWT + bcrypt, chart.js, Docker Compose. See CHANGELOG for the build log and ROADMAP for what's next.