QueryForge

Governed AI analytics, from natural language to auditable SQL

Turn business questions into safe, traceable SQLite queries—with a semantic layer, policy enforcement, bounded recovery, and production-friendly delivery interfaces.

简体中文 · Quick start · Architecture · Documentation

QueryForge is a local-first, domain-first AI data analytics platform built around one principle: generated SQL should be governed like application code, not trusted like prose.

Users create or select a data domain first—such as retail, finance, product, or the bundled Anime Streaming sample—then onboard that domain's data, review its semantic contract, and ask questions inside the same governance boundary. QueryForge combines that workflow with natural-language-to-SQL, AST-level security, read-only execution, bounded recovery, and complete run artifacts.

QueryForge currently targets SQLite and controlled environments. It is a portfolio-grade reference architecture, not a multi-tenant analytics service.

The governance layer is deterministic, so it is tested exhaustively offline; the model layer is not, so its numbers are reported separately and with their sample size. Both are reproducible from this repository.

Two honest qualifications on that last row, because they matter more than the number:

• It is one run of 40 cases. Differences of ±0.03 have been observed across identical code, so this figure cannot resolve small changes.
• It covers the anime sample domain only. It is evidence that the pipeline works end to end on a real model, not a general accuracy claim.

Tier-1's 23/23 measures the engineering chain (governance, execution, evidence, budgeting, failure classification), not model capability: a fixture supplies the SQL. See NL2SQL evaluation for the method and the frozen baselines.

_{Data Domain Center — create or select a governed context before adding data or semantics.}

_{Domain overview — the Anime Streaming dataset is shown as one selected sample, not the platform identity.}

_{Semantic Studio  ·  Governed analysis with auditable SQL and Trust Trace}

Most NL2SQL demos stop after a model emits a query. QueryForge covers the full delivery loop:

• Defense in depth — candidates are checked before execution and revalidated at the database boundary.
• Semantic contracts — YAML models describe business metrics, entities, relationships, cardinality, ownership, SLA, sensitivity, and quality rules.
• Adaptive workflow — simple questions stay fast; complex questions can activate a bounded tool loop.
• Read-only by default — normal analysis opens SQLite databases in read-only mode and rejects write or administrative SQL.
• Multiple delivery surfaces — use the same application service through the CLI, REST/SSE, MCP, or a webhook gateway.
• Reproducible evaluation — the repository ships a 32-task deterministic agent benchmark over three independent schemas and a 120-case, three-domain NL2SQL gold set, with the frozen real-model baselines recorded in the docs.

Requirements: Python 3.11 or 3.12 and SQLite.

python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e .
cp .env.example .env

Set one provider in .env. OpenAI-compatible, Claude, Gemini, Qwen, DeepSeek, and GLM configurations are included.

LLM_PROVIDER=openai
OPENAI_API_KEY=your-api-key
OPENAI_MODEL=gpt-4.1-mini

Provider defaults and environment-variable mappings live in models.yml.

queryforge --prepare-sample-data

queryforge \
  --database sample_data/anime_streaming/anime_streaming.sqlite \
  --question "Which anime generated the most watch hours?"

Try a multi-hop semantic query:

queryforge \
  --database sample_data/anime_streaming/anime_streaming.sqlite \
  --semantic-model sample_data/anime_streaming/semantic_model.yml \
  --sql-policy sample_data/anime_streaming/sql_policy.yml \
  --question "Compare watch completion and merchandise GMV by anime genre"

The repository includes a complete visual workspace for creating and switching data domains, onboarding domain-owned data, reviewing the required semantic layer, asking governed questions, inspecting SQL and Trust Trace evidence, and auditing domain-scoped run history.

# Terminal 1: QueryForge API
python -m pip install -e ".[api]"
queryforge --serve-api

# Terminal 2: QueryForge Studio
make web-install
make web-dev

Open http://localhost:3000. Start in Data Domains, select the bundled Anime Streaming example or create a clean domain, then add data inside that domain. If the Python API is offline, the sample-domain analysis remains explorable with a deterministic demo response. See the Studio guide.

The public lifecycle stays intentionally small:

analysis → candidate → execution → completion → delivery

Role-specific agents operate inside those stages. A deterministic router chooses the path; the model does not control the security boundary.

Every generated query passes through an auditable pipeline:

SQL candidate
  → SQLite AST parse
  → single read-only statement
  → table and column scope
  → dangerous-function checks
  → recursive CTE and cross-join checks
  → table, join, and LIMIT budgets
  → governed preview
  → execution-boundary revalidation

A policy can be as small as:

version: 1
name: anime_streaming
allowed_tables:
  - fact_watch_session
  - dim_anime
require_limit: true
max_limit: 500
max_tables: 2
max_joins: 1
allow_cross_join: false

Policy denials return a structured SQL_SECURITY_ERROR before SQLite execution.

QueryForge's YAML semantic models give generated SQL business context that raw schemas cannot provide:

metrics:
  - name: watch_hours
    description: Total valid viewing time in hours.
    entity: watch_session
    aggregation: sum
    expression: SUM(fact_watch_session.watch_seconds) / 3600.0
    default_filters:
      - fact_watch_session.is_valid = 1
    owner: audience-analytics
    sensitivity: internal

Models can declare entities, dimensions, metrics, grain, relationships, join paths, fan-out constraints, operational metadata, and physical quality rules. QueryForge requires a validated model by default; it auto-discovers a model beside the database and rejects schema-only analysis unless the caller explicitly selects the diagnostic escape hatch.

In Studio, semantic construction is domain-first and gated:

Create/select domain
  → upload domain-owned sources
  → profile physical schema
  → confirm entity identity and grain
  → define dimensions, measures, metrics, and time
  → review relationships, cardinality, and Join Paths
  → classify sensitivity, ownership, policy, and quality
  → validate 100% of blocking checks
  → publish data + semantics atomically

Technical names are treated as evidence, not business truth. A new domain starts empty and never inherits the Anime sample's entities or metrics.

Build or incrementally refresh one:

python scripts/build_semantic_model.py \
  --database warehouse.sqlite \
  --output warehouse.semantic.yml \
  --owner data-platform

The generated report separates high-confidence physical evidence from definitions that need business review. See Semantic layer authoring and Semantic contracts.

The repository also includes a Monday-morning semantic drift workflow that checks schema, metrics, relationships, Join Paths, and data-quality contracts against a reviewed baseline.

Anime Streaming is one ready-to-run example data domain, not a product-wide schema. The dataset is purpose-built for QueryForge and fully synthetic: 370,762 rows, 15 tables, 30 declared relationships, 7 governed Join Paths, and 11 business metrics across content, engagement, subscriptions, advertising, community, and merchandise.

flowchart LR
  Studio[Studio] --> Anime[Anime]
  Genre[Genre] --- Bridge[Anime–Genre Bridge] --- Anime
  Anime --> Episode[Episode] --> Watch[Watch Session]
  User[User] --> Watch
  User --> Rating[Rating] --> Anime
  User --> Subscription[Subscription]
  Watch --> Ad[Ad Impression]
  User --> Order[Merch Order] --> Item[Order Item]
  Product[Merch Product] --> Item
  Anime --> Product
  User --> Follow[User Follow] --> User

  classDef dimension fill:#111827,stroke:#7c3aed,color:#f9fafb;
  classDef fact fill:#172554,stroke:#22d3ee,color:#f9fafb;
  class Anime,Studio,Genre,Episode,User,Product,Bridge dimension;
  class Watch,Rating,Subscription,Ad,Order,Item,Follow fact;

See the dataset contract, inspect the semantic model, or regenerate the database and optional CSV exports with python sample/generate_anime_streaming.py.

queryforge \
  --plan-mode \
  --question "Compare monthly watch hours by subscription tier"

queryforge \
  --complexity-mode complex \
  --parallel-candidates 3 \
  --question "Explain completion-rate changes by genre, device, and membership tier"

queryforge --stream --question "List the top ten anime by watch hours"
queryforge --report --question "Build a report for monthly engagement by genre"

python -m pip install -e ".[assets]"

python scripts/scaffold_data_asset.py \
  --source events.csv \
  --output events.assets.yml \
  --owner engagement-analytics

# Review the semantic draft and set semantic_model.reviewed: true.
python scripts/build_data_assets.py \
  --config sample_data/data_assets/assets.yml \
  --publish-database .queryforge/demo/analytics.sqlite

Every uploaded asset must carry entity semantics. Data and semantics publish atomically, so a failed metric, relationship, grain, or quality contract rolls the whole upload back.

python -m pip install -e ".[api]"
queryforge --serve-api --api-host 127.0.0.1 --api-port 8000

curl -X POST http://127.0.0.1:8000/ask \
  -H 'content-type: application/json' \
  -d '{
    "question": "List the ten anime with the highest completion rate",
    "database": "sample_data/anime_streaming/anime_streaming.sqlite"
  }'

python -m pip install -e ".[mcp]"
python -m queryforge.interfaces.mcp.server --transport stdio

queryforge/
├── cli.py             # Installed CLI implementation
├── application/       # Transport-neutral service facade and resources
├── core/              # Configuration, schemas, workspace paths, observability
├── data_assets/       # Ingestion, quality, lineage, and publication
├── domain/            # SQL policy, semantics, contracts, and skills
├── infrastructure/    # Database adapters, model providers, storage, and tools
├── evaluation/        # Benchmark thresholds and evaluator-side contract rules
├── interfaces/        # CLI-adjacent API, MCP, and gateway adapters
├── orchestration/     # Router, role agents, lifecycle, and state
├── workflow/          # NL2SQL nodes, selection, repair, and reporting
└── bundled_skills/    # Prompt-only skill definitions shipped with the package

evaluation/gold/       # Multi-domain NL2SQL evaluation cases
evaluation/tasks/      # Deterministic agent-benchmark tasks (dev/regression/holdout)
sample_data/           # Ready-to-run SQLite datasets and semantic models
web/                   # QueryForge Studio and hosted persistence adapters
scripts/               # Build, benchmark, evaluation, and acceptance tools
tests/                 # Unit, integration, boundary, and acceptance tests
docs/                  # Architecture and feature documentation
.github/               # CI, semantic drift audit, and contribution templates

Dependencies flow inward from interfaces and application code toward domain, infrastructure, and core contracts.

Run the complete offline quality gate:

./init.sh                # environment + 928-test suite + repository state
make check               # repository hygiene + 13 offline acceptance checks

Live model evaluation reports execution success, semantic equivalence, policy precision/recall, latency, measured token usage, and projection tolerance:

python scripts/evaluate_sql.py \
  --cases evaluation/gold/nl2sql_multidomain.jsonl \
  --model-provider openai \
  --output .queryforge/evaluations/openai.json

CI runs the offline acceptance gate (including the deterministic agent benchmark) on Python 3.11 and 3.12, plus an integration job that requires the optional transport dependencies. Real-model evaluation is a manual workflow (.github/workflows/model-eval.yml) because it spends money.

Every claim in this section is reproducible from the repository. The point of the table is the third column: what has not been shown is stated as plainly as what has.

The demos and tier-1 are offline and deterministic: no model call, no network, no API key. The agent benchmark's tier 1 supplies the SQL as a fixture, so its 23/23 measures the engineering chain — not model accuracy. Real-model numbers must come from a tier-3 run with credentials and are reported separately.

Deployment level: controlled environment, single tenant, read-only data access. SQLite is the default backend; DuckDB and PostgreSQL adapters exist behind optional extras (see Database adapters). The system is not hardened for arbitrary untrusted multi-tenant input; the honest blank spots are general-domain model accuracy and the PostgreSQL backend.

• Contributing guide
• Security policy
• Code of conduct
• Changelog

The source tree is ready for GitHub review and CI. The repository owner still needs to select and add a LICENSE before public release; no license has been assumed on their behalf.

QueryForge's guarantees apply to its configured SQLite execution boundary. The project does not currently include:

• production authentication, authorization, tenant isolation, or rate limiting;
• durable distributed workflow recovery or token-level cancellation;
• MySQL, warehouse, lakehouse, or streaming-system adapters (a PostgreSQL connector exists but is neither exported from the package API nor verified against a live server);
• provider-normalized billing or a trained-model lifecycle.

Keep REST and MCP transports inside a controlled environment. Do not commit provider secrets, generated run state, or local databases containing sensitive data.

Network transports can be hardened without code changes:

• QUERYFORGE_API_KEY — when set, REST/Gateway endpoints (except /health) require Authorization: Bearer <key> or X-API-Key: <key>.
• DATABASE_ALLOWLIST / REPORT_ROOT_ALLOWLIST — comma-separated directories that confine caller-supplied database/semantic_model_path/sql_policy_path and report output paths. Without them, network transports fall back to the project root plus the default database directory.

POST /ask/stream delivers progress events followed by one terminal final_result event carrying the serialized answer (or an error); clients that disconnect cancel the run cooperatively at the next node boundary.

• Database adapters beyond SQLite
• First-class authentication and tenant policy boundaries
• Durable workflow execution and cancellation
• Warehouse-catalog integrations
• Provider-independent usage and cost accounting

Contributions and design discussions are welcome after the repository license is selected.