Data Sources

A data source is an external database table or file that a concept can read from via a bind annotation. The Data Sources API lets you register connections, browse uploaded files, infer Vadalog schemas from tables, and preview raw rows before building logic. All paths are relative to the base URL and require authentication. All responses use the standard envelope unless noted otherwise. The scope parameter follows the scope conventions; paginated endpoints follow pagination.

Connections

Connect a data source

Register an external database or file-based source. On success the platform stores the connection and returns the full updated source list.

POST /data/connect

Parameter	In	Required	Default	Description
`scope`	body	no	`"user"`	Scope of the connection.
`database`	body	yes	—	Connection descriptor (see fields below).
`computeRowCount`	body	no	`false`	Whether to count rows per table after connecting.

database object fields:

Field	Required	Description
`databaseType`	yes	Database type: `postgresql`, `mysql`, `snowflake`, `databricks`, `csv`, `excel`, `json`, `parquet`, `ttl`, `owl`, and others.
`host`	no	Host or path. For file-based types, omit to default to `disk/`; relative paths are automatically prefixed with `disk/`.
`port`	no	Port number.
`database`	no	Database name.
`schema`	no	Schema name.
`username`	no	Database username.
`password`	no	Database password.
`tables`	no	Array of table names to include.

Example

curl -X POST "$BASE_URL/data/connect" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "scope": "user",
    "database": {
      "databaseType": "postgresql",
      "host": "db.example.com",
      "port": 5432,
      "database": "customers",
      "schema": "public",
      "username": "reader",
      "password": "s3cr3t",
      "tables": ["orders", "users"]
    },
    "computeRowCount": false
  }'

{
  "status": "success",
  "message": "Database connection successful",
  "data": {
    "connectionStatus": true,
    "sources": [
      { "id": "src_a1b2", "name": "orders", "databaseType": "postgresql" },
      { "id": "src_c3d4", "name": "users",  "databaseType": "postgresql" }
    ],
    "errorMessage": null
  }
}

On a failed connection connectionStatus is false and errorMessage contains the driver error.

List data sources

Return all registered data sources for the authenticated user.

GET /data/list

Parameter	In	Required	Default	Description
`scope`	query	no	`"user"`	Scope of sources to return.

curl "$BASE_URL/data/list?scope=user" \
  -H "Authorization: Bearer $TOKEN"

data is an array of source objects (id, name, databaseType, host, schema, tables, and row counts if computed).

List sheets

Return the sheet names available in a spreadsheet or multi-tab source (e.g. an Excel file or Google Sheet connection).

POST /data/list-sheets

Parameter	In	Required	Default	Description
`database`	body	yes	—	Connection descriptor for the spreadsheet source.

curl -X POST "$BASE_URL/data/list-sheets" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "database": {
      "databaseType": "excel",
      "host": "reports.xlsx"
    }
  }'

{
  "status": "success",
  "message": "Success",
  "data": { "sheets": ["Sheet1", "Summary", "Raw"] }
}

Demo sources

Return the pre-built demo data sources shown during onboarding. Each preset includes its full database connection payload so the frontend can pre-populate the connect form.

GET /data/demo-sources

No parameters. Requires authentication.

curl "$BASE_URL/data/demo-sources" \
  -H "Authorization: Bearer $TOKEN"

{
  "status": "success",
  "message": "Demo datasources retrieved successfully",
  "data": [
    {
      "id": "northwind",
      "label": "Northwind (PostgreSQL)",
      "description": "Classic orders and products dataset.",
      "category": "database",
      "databaseType": "postgresql",
      "database": { "host": "demo.prometheux.ai", "port": 5432, "..." : "..." }
    }
  ]
}

Refresh sources

Re-probe every stored connection group and reconcile the source list. Database groups are re-discovered by (type, host, port, database, schema); file-based sources are removed if the file is no longer accessible.

POST /data/refresh

Parameter	In	Required	Default	Description
`scope`	body	no	`"user"`	Scope of sources to refresh.
`group_filter`	body	no	—	If set, refresh only sources matching this connection group identifier.

curl -X POST "$BASE_URL/data/refresh" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "scope": "user" }'

{
  "status": "success",
  "message": "Data sources refreshed successfully",
  "data": [ /* updated source list */ ],
  "failed_datasources": []   // present only when some sources failed
}

Infer schema

Inspect a connection and generate Vadalog bind annotations (and optionally a concept stub) for every table in the descriptor.

POST /data/infer-schema

Parameter	In	Required	Default	Description
`database`	body	yes	—	Connection descriptor for the source to inspect.
`addBind`	body	no	`true`	Emit a `@bind` annotation for each table.
`addModel`	body	no	`false`	Also emit a concept model stub for each table.

curl -X POST "$BASE_URL/data/infer-schema" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "database": {
      "databaseType": "postgresql",
      "host": "db.example.com",
      "port": 5432,
      "database": "customers",
      "username": "reader",
      "password": "s3cr3t"
    },
    "addBind": true,
    "addModel": false
  }'

{
  "status": "success",
  "message": "Schema inferred successfully",
  "data": {
    "binds": [
      "@bind(\"orders\", \"postgresql\", \"db.example.com\", ...).",
      "@bind(\"users\",  \"postgresql\", \"db.example.com\", ...)."
    ],
    "models": []
  }
}

Preview data

Return a paginated sample of raw rows from a bound data source, with optional search and column filters.

POST /data/preview

Parameter	In	Required	Default	Description
`bind_annotation`	body	yes	—	The full `@bind(...)` annotation string identifying the source.
`scope`	body	no	`"user"`	Scope of the source.
`page`	body	no	`1`	Page number.
`page_size`	body	no	`0`	Rows per page (`0` falls back to the server default of `10`).
`order_by`	body	no	—	e.g. `"0:asc,2:desc"`.
`search_term`	body	no	—	Free-text term matched across all columns.
`column_filters`	body	no	—	Array of `{ "position": <col index>, "value": <string> }`.

curl -X POST "$BASE_URL/data/preview" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "bind_annotation": "@bind(\"orders\", \"postgresql\", \"db.example.com\", 5432, \"customers\", \"public\", \"reader\", \"s3cr3t\").",
    "scope": "user",
    "page": 1,
    "page_size": 20
  }'

{
  "status": "success",
  "message": "Data preview retrieved successfully",
  "data": {
    "columns": ["order_id", "customer", "total"],
    "rows": [
      [1001, "Acme Corp", 450.00],
      [1002, "Globex",    129.99]
    ],
    "count": 2,
    "page": 1,
    "page_size": 20
  }
}

Joins

All-pairs join

Discover join relationships across a set of data sources. Given a list of source connection descriptors, the platform analyzes every pair to find candidate join keys, and optionally evaluates the resulting joins.

POST /data/all-pairs-join

Parameter	In	Required	Default	Description
`database_payloads`	body	yes	—	Array of connection descriptors (each with at least a `databaseType`) to analyze pairwise.
`to_evaluate`	body	no	`false`	When `true`, evaluate the discovered joins rather than only detecting them.
`parallel`	body	no	`true`	Analyze source pairs in parallel.

curl -X POST "$BASE_URL/data/all-pairs-join" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "database_payloads": [
      { "databaseType": "csv", "host": "disk/orders.csv" },
      { "databaseType": "csv", "host": "disk/customers.csv" }
    ],
    "to_evaluate": false,
    "parallel": true
  }'

{
  "status": "success",
  "message": "All pairs join completed successfully",
  "data": { /* discovered join keys and (optionally) evaluated results */ }
}

At least two sources are needed to produce any pairs — a single descriptor in database_payloads yields no joins.

Files

All file operations are scoped to the server-side disk/ storage directory. Paths are relative to that root; path traversal (..) is rejected.

Upload file

Upload a file to disk/. The request must be sent as multipart/form-data. The returned filePath (e.g. disk/reports.csv) can be used directly as the host in a subsequent /data/connect call.

POST /data/files/upload

Parameter	In	Required	Default	Description
`file`	form	yes	—	The file to upload (`multipart/form-data` field).
`path`	form	no	`""`	Subdirectory within `disk/` to upload into. Must already exist.

Allowed extensions: .csv, .xlsx, .xls, .json, .parquet, .pdf, .txt, .docx, .yaml, .ttl, .rdf, .owl, .cobol, and others configured by the server. Maximum file size: 10 GB (server-configurable).

curl -X POST "$BASE_URL/data/files/upload" \
  -H "Authorization: Bearer $TOKEN" \
  -F "file=@/local/path/customers.csv" \
  -F "path=datasets"

{
  "status": "success",
  "message": "File uploaded successfully",
  "data": {
    "filePath": "disk/datasets/customers.csv",
    "fileName": "customers.csv",
    "fileSize": 204800
  }
}

List files

List the contents of a directory within disk/. Directories are returned before files; hidden entries (names starting with .) are excluded.

GET /data/files/list

Parameter	In	Required	Default	Description
`path`	query	no	`""`	Subdirectory within `disk/` to list. Defaults to the root.

curl "$BASE_URL/data/files/list?path=datasets" \
  -H "Authorization: Bearer $TOKEN"

{
  "status": "success",
  "message": "Files listed successfully",
  "data": [
    { "name": "archive",       "type": "directory", "size": null,   "modified": "2026-06-01T09:00:00+00:00" },
    { "name": "customers.csv", "type": "file",      "size": 204800, "modified": "2026-06-20T14:32:11+00:00", "extension": "csv" }
  ]
}

Download file

Download a file from disk/. Returns the raw file bytes, not the standard envelope. The Content-Type is inferred from the file extension.

GET /data/files/download

Parameter	In	Required	Default	Description
`path`	query	yes	—	File path relative to `disk/` (e.g. `datasets/customers.csv`).

curl "$BASE_URL/data/files/download?path=datasets/customers.csv" \
  -H "Authorization: Bearer $TOKEN" \
  -O customers.csv

Make directory

Create a new subdirectory inside disk/. The directory must not already exist; its parent must exist.

POST /data/files/mkdir

Parameter	In	Required	Default	Description
`path`	body	yes	—	Path of the new directory relative to `disk/`.

curl -X POST "$BASE_URL/data/files/mkdir" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "path": "datasets/q2" }'

{
  "status": "success",
  "message": "Directory 'datasets/q2' created successfully",
  "data": { "path": "datasets/q2" }
}

Delete files

Delete one or more files or directories within disk/. Non-empty directories require recursive: true.

POST /data/files/delete

Parameter	In	Required	Default	Description
`paths`	body	yes	—	Array of paths relative to `disk/` to delete.
`recursive`	body	no	`false`	Delete non-empty directories recursively.

curl -X POST "$BASE_URL/data/files/delete" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "paths": ["datasets/old_report.csv", "archive"],
    "recursive": false
  }'

{
  "status": "success",
  "message": "Deleted 1 item(s), 1 error(s)",
  "data": {
    "deleted": ["datasets/old_report.csv"],
    "errors": [
      { "path": "archive", "error": "Directory not empty (use recursive: true)" }
    ]
  }
}

Move / rename file

Move or rename a file or directory within disk/. The destination must not already exist and its parent directory must exist.

POST /data/files/move

Parameter	In	Required	Default	Description
`source`	body	yes	—	Current path relative to `disk/`.
`destination`	body	yes	—	New path relative to `disk/`.

curl -X POST "$BASE_URL/data/files/move" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "source":      "datasets/customers_v1.csv",
    "destination": "datasets/customers.csv"
  }'

{
  "status": "success",
  "message": "Moved 'datasets/customers_v1.csv' to 'datasets/customers.csv'",
  "data": {
    "source":      "datasets/customers_v1.csv",
    "destination": "datasets/customers.csv"
  }
}

Other operations

Operation	Endpoint	Notes
Remove data sources	`POST /data/cleanup`	Body: `scope` (default `"user"`), `source_ids` (array; omit to delete all sources in scope).

Introduction

Get Started

Platform

Integrations

Examples

Vadalog Reference

Resources

Connections

Connect a data source

Example

List data sources

List sheets

Demo sources

Refresh sources

Infer schema

Preview data

Joins

All-pairs join

Files

Upload file

List files

Download file

Make directory

Delete files

Move / rename file

Other operations

​Connections

​Connect a data source

​Example

​List data sources

​List sheets

​Demo sources

​Refresh sources

​Infer schema

​Preview data

​Joins

​All-pairs join

​Files

​Upload file

​List files

​Download file

​Make directory

​Delete files

​Move / rename file

​Other operations

Connections

Connect a data source

Example

List data sources

List sheets

Demo sources

Refresh sources

Infer schema

Preview data

Joins

All-pairs join

Files

Upload file

List files

Download file

Make directory

Delete files

Move / rename file

Other operations