Prometheux executes Vadalog reasoning on a configurable compute layer. Depending on your deployment, workloads can run on Databricks, on a self-managed Yarn or Kubernetes cluster, or locally in a single JVM for development and testing. Under the hood, the engine converts its primitives (project, select, join) into map, filter, reduce, and shuffle transformations executed in parallel on Apache Spark.

​
Compute options

OptionBest forReference
DatabricksTeams already on DatabricksDatabricks installation
Self-managed cluster (Yarn / Kubernetes)Full control over compute, networking, and storageCluster installation
Local modeDevelopment and testingSee below

​
Databricks

Prometheux integrates with Databricks in two ways — as a native application in your workspace, or by installing the engine directly on your clusters.
MethodBest forDescription
Databricks Native AppFull platform experienceDeploy the complete Prometheux UI, backend, and services as a Databricks app
Installing PX on Databricks & ConnectorsEngine-level integrationConnect Prometheux to Databricks via JDBC and install the engine JAR on clusters

​
Self-managed clusters

Prometheux can run on your own compute infrastructure using Apache Spark as the execution layer. This suits organisations that require full control over their cluster, networking, and storage.
This is an advanced deployment path. We recommend contacting the Prometheux team for guidance before proceeding.

​
Supported cluster managers

  • Yarn — supports both client mode (the driver resides on the client machine and submits the program to the Yarn Resource Manager) and cluster mode (the driver runs inside the cluster on the application master node).
  • Kubernetes — supports client mode (the driver runs outside the cluster and schedules executor pods via the API Server) and cluster mode (the driver runs inside a pod on a worker node).
  • Local mode — the driver, master, and executor run in a single JVM on the workstation. Useful for development and testing.
For full prerequisites and the complete configuration reference (Spark settings, database properties, GPU acceleration, and the Livy REST service), see the Cluster installation guide.

​
Tuning Spark resources

Compute behaviour is controlled through Spark configuration. A few of the most common properties:
PropertyDefaultDescription
spark.masterlocal[*]Master URL (local[*], spark://HOST:PORT, yarn)
spark.submit.deployModeclientclient or cluster
spark.driver.memory4gDriver memory
spark.executor.memory4gExecutor memory
spark.executor.instances1Number of executors
spark.executor.cores4Cores per executor
spark.dynamicAllocation.enabledfalseDynamic executor allocation
computeAcceleratorPreferencecpucpu or gpu (GPU-enabled environments only)
For the full set of Spark, database, GPU (Spark-RAPIDS), and Livy properties, see the Cluster installation guide.