Skip to content

Service Deployment ​

Deployment has two stages: prepare a Kubernetes cluster, then deploy SQLRec and its dependencies. Minikube and existing clusters use the same download_resource.sh and deploy_components.sh; STORAGE_MODE selects how client files are stored.

System Requirements ​

The deployment machine can run AMD64/ARM64 Linux or Apple Silicon macOS. The first deployment needs access to image registries, Helm repositories, and resource download URLs.

DeploymentPreparation
Minikube development environmentLinux uses the Docker driver; macOS uses vfkit, vmnet-shared networking, and VirtioFS mounts. deploy_minikube.sh installs missing tools and starts the cluster
Existing Kubernetes clusterConfigure kubeconfig and install kubectl, helm, curl, tar, gzip, envsubst, psql, and either sha256sum or shasum; Minikube is not required

Minikube deployment on macOS requires macOS 14 or later and Homebrew. The main packages installed by the script are equivalent to:

bash
brew install minikube vfkit docker docker-buildx helm gettext libpq

The script also installs kubernetes-cli if kubectl is missing and configures Buildx and vmnet-helper. Docker Desktop is not required on macOS. Use the checks under deploy/ for exact tool version requirements.

Quick Deployment (Minikube) ​

bash
git clone https://github.com/sqlrec/sqlrec.git
cd sqlrec/deploy

# Install tools, start the cluster, and configure its default StorageClass
bash ./deploy_minikube.sh
kubectl get nodes -o wide

# Download clients/JARs, add Helm repositories, and install required operators
bash ./download_resource.sh

# Deploy SQLRec and its dependencies
bash ./deploy_components.sh
kubectl get pods --all-namespaces

# Verify the connection after required components are ready
cd ..
bash ./bin/beeline.sh

A successful SHOW TABLES; in Beeline confirms that the core SQLRec service is ready.

Minikube configuration:

  • The script mounts the host's data directory into Minikube at the same path. Clients and JARs use the host files directly, without uploading a second copy to PVCs.
  • On macOS, defaults are the host's physical core count, 80% of total memory, and a 256GB disk. Override them with MINIKUBE_CPUS, MINIKUBE_MEMORY_PERCENT, MINIKUBE_MEMORY, or MINIKUBE_DISK_SIZE; an explicit MINIKUBE_MEMORY takes precedence. Actual resource needs depend on enabled components and data volume.
  • Dynamic volumes use the project-installed local-path StorageClass by default, with data at /data/local-path-provisioner on the node. Set LOCAL_PATH_PROVISIONER_DATA_DIR before startup to use another absolute path. These dynamic volumes are separate from the hostPath PVs mounting host client directories.
  • NodePorts use the Minikube node address. Access through the host's physical IP from other LAN machines is not guaranteed.
Save Image Caches Manually

After deploying components, optionally run:

bash
# Run from deploy; save workload images that have already been pulled
bash ./cache_images.sh save

Caches are stored in data/image-cache/<arch>. Later runs of deploy_minikube.sh load existing caches automatically. deploy_components.sh prints the save command but does not save images automatically. Image caches do not contain database data; deleting the Minikube cluster loses data in node dynamic volumes.

Deploy to an Existing Kubernetes Cluster ​

This flow installs components in the cluster selected by the current kubeconfig context. It does not create a cluster.

Prerequisites ​

  1. Ensure the deployment account can create namespaces, install operators/CRDs, and create the RBAC resources used by the scripts. Besides downloading files, download_resource.sh installs operators including CloudNativePG and Flink and configures Helm repositories.
  2. Prepare a StorageClass supporting ReadWriteMany (RWX) and set STORAGE_CLASS so clients and JARs can be shared across nodes. The script checks that the class exists; the cluster administrator must confirm RWX support.
  3. Ensure a usable default StorageClass is available for data volumes such as PostgreSQL and RustFS. It can differ from the RWX class used for clients.
  4. Select a worker node address reachable from both the deployment machine and pods as NODE_IP, and ensure the required NodePorts are accessible. Discovery selects a Ready worker that is not cordoned, excludes control-plane nodes, and uses its InternalIP. Set a reachable address explicitly if the deployment machine cannot reach that InternalIP.
  5. If the deployment machine and cluster nodes have different architectures, set CONTAINER_ARCH=amd64 or arm64 before downloading resources to select container Java and Hadoop native libraries. This does not change host Java or automatically reconcile different architectures within a cluster.

Deployment Example ​

Run these commands in the same terminal, replacing the example context, address, and StorageClass:

bash
cd /path/to/sqlrec
kubectl config use-context your-cluster-context
kubectl get nodes -o wide
kubectl get storageclass

export NAMESPACE=sqlrec
export NODE_IP=10.0.0.10
export STORAGE_MODE=shared
export STORAGE_CLASS=nfs-rwx
export CONTAINER_ARCH=amd64

bash ./deploy/download_resource.sh
bash ./deploy/deploy_components.sh

kubectl get pods -n "${NAMESPACE}"
kubectl get pvc -n "${NAMESPACE}"
bash ./bin/beeline.sh

K8S_APISERVER_ADDR is read from the current kubeconfig by default, in the form k8s://https://.... It accesses the Kubernetes API and is separate from NODE_IP, which is used for NodePorts.

deploy_components.sh deploys PostgreSQL, RustFS/JuiceFS, Hadoop configuration, HDFS, Spark configuration, HMS, Flink SQL Gateway, SQLRec, and Kafka, Redis, and Milvus by default. Milvus runs in ${NAMESPACE}-milvus. MongoDB, Kyuubi, Jupyter, and observability components use their own deployment scripts. The default filesystem remains jfs://myjfs; access HDFS explicitly through hdfs://myhdfs.

HDFS uses one NameNode and one DataNode on the selected NODE_IP node. The deployment preserves existing metadata. Use separate capacity and availability planning for workloads that need more than this development setup.

Storage and Resource Synchronization ​

Storage Modes ​

Itemhostpath (Minikube)shared (existing cluster)
Templatedeploy/storage/hostpath_pvc.yamldeploy/storage/shared_pvc.yaml
ResourcesStatic hostPath PVs + PVCs, ReadWriteOncePVCs provisioned by the specified StorageClass, ReadWriteMany
File sourceHost directories mounted into MinikubeDeployment machine files uploaded to shared PVCs
Upload behaviorCopy local configuration only; skip uploadA temporary pod mounts the PVCs and uploads changed files

If STORAGE_MODE is unset, a context named minikube selects hostpath; all other contexts select shared. deploy_minikube.sh itself always uses hostpath. Remote clusters should use shared: local deployment directories do not automatically exist on remote nodes.

Default PVC nameContentsCapacity setting
sqlrec-lib-pvcDependency JARsLIB_STORAGE_SIZE, default 128Gi
sqlrec-client-pvcHadoop, Hive, Spark, container Java clients and configurationCLIENT_STORAGE_SIZE, default 128Gi

Override PVC names with LIB_PVC_NAME and CLIENT_PVC_NAME. Existing PVC storage classes and access modes cannot be switched in place; the script stops on a mismatch. Use new PVC names and plan file migration when changing storage backends. hostPath PVs use the Retain reclaim policy; override their names with LIB_PV_NAME and CLIENT_PV_NAME.

RustFS provides S3 storage in standalone mode, and the deployment creates the buckets required by JuiceFS and Milvus. Data volumes always use the cluster's default StorageClass; STORAGE_CLASS for clients/JARs does not affect RustFS. Set capacities with RUSTFS_DATA_STORAGE_SIZE and RUSTFS_LOG_STORAGE_SIZE, defaulting to 128Gi and 1Gi respectively.

Resource Synchronization ​

Deployment scripts prepare clients, JARs, and configuration automatically. No manual PV/PVC template application or separate HMS configuration step is needed. After updating prepared resources, run bash ./deploy/storage/sync.sh from the repository root. Synchronization does not remove extra remote files; do not run it concurrently in one namespace.

Local resources are under deploy/data by default, or ${BASE_DIR}/data when configured: conf/ contains generated configuration, lib/ contains JARs, and client/ contains clients and Java. Shared-mode synchronization uploads runtime files and skips unchanged ones.

bash
bash ./deploy/storage/sync.sh

Production Configuration ​

The existing-cluster flow can be reused, but SRE teams should review component capacity, persistent storage, networking, credentials, and permissions for production. Default deployments include single-instance services and NodePorts; adapt them to your operational requirements.

Dependencies ​

ServicePurpose
KubernetesManage model training, export, and serving
PostgreSQLModel, service, and function metadata
Hive MetastoreHive table metadata
Flink SQL GatewayOptional; executes forwarded Flink SQL/RPCs
Distributed storageModel files and training data; RustFS + JuiceFS by default

Use Kafka, Redis, Milvus, Spark, Kyuubi, and Jupyter according to workload needs. Kafka, Redis, and Milvus are already included in the default component deployment flow.

Script Parameters and Container Configuration ​

Scripts read deploy/env.sh. Override supported parameters using matching environment variables before execution, for example:

bash
export NAMESPACE=dev
export SQLREC_VERSION=your-version
bash ./deploy/download_resource.sh
bash ./deploy/deploy_components.sh

Keep namespace, version, path, and architecture settings consistent between resource download and deployment. Main script parameters and container settings include:

ParameterPurpose
NODE_IPNodePort address reachable from both the deployment machine and pods
SQLREC_POSTGRESQL_USER / SQLREC_POSTGRESQL_PASSWORDSQLRec metadata database credentials
HMS_POSTGRESQL_USER / HMS_POSTGRESQL_PASSWORDHMS metadata database credentials
SQLREC_POSTGRESQL_PORT / HMS_POSTGRESQL_PORTDatabase NodePorts
HMS_PORTHMS NodePort
FLINK_SQL_GATEWAY_ADDRESSGateway host; defaults to NODE_IP during deployment, and an empty address disables forwarding
FLINK_SQL_GATEWAY_PORTGateway NodePort and SQLRec connection port, default 30018
FLINK_SQL_GATEWAY_CONNECT_TIMEOUTSQLRec container runtime setting; connection and RPC read timeout, default 600000 ms (10 minutes)
SQLREC_THRIFT_PORT / SQLREC_REST_PORTJDBC/Beeline / REST NodePorts, default 30000 / 30001
RUSTFS_ACCESS_KEY / RUSTFS_SECRET_KEYShared S3 credentials
DEPLOY_TIMEOUTDeployment wait timeout, default 3600 seconds

The SQLRec container's META_DB_URL and HIVE_METASTORE_URI are generated in deploy/sqlrec/sqlrec.yaml from NODE_IP and the corresponding ports. MODEL_BASE_PATH is fixed to /user/sqlrec/models in that template.

When starting SQLRec directly, set META_DB_URL and HIVE_METASTORE_URI for remote metadata. FLINK_SQL_GATEWAY_ADDRESS and TRACE_ENDPOINT default to empty and disable those features. For SQL-file mode, set SQL_SCHEMA_DIR instead of metadata endpoints.

Connect using CLI, Beeline, or Python. Source development and integration tests are covered in Development and Testing.

To use dependencies managed by SRE teams, adjust the corresponding deployment steps and YAML connection settings. The scripts do not skip component deployment when an external address is set. Exporting META_DB_URL or HIVE_METASTORE_URI alone does not replace the template addresses. Even running deploy/sqlrec/deploy.sh alone deploys PostgreSQL first.

SQLRec and Spark scripts create their ServiceAccounts and grant cluster-wide edit access through ClusterRoleBindings. For production, scope permissions to the required resources and namespaces using restricted Roles/RoleBindings.

Execution Variables and Result Parameters ​

These options are read only from the server process environment and default to enabled. SQL SET and request params cannot override them.

Environment variableDefaultDescription
ENABLE_VARIABLE_ENV_FALLBACKtrueWhen an execution variable is absent (null), SQL reads the same-named environment variable; explicit empty strings do not trigger fallback
EXECUTE_RESULT_INCLUDE_PARAMStrueInclude execution variables as params in SQL and function REST API results; disabling omits this JSON field without affecting variable use during execution

Set either environment variable to false and restart the service to disable it. Fallback values are never written into execution variables: get_variables() and response params contain only explicit execution variables. Removing an explicit variable restores environment fallback on subsequent reads. With fallback enabled, SQL can read server process environment settings by variable name.

Entry Authentication ​

Direct server startup disables authentication by default. Deployment scripts enable it with development credentials sqlrec / abc123456. Override them before deployment:

Environment variableServer defaultDescription
AUTH_ENABLEDfalseEnable HTTP Basic and JDBC/Thrift SASL PLAIN
AUTH_USERNAMEEmptyComma-separated usernames
AUTH_PASSWORDEmptyPasswords paired with usernames by position
bash
AUTH_ENABLED=true AUTH_USERNAME=alice AUTH_PASSWORD='replace-with-your-password' \
  bash deploy/sqlrec/deploy.sh

For multiple accounts, set AUTH_USERNAME=alice,bob and AUTH_PASSWORD=alice-secret,bob-secret. Both lists must have the same length; entries must be non-blank, usernames unique, and usernames must not contain colons. Commas separate entries and cannot be escaped. Spaces are part of credentials; control characters are rejected. Invalid credentials prevent startup.

Set these values in the container environment or in a file loaded by bin/server.sh through SQLREC_ENV_CONF. Changes require a restart or redeployment; SQL SET and request params cannot override them. Deployment templates write credentials into container environment variables, including the rendered YAML.

  • HTTP and UI require Basic credentials; missing or incorrect credentials return 401. GET /metrics remains anonymous.
  • Authenticated clients have the same permissions; role-based and per-resource authorization are not available.
  • Basic and SASL PLAIN do not encrypt transport. Use HTTPS or an appropriate TLS tunnel where required.

Client examples, including Beeline and PyHive, are in Client Connections.

Gateway Forwarding Configuration ​

Deployment scripts install Flink SQL Gateway and configure SQLRec's connection address by default. Gateway serves requests that require Flink execution; local SQL, metadata queries such as SHOW CREATE TABLE and DESCRIBE, JDBC metadata, and persistent table/database/UDF metadata DDL do not depend on it.

Default settings require no changes. For an external Gateway, set the address and port listed above to its HiveServer2/Thrift endpoint. An empty address followed by redeployment disables forwarding; this does not skip default Flink component deployment or remove existing components.

bash
export FLINK_SQL_GATEWAY_ADDRESS=""
bash ./deploy/sqlrec/deploy.sh

See Architecture for execution boundaries and the SQL Reference for SET/RESET behavior.

Local SQL-File Metadata ​

Environment variableDefaultDescription
SQL_SCHEMA_DIREmpty stringRecursively load SQL definitions from this directory when nonempty; otherwise use remote metadata
SQL_SCHEMA_DDL_ENABLEDfalseEnable in-memory database, table, Java UDF, SQL function, and API DDL in file mode only

Configure these in the container environment or SQLREC_ENV_CONF, then restart the service. Session SET cannot change the option. Changes are shared by sessions in one process, never rewrite SQL files, and are lost on restart. Separate instances do not share changes. FLUSH preserves in-memory changes. Model and Service lifecycle commands retain their existing restrictions. See Metadata DDL for supported statements.

SQL Initialization ​

deploy/sqlrec/deploy.sh waits for PostgreSQL, runs deploy/sql/master.sql, then deploys SQLRec. No manual SQL import is needed. Existing tables are reused; this file does not migrate existing table schemas.

If PostgreSQL is not ready, adjust PSQL_MAX_ATTEMPTS (default 30), PSQL_RETRY_INTERVAL (5 seconds), or PGCONNECT_TIMEOUT (10 seconds). SQL initialization errors stop deployment and roll back that initialization transaction; fix the reported SQL error before retrying.

Verification and Troubleshooting ​

Run from the repository root. If NAMESPACE is unset, these commands use the default sqlrec:

bash
kubectl get pods -n "${NAMESPACE:-sqlrec}"
kubectl get pvc -n "${NAMESPACE:-sqlrec}"
bash ./bin/beeline.sh
SymptomCheck
Missing or invalid client/JARRun deploy/download_resource.sh first; if the error persists, remove the affected corrupt file or extracted directory and rerun
Shared PVC or sync pod remains PendingUse kubectl describe pvc <name> -n <namespace>; check RWX support, capacity, and permissions. WaitForFirstConsumer classes begin binding after the sync pod is created
PostgreSQL connection retries exhaustedCheck database pod readiness, NODE_IP, NodePorts, routes, and credentials. Fix SQL errors using the psql output
Sync pod already existsWait for the active sync to finish; remove a leftover pod only after confirming that the previous sync has stopped

See Observability for metrics, traces, and logs after deployment.