Docker & Kubernetes
Deployment Best Practices

The canonical compose file for Smartflow is docker-compose.simple.yaml in the Smartflow_docker repo. All four services share a single Docker image (langsmartai/safechat-enterprise:latest); the SERVICE_TYPE environment variable selects which binary runs.

Always set mem_limit and cpus constraints in Compose. Without them a runaway request loop (e.g. a streaming response that never terminates) can exhaust host memory and take down all services.

# docker-compose.simple.yaml — recommended resource constraints
services:
  smartflow-proxy:
    mem_limit: 1g
    memswap_limit: 1g       # disable swap for this container
    cpus: 2.0              # allow up to 2 cores; adjust for your host
    restart: unless-stopped
    ulimits:
      nofile:
        soft: 65536
        hard: 65536

  smartflow-api-server:
    mem_limit: 512m
    cpus: 1.0
    restart: unless-stopped

  smartflow-compliance:
    mem_limit: 768m         # compliance ML models need headroom
    cpus: 1.5
    restart: unless-stopped

  smartflow-policy-perfect:
    mem_limit: 256m
    cpus: 0.5
    restart: unless-stopped

Caddy handles TLS termination and routes to the correct service port. Its automatic HTTPS via Let's Encrypt requires ports 80 and 443 to be open and the DNS A record to resolve to this host before Caddy starts.

# /etc/caddy/Caddyfile — production routing rules
your-host.example.com {
    # Proxy routes → port 7775
    handle /v1/*            { reverse_proxy localhost:7775 }
    handle /anthropic/*     { reverse_proxy localhost:7775 }
    handle /cursor/*        { reverse_proxy localhost:7775 }
    handle /a2a/*           { reverse_proxy localhost:7775 }
    handle /api/mcp/*       { reverse_proxy localhost:7775 }
    handle /.well-known/*   { reverse_proxy localhost:7775 }

    # Management routes → port 7778
    handle /api/guardrails*  { reverse_proxy localhost:7778 }
    handle /api/policies*    { reverse_proxy localhost:7778 }
    handle /api/auth*        { reverse_proxy localhost:7778 }
    handle /api/enterprise*  { reverse_proxy localhost:7778 }
    handle /api/routing*     { reverse_proxy localhost:7778 }
    handle /api/mcp/tools*   { reverse_proxy localhost:7778 }
    handle /api/mcp/auth*    { reverse_proxy localhost:7778 }
    handle /api/admin/mcp*   { reverse_proxy localhost:7778 }
    handle /api/metacache*   { reverse_proxy localhost:7778 }

    # Compliance → port 7777, Policy Perfect → port 7782
    handle /api/compliance*  { reverse_proxy localhost:7777 }
    handle /api/policy*      { reverse_proxy localhost:7782 }

    # Chat UI
    handle { reverse_proxy localhost:3600 }

    # Performance
    encode gzip
    header Strict-Transport-Security "max-age=31536000; includeSubDomains"
}

Always configure health checks so Docker can automatically restart unhealthy containers:

healthcheck:
  test: ["CMD", "curl", "-sf", "http://localhost:7775/health"]
  interval: 30s
  timeout: 5s
  retries: 3
  start_period: 20s    # allow binary startup time

The Smartflow Helm chart (helm/smartflow) deploys all four services plus TimescaleDB, Redis, and the SafeChat frontend. The chart is designed for a 3-node cluster with at least 4 vCPU and 8 GB RAM per node (DigitalOcean s-4vcpu-8gb or equivalent) for comfortable production workloads.

# Install cert-manager first (required for TLS)
helm repo add jetstack https://charts.jetstack.io --force-update
helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager --create-namespace \
  --version v1.19.4 \
  --set crds.enabled=true \
  --set resources.requests.cpu=10m \
  --set resources.requests.memory=64Mi \
  --set resources.limits.cpu=100m \
  --set resources.limits.memory=128Mi \
  --set webhook.resources.requests.cpu=10m \
  --set webhook.resources.requests.memory=32Mi \
  --set webhook.resources.limits.cpu=50m \
  --set webhook.resources.limits.memory=64Mi \
  --set cainjector.resources.requests.cpu=10m \
  --set cainjector.resources.requests.memory=32Mi \
  --set cainjector.resources.limits.cpu=50m \
  --set cainjector.resources.limits.memory=64Mi \
  --set webhook.timeoutSeconds=29

# Install NGINX ingress controller
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace

# Deploy Smartflow
helm upgrade --install smartflow ./helm/smartflow \
  --namespace smartflow --create-namespace \
  -f values.yaml

# helm/smartflow/values.yaml — production settings

proxy:
  replicas: 2
  image:
    repository: langsmartai/safechat-enterprise
    tag: latest
    pullPolicy: Always
  resources:
    requests:
      cpu: 200m
      memory: 256Mi
    limits:              # No CPU limit — avoids CFS throttle jitter (see Perf section)
      memory: 512Mi

apiServer:
  replicas: 2
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      memory: 256Mi

compliance:
  replicas: 1
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      memory: 512Mi

policyPerfect:
  replicas: 1
  resources:
    requests:
      cpu: 50m
      memory: 128Mi
    limits:
      memory: 256Mi

ingress:
  enabled: true
  host: smartflow.your-domain.com
  tls: true
  clusterIssuer: letsencrypt-prod
  annotations:
    # Disable buffering — critical for streaming LLM responses
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    # Large body support for file/image uploads
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    # Long timeouts for streaming responses
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"

Smartflow uses cert-manager for automatic TLS certificate issuance and renewal via Let's Encrypt. The ClusterIssuer must be configured before enabling TLS on the ingress.

# Apply after cert-manager is running
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ops@your-domain.com
    privateKeySecretRef:
      name: letsencrypt-prod-key
    solvers:
      - http01:
          ingress:
            class: nginx

DigitalOcean's cluster upgrade process requires all admission webhook timeouts to be between 1 and 29 seconds. The default cert-manager install sets this to 30s, which blocks node upgrades. The Helm install command above sets webhook.timeoutSeconds=29 to handle this automatically. Verify with:

kubectl get validatingwebhookconfiguration cert-manager-webhook \
  -o jsonpath='{.webhooks[*].timeoutSeconds}'
# Should output: 29

Out of the box, a Kubernetes deployment adds 2–5ms of per-request overhead compared to bare metal, primarily from the overlay network, kube-proxy iptables chains, and NGINX ingress buffering. The configuration in this guide reduces that to approximately 1–2ms by applying the following tunings.

service:
  externalTrafficPolicy: Local

dnsConfig:
  options:
    - name: ndots
      value: "2"

proxy-buffering: "off"
proxy-request-buffering: "off"
proxy-read-timeout: "300"

affinity:
  nodeAffinity:
    requiredDuringScheduling...:
      nodeSelectorTerms:
        - matchExpressions:
            - key: role
              operator: In
              values: [smartflow]

# Check a specific feature string is present in the running binary
# (never assume the running container matches the source code)
docker exec smartflow-proxy strings /usr/local/bin/smartflow | grep "semantic_cache"

# On Kubernetes
kubectl exec -n smartflow deploy/smartflow-proxy -- \
  strings /usr/local/bin/smartflow | grep "semantic_cache"

# Verify cert-manager webhook timeout before upgrade
kubectl get validatingwebhookconfiguration cert-manager-webhook \
  -o jsonpath='{.webhooks[*].timeoutSeconds}'
# Expected: 29

# Verify all Smartflow pods are healthy
kubectl get pods -n smartflow
kubectl top pods -n smartflow   # check for memory pressure

# Manual scale — proxy pods
kubectl scale deploy smartflow-proxy -n smartflow --replicas=3

# HPA — autoscale proxy between 2 and 8 replicas based on CPU
kubectl autoscale deploy smartflow-proxy -n smartflow \
  --cpu-percent=60 --min=2 --max=8

# Bash quick test
export SMARTFLOW_HOST=https://smartflow.your-domain.com
export VIRTUAL_KEY=sk-sf-your-key
bash smartflow_integration_test.sh

# Full Python suite (requires httpx, openai, anthropic)
python3 smartflow_integration_test.py