Smartflow — API and SDK Reference

Architecture Overview

Smartflow runs as five cooperating services:

All five services share one Redis instance for shared state: routing tables, semantic cache, VAS logs, provider latency metrics, virtual key budgets, and MCP server registry. The Policy Perfect API additionally requires PostgreSQL for durable policy and preset storage. In production the proxy sits behind a TLS-terminating reverse proxy (Caddy or nginx). Management, compliance, and policy APIs are backend surfaces.

Authentication

Virtual Keys

The primary credential for clients. Issue sk-sf-{48-hex} tokens via the management API (POST /api/enterprise/vkeys). Each key is stored in Redis and carries optional USD budget caps, period resets, and rate limits. The raw token is returned exactly once at creation — it cannot be retrieved again.

# OpenAI-compatible and all /v1/* routes
Authorization: Bearer sk-sf-a1b2c3d4...

# Anthropic native /anthropic/v1/* routes
x-api-key: sk-sf-a1b2c3d4...

On every request the proxy:

1. Extracts the sk-sf-* token from Authorization: Bearer or x-api-key
2. Looks up the key in Redis — returns 429 if not found or revoked
3. Checks USD budget against accumulated spend — returns 429 with X-Smartflow-Budget-Exceeded: true if exceeded
4. Forwards request to the provider using the server-side provider API key
5. Records actual spend (cost_usd) back to the key's spend counter after completion

Provider API Keys

Stored server-side in Redis (smartflow:api_keys:{provider}_api_key). Clients never send raw provider credentials. The proxy resolves the correct key from the key store when forwarding to each provider.

Anthropic Native Passthrough

For /anthropic/* routes, clients send their Smartflow virtual key as x-api-key. The proxy validates budget, then replaces it with the real ANTHROPIC_API_KEY before forwarding.

JWT (SafeChat / Dashboard)

The SafeChat web app and admin dashboard use a smartflow_token cookie-based JWT for browser sessions. JWT validation occurs at the application layer, not in the proxy core.

LLM Proxy Endpoints

All proxy endpoints are on port 7775 by default.

/v1/chat/completions

OpenAI-compatible chat completions. Accepts any OpenAI-format request body. Provider and model are resolved from the model name or an explicit prefix.

{
  "model": "gpt-4o",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "What is the capital of France?"}
  ],
  "temperature": 0.7,
  "max_tokens": 256,
  "stream": false
}

Model-prefix routing:

No prefix is required for the primary supported providers — model name heuristic detects gemini-*, claude-*, gpt-*, etc. automatically. An explicit provider/model prefix always takes precedence.

Multimodal — Image

{
  "model": "gpt-4o",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "What is in this image?"},
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
    ]
  }]
}

Multimodal — Audio (gpt-4o-audio-preview)

{
  "type": "input_audio",
  "input_audio": {"data": "<base64>", "format": "mp3"}
}

Response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "gpt-4o",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": "Paris."},
    "finish_reason": "stop"
  }],
  "usage": {"prompt_tokens": 24, "completion_tokens": 3, "total_tokens": 27}
}

/anthropic/v1/messages

Native Anthropic Messages API passthrough. The proxy injects the API key from the server key store. The full Anthropic request and response format is preserved with no translation. Also accessible as /cursor/v1/messages for Cursor IDE passthrough. The [1m] suffix that Claude Code appends to model names is stripped automatically.

{
  "model": "claude-sonnet-4-6",
  "max_tokens": 1024,
  "system": "You are a helpful assistant.",
  "messages": [
    {"role": "user", "content": "Hello, Claude."}
  ]
}

Multimodal — Image (native Anthropic)

{
  "type": "image",
  "source": {"type": "base64", "media_type": "image/png", "data": "..."}
}

Multimodal — PDF Document (native Anthropic)

{
  "type": "document",
  "source": {"type": "base64", "media_type": "application/pdf", "data": "..."}
}

/v1/embeddings

Generate vector embeddings. Supports multi-provider routing via model prefix.

{"model": "text-embedding-3-small", "input": "Your text here"}

Response follows the OpenAI embeddings format with data[].embedding float arrays.

/v1/audio/transcriptions

Transcribe audio. Multipart form upload. Routes to OpenAI Whisper by default. Use groq/whisper-large-v3 for Groq, deepgram/nova-2 for Deepgram.

Content-Type: multipart/form-data

file=@audio.mp3
model=whisper-1

/v1/audio/speech

Text-to-speech synthesis. Returns raw audio bytes.

{
  "model": "tts-1", "input": "Hello, world.",
  "voice": "nova", "response_format": "mp3"
}

/v1/images/generations

{
  "model": "dall-e-3", "prompt": "A futuristic city at sunrise",
  "n": 1, "size": "1024x1024", "quality": "hd",
  "style": "vivid", "response_format": "url"
}

/v1/rerank

Document reranking. Compatible with Cohere's rerank API.

{
  "model": "rerank-english-v3.0",
  "query": "What is the return policy?",
  "documents": ["Document one text.", "Document two text."],
  "top_n": 3
}

/v1/models

List available models across all enabled providers.

/v1/completions

Legacy text completions. Forwarded to the configured provider.

Routing and Provider Selection

Automatic Model-Name Heuristic

For requests to /v1/chat/completions with no explicit provider prefix, the proxy infers the provider from the model name. An explicit provider/model prefix always takes precedence over heuristic detection.

Routing Strategies

Configured per fallback chain via the management API:

Fallback Chains

Named ordered provider lists with retry logic. Configured at POST /api/routing/fallback-chains.

{
  "name": "production-chain",
  "targets": [
    {"provider": "openai",    "model": "gpt-4o",                        "weight": 1},
    {"provider": "anthropic", "model": "claude-sonnet-4-6", "weight": 1},
    {"provider": "google",    "model": "gemini-1.5-pro",             "weight": 1}
  ],
  "retry_on": ["429", "500", "502", "503"],
  "max_retries": 2,
  "backoff_ms": 500
}

On 429 or 5xx the proxy retries the next target with exponential backoff. Non-retryable 4xx errors bypass retry. Providers that have exceeded their daily budget cap are excluded from selection automatically.

MetaCache — Semantic Caching

The MetaCache intercepts every /v1/chat/completions request before any provider call is made.

How It Works

The incoming query is embedded and its cosine similarity is computed against stored request embeddings. If similarity exceeds the configured threshold, the cached response is returned. Otherwise the request is forwarded to the provider and the response is stored. Responses are semantically compressed before storage to reduce Redis footprint while preserving meaning.

Three tiers operate in sequence: L1 in-process memory, L2 Redis semantic similarity, L3 Redis exact match. Every cache lookup traverses all three before forwarding.

Per-Request Cache Controls

Cached responses return x-smartflow-cache-hit: true and x-smartflow-cache-key for client-side correlation.

MCP Gateway

Smartflow implements the Model Context Protocol (MCP) gateway. Register external MCP servers and invoke their tools through the proxy with shared authentication, budgeting, and audit logging.

Server Registry

List registered MCP servers.

Register an MCP server.

{
  "id": "github-tools",
  "name": "GitHub MCP Server",
  "base_url": "https://mcp.github.example.com",
  "auth_type": "bearer",
  "allowed_tools": ["list_repos", "create_issue"],
  "disallowed_tools": [],
  "cost_info": {"per_call_usd": 0.001},
  "guardrail_mode": "strict"
}

Tool Invocation

The proxy authenticates the request, applies per-tool access controls, records cost, and forwards to the server.

Catalog and Search

Browse the tool catalog across all registered servers.

Semantic search over the tool catalog. Returns the top k tools matching the natural-language query.

Full indexed tool list with embedding metadata.

Access Control

Per-server configuration fields for access control:

Access Request Flow

OAuth Flow

Usage and Logs

Aggregated cost and call counts per server and tool.

Per-invocation audit logs.

API Generation from OpenAPI Spec

Auto-generate an MCP server adapter from an OpenAPI specification.

{
  "spec": "<OpenAPI JSON or YAML string>",
  "server_id": "my-api",
  "server_name": "My REST API",
  "base_url": "https://api.example.com",
  "include_methods": ["GET", "POST"]
}

A2A Agent Gateway

Smartflow implements the A2A (Agent-to-Agent) protocol for inter-agent communication. Register external agents and invoke them with full logging and routing.

Agent Card

Returns the agent's machine-readable capability card: name, capabilities, supported task types, and authentication requirements.

Task Invocation

Send a task to a registered agent. The proxy forwards the request, captures the response, and logs both.

{
  "id": "task-uuid",
  "message": {
    "role": "user",
    "parts": [{"type": "text", "text": "Summarize the latest earnings report."}]
  }
}

Supports synchronous JSON responses and SSE streaming for long-running tasks. Include x-a2a-trace-id to correlate task invocations across agents in distributed workflows.

Vector Store API

Built-in vector store backed by Redis. No external vector database required. All endpoints are on the proxy at port 7775.

Create a vector store.

{
  "name": "product-documentation",
  "description": "Internal product docs",
  "metadata": {"team": "engineering"}
}

Response includes id, name, description, file_count, created_at.

List all vector stores.

Get a specific vector store.

Delete a vector store and all its files.

Add a text document. The document is chunked and embedded automatically.

{
  "content": "Full document text...",
  "filename": "architecture.md",
  "metadata": {"version": "3.0"}
}

List files in a vector store.

Semantic search over stored documents.

{
  "query": "How does the caching layer work?",
  "max_results": 5,
  "score_threshold": 0.7
}

// Response
{
  "results": [
    {"file_id": "vf_xyz", "filename": "architecture.md", "content": "...chunk...", "score": 0.91}
  ],
  "total": 1
}

RAG Pipeline API

Built on top of the vector store. Ingest documents with automatic chunking, then retrieve context for LLM augmentation.

Chunk a document, embed each chunk, and store in a named vector store.

{
  "content": "Full document text...",
  "vector_store_id": "vs_abc123",
  "filename": "report-q4.txt",
  "chunk_size": 512,
  "chunk_overlap": 64,
  "metadata": {"source": "internal"}
}

Response: { "store_id", "file_id", "chunks_created", "status": "completed" }

Embed a question, retrieve matching chunks, and optionally assemble a context string for injection into an LLM system prompt.

{
  "query": "What were the Q4 revenue figures?",
  "vector_store_id": "vs_abc123",
  "max_results": 5,
  "score_threshold": 0.0,
  "include_context": true
}

Response includes chunks[], context (concatenated string for prompt injection), and total.

Management API

Management API runs on port 7778.

Virtual Keys

All virtual key endpoints are on the management port (7778 / /api/enterprise/vkeys). Requires admin-level auth.

Create a virtual key. The token is returned once only — store it immediately. All fields except name and owner_id are optional.

// Request
{
  "name":       "team-alpha-prod",       // human label
  "owner_id":   "alice@example.com",     // user or team id
  "provider":   null,                    // null = any provider; or "openai","anthropic"…
  "budget_usd": 100.00,                  // USD ceiling per period; null = unlimited
  "period":     "monthly",               // daily | weekly | monthly | lifetime
  "tpm_limit":  100000,                  // tokens per minute; null = unlimited
  "rpm_limit":  60,                      // requests per minute; null = unlimited
  "tags": { "team": "alpha", "env": "prod" }
}

// Response — token shown ONCE
{
  "success": true,
  "token":   "sk-sf-a1b2c3d4e5f6...",  // 48-hex — deliver to your client now
  "key_id":  "sk-sf-a1b2c3d4e5f6...",
  "message": "Virtual key created. Store the token securely — it will not be shown again."
}

List all virtual keys. Tokens are never returned here — only metadata (name, owner, budget, active status).

{
  "success": true,
  "total": 3,
  "keys": [
    {
      "key_id":       "sk-sf-a1b2...",
      "name":         "team-alpha-prod",
      "owner_id":     "alice@example.com",
      "active":       true,
      "budget_usd":   100.00,
      "period":       "monthly",
      "period_start": "2026-03-01T00:00:00Z",
      "tpm_limit":    100000,
      "rpm_limit":    60,
      "created_at":   "2026-02-15T10:30:00Z",
      "last_used":    "2026-03-10T08:22:00Z",
      "tags": { "team": "alpha" }
    }
  ]
}

List all keys owned by a specific user or team id.

Get current period spend and remaining budget for a key.

{
  "success": true,
  "budget": {
    "key_id":         "sk-sf-a1b2...",
    "name":           "team-alpha-prod",
    "active":         true,
    "budget_usd":     100.00,
    "period":         "monthly",
    "spent_usd":      23.47,
    "remaining_usd":  76.53,
    "percent_used":   23.47,
    "last_used":      "2026-03-10T08:22:00Z"
  }
}

Reset the spend counter to zero and restart the budget period. No request body needed.

{ "success": true, "message": "Budget reset for key 'sk-sf-a1b2...'" }

Deactivate a key. Revoked keys return 429 Virtual key is revoked on all subsequent requests. The key metadata is retained in Redis for audit purposes.

{ "success": true, "message": "Virtual key 'sk-sf-a1b2...' revoked" }

Budget Exceeded Response

When a request is blocked by budget enforcement, the proxy returns:

HTTP/1.1 429 Too Many Requests
X-Smartflow-Budget-Exceeded: true
X-Smartflow-Budget-Remaining: 0.0000

{
  "error": {
    "message": "Budget limit exceeded: $100.00 spent of $100.00 (monthly budget)",
    "type":    "budget_exceeded",
    "code":    "insufficient_quota"
  },
  "spent_usd":     100.04,
  "remaining_usd": 0.0
}

Quick Start — cURL

# Create a key with a $50/month budget
curl -s https://YOUR_SMARTFLOW_HOST/api/enterprise/vkeys \
  -H "Authorization: Bearer YOUR_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-app-prod",
    "owner_id": "alice@example.com",
    "budget_usd": 50.0,
    "period": "monthly"
  }'

# Check spend
curl -s https://YOUR_SMARTFLOW_HOST/api/enterprise/vkeys/sk-sf-xxx/budget \
  -H "Authorization: Bearer YOUR_ADMIN_KEY"

# Revoke
curl -s -X POST https://YOUR_SMARTFLOW_HOST/api/enterprise/vkeys/sk-sf-xxx/revoke \
  -H "Authorization: Bearer YOUR_ADMIN_KEY"

Routing API

Current routing state: active provider, fallback chain, last failure.

{"provider": "openai", "duration_seconds": 600}

Audit Logs (VAS)

Retrieve VAS audit logs. Every request proxied through Smartflow produces a log entry including: timestamp, provider, model, prompt tokens, completion tokens, cost in USD, cache hit flag, compliance flags, user context, and latency.

Retrieve logs aggregated across multiple Smartflow instances via the hybrid bridge.

Analytics

Usage analytics: request volume, cost by provider, cache hit rate, top models, top users.

Compliance API

The Compliance API runs on port 7777. It provides ML-based content scanning, PII detection and redaction, and an adaptive learning loop that improves over time based on human feedback. The proxy integrates with this service on every request when pre/post-call scanning is enabled.

Rule-based compliance scan against configured policies.

{
  "content": "Text to scan",
  "policy": "enterprise_standard",
  "user_id": "user-123",
  "org_id": "acme"
}

// Response
{
  "has_violations": false,
  "compliance_score": 0.97,
  "risk_level": "low",
  "recommended_action": "Allow",
  "violations": [],
  "pii_detected": []
}

Maestro ML policy engine. Evaluates intent against your organization's policy documents — not keyword matching.

Response includes risk_score (0–1), risk_level, recommended_action (Allow / Flag / Block), violations, explanation.

Submit a correction to improve the ML model's future predictions.

{
  "scan_id": "scan-xyz",
  "correct_action": "Allow",
  "correct_risk_level": "low",
  "notes": "False positive — internal terminology"
}

Detect and redact PII from content. Returns the redacted string.

Policy Perfect API

The Policy Perfect API runs on port 7782. It manages the organization's compliance policy library — the source documents the Maestro ML engine reads when evaluating requests. Backed by PostgreSQL for durable policy storage.

Liveness check for the Policy Perfect service.

Aggregate counts for the current state of the policy library.

{
  "total_policies": 42,
  "total_presets": 8,
  "total_applications": 1204,
  "compliance_violations": 3
}

Policies

Policies are named compliance rules attached to scopes. The Maestro engine evaluates all active policies on every request.

Policy types:

List all active policies.

Create a policy.

{
  "name": "HIPAA PHI Protection",
  "description": "Prevent transmission of protected health information",
  "policy_type": "compliance",
  "content": "Do not include patient names, diagnoses, medical record numbers, or any PHI in AI responses.",
  "priority": 90,
  "applicable_providers": ["all"],
  "applicable_models": ["all"],
  "regulatory_framework": "HIPAA",
  "severity": "critical",
  "metadata": {
    "departments": ["clinical", "billing"],
    "ad_groups": ["clinicians", "admins"]
  }
}

Get a policy by ID.

Update a policy. All fields optional; only supplied fields are changed. Set "is_active": false to deactivate without deleting.

Delete a policy permanently.

Presets

Presets are named, ordered collections of policies. Assign a preset to a team, role, or virtual key instead of managing individual policies per scope.

List all presets. Each entry includes the preset metadata and its ordered policy list.

Create a preset.

{
  "name": "Healthcare Standard",
  "description": "Default policy set for all clinical staff",
  "use_case": "Clinical AI assistant",
  "policy_ids": ["pol_hipaa_phi", "pol_brand_tone", "pol_no_diagnosis"]
}

Policy order in policy_ids determines evaluation priority.

Get a preset and its full ordered policy list.

AI Document-to-Policy Generation

Upload a compliance document (PDF, DOCX, TXT — up to 50 MB). The service uses GPT-4o to extract structured policy suggestions automatically. Processing is asynchronous; poll for progress with the returned job ID.

Multipart form upload. Field name: file.

Content-Type: multipart/form-data

file=@hipaa-policy-handbook.pdf

Immediate response:

{
  "success": true,
  "job_id": "550e8400-e29b-41d4-a716-446655440000",
  "message": "Document processing started."
}

Poll for processing status. Status values: pending, processing, completed, failed.

{
  "success": true,
  "job": {
    "id": "550e8400-...",
    "filename": "hipaa-policy-handbook.pdf",
    "status": "processing",
    "progress_pct": 62,
    "created_at": "2026-02-19T10:00:00Z"
  }
}

Retrieve suggested policies once status is completed. Each suggestion includes a confidence score (0–1). Review suggestions and create live policies via POST /api/policies.

{
  "success": true,
  "job_id": "550e8400-...",
  "filename": "hipaa-policy-handbook.pdf",
  "total_policies": 7,
  "suggested_policies": [
    {
      "id": "sugg_abc",
      "name": "Minimum Necessary Standard",
      "type": "compliance",
      "content": "Limit PHI access and disclosure to the minimum necessary...",
      "priority": 85,
      "regulatory_framework": "HIPAA",
      "confidence": 0.94
    }
  ]
}

Alerting

Smartflow fires HTTP POST webhooks when threshold events occur. Configuration is via environment variables on the proxy server.

Configure any combination of webhook destinations:

SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
TEAMS_WEBHOOK_URL=https://outlook.office.com/webhook/...
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
SMARTFLOW_ALERTS_ENABLED=true

Alerts are fire-and-forget — they do not block the request that triggered them.

Observability

Returns 200 OK with {"status":"ok"} when the proxy process is running.

Returns 200 OK when Redis is connected and providers are reachable.

Prometheus-compatible metrics. Exposed metrics:

Python SDK

Installation

pip install smartflow-sdk

# or from source
pip install git+https://github.com/SRAGroupTX/SmartflowV3.git#subdirectory=sdk/python

Requirements: Python 3.10+, httpx >= 0.24

SmartflowClient

The primary async client.

class SmartflowClient(
    base_url: str,
    api_key: Optional[str] = None,
    timeout: float = 30.0,
    management_port: int = 7778,
    compliance_port: int = 7777,
    bridge_port: int = 3500,
)

from smartflow import SmartflowClient

async with SmartflowClient("https://smartflow.example.com", api_key="sk-sf-...") as sf:
    reply = await sf.chat("What is the capital of France?")
    print(reply)

Core AI Methods

chat()

async def chat(
    message: str,
    model: str = "gpt-4o",
    system_prompt: Optional[str] = None,
    temperature: float = 0.7,
    max_tokens: Optional[int] = None,
    **kwargs,
) -> str

Send a message, receive the reply as a plain string.

reply = await sf.chat("Summarise this in one sentence.", model="claude-sonnet-4-6")

chat_completions()

async def chat_completions(
    messages: List[Dict[str, str]],
    model: str = "gpt-4o",
    temperature: float = 0.7,
    max_tokens: Optional[int] = None,
    stream: bool = False,
    **kwargs,
) -> AIResponse

Full OpenAI-compatible completions. Returns an AIResponse object.

response = await sf.chat_completions(
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user",   "content": "What is 2 + 2?"},
    ],
    model="gpt-4o-mini", max_tokens=50,
)
print(response.content)
print(response.usage.total_tokens)

stream_chat()

async def stream_chat(message: str, model: str = "gpt-4o", ...) -> AsyncIterator[str]

Async generator that yields text delta strings as they stream.

async for chunk in sf.stream_chat("Tell me a story about a robot"):
    print(chunk, end="", flush=True)

embeddings()

async def embeddings(
    input: Union[str, List[str]],
    model: str = "text-embedding-3-small",
    encoding_format: str = "float",
    dimensions: Optional[int] = None,
    input_type: Optional[str] = None,
    **kwargs,
) -> Dict[str, Any]

result = await sf.embeddings("Hello world")
vector = result["data"][0]["embedding"]

# Cohere with input_type
result = await sf.embeddings(
    ["doc one", "doc two"],
    model="cohere/embed-english-v3.0",
    input_type="search_document",
)

# Reduce dimensions
result = await sf.embeddings("Hello", model="text-embedding-3-large", dimensions=256)

rerank()

result = await sf.rerank(
    "What is the return policy?",
    ["We accept returns within 30 days.", "Contact support@example.com."],
    top_n=1,
)

claude_message()

Send a message to Claude via the native Anthropic Messages API path (/anthropic/v1/messages). The proxy injects the API key automatically.

async def claude_message(
    message: str,
    model: str = "claude-sonnet-4-6",
    max_tokens: int = 1024,
    system: Optional[str] = None,
    anthropic_key: Optional[str] = None,
) -> str

reply = await sf.claude_message(
    "Summarise this contract in three bullet points.",
    model="claude-sonnet-4-6",
    system="You are a legal assistant.",
    max_tokens=512,
)

chatbot_query()

Query Smartflow's built-in system chatbot for natural-language operational queries about logs, cache stats, cost analysis, and system health.

result = await sf.chatbot_query("show me today's cache stats")
print(result["response"])

result = await sf.chatbot_query("which provider had the most errors this week?")

Provider Routing Examples

All chat and completion methods accept a model argument that determines which provider is used. No additional client-side configuration is required.

# OpenAI
reply = await sf.chat("Hello", model="gpt-4o")
reply = await sf.chat("Hello", model="gpt-4o-mini")
reply = await sf.chat("Hello", model="o3-mini")

# Anthropic — model-name heuristic, no prefix needed
reply = await sf.chat("Hello", model="claude-sonnet-4-6")
reply = await sf.chat("Hello", model="claude-3-opus-20240229")

# Google Gemini — model-name heuristic
reply = await sf.chat("Hello", model="gemini-1.5-pro")
reply = await sf.chat("Hello", model="gemini-2.0-flash")

# xAI Grok — explicit prefix
reply = await sf.chat("Hello", model="xai/grok-2-latest")

# Mistral
reply = await sf.chat("Hello", model="mistral/mistral-large-latest")

# Cohere
reply = await sf.chat("Hello", model="cohere/command-r-plus")

# Groq (fast Llama inference)
reply = await sf.chat("Hello", model="groq/llama-3.1-70b-versatile")

# OpenRouter (200+ models through one key)
reply = await sf.chat("Hello", model="openrouter/meta-llama/llama-3.1-405b")

# Local Ollama
reply = await sf.chat("Hello", model="ollama/llama3.2")

# Azure OpenAI — deployment name as suffix
reply = await sf.chat("Hello", model="azure/my-gpt4o-deployment")

# Native Anthropic path (uses /anthropic/v1/messages directly)
reply = await sf.claude_message("Hello", model="claude-sonnet-4-6")

MCP Tool Invocation via SDK

The Python SDK does not expose dedicated MCP methods. MCP tool calls are made as direct HTTP requests to the proxy. Use httpx or any HTTP client. All requests go through the proxy's authentication, access control, and cost tracking.

import httpx

async with httpx.AsyncClient() as client:
    response = await client.post(
        "https://smartflow.example.com/github-tools/mcp/",
        headers={
            "Authorization": "Bearer sk-sf-...",
            "Content-Type": "application/json",
        },
        json={
            "jsonrpc": "2.0", "id": 1, "method": "tools/call",
            "params": {
                "name": "create_issue",
                "arguments": {
                    "repo": "my-org/my-repo",
                    "title": "Bug: login fails on mobile",
                    "body": "Steps to reproduce..."
                }
            }
        }
    )
    print(response.json()["result"]["content"])

Discover available tools with a natural-language search:

async with httpx.AsyncClient() as client:
    r = await client.get(
        "https://smartflow.example.com/api/mcp/tools/search",
        params={"q": "create github issue", "k": 3},
        headers={"Authorization": "Bearer sk-sf-..."},
    )
    for tool in r.json()["results"]:
        print(f"{tool['server_id']}.{tool['name']}: {tool['description']}")

A2A Agent Invocation via SDK

A2A tasks are sent as HTTP POST requests to the proxy. The proxy forwards to the registered agent, logs the exchange, and returns the result. Include x-a2a-trace-id to correlate across multi-agent workflows.

import httpx

async with httpx.AsyncClient() as client:
    response = await client.post(
        "https://smartflow.example.com/a2a/summarizer-agent",
        headers={
            "Authorization": "Bearer sk-sf-...",
            "Content-Type": "application/json",
            "x-a2a-trace-id": "trace-abc-123",
        },
        json={
            "id": "task-uuid-001",
            "message": {
                "role": "user",
                "parts": [{"type": "text", "text": "Summarise the Q4 earnings report."}]
            }
        }
    )
    print(response.json()["result"]["parts"][0]["text"])

# Retrieve the agent's capability card
r = await client.get(
    "https://smartflow.example.com/a2a/summarizer-agent/.well-known/agent.json",
    headers={"Authorization": "Bearer sk-sf-..."},
)
print(r.json()["capabilities"])

Audio and Image Methods

audio_transcription()

with open("recording.mp3", "rb") as f:
    result = await sf.audio_transcription(f, model="whisper-1")
print(result["text"])

# Groq Whisper (faster, same format)
with open("recording.mp3", "rb") as f:
    result = await sf.audio_transcription(f, model="groq/whisper-large-v3")

text_to_speech()

audio = await sf.text_to_speech("Hello, this is Smartflow.", voice="nova")
with open("output.mp3", "wb") as f:
    f.write(audio)

image_generation()

result = await sf.image_generation(
    "A mountain landscape at dawn",
    model="dall-e-3", size="1792x1024", quality="hd",
)
print(result["data"][0]["url"])

Compliance Methods

check_compliance()

result = await sf.check_compliance("User message text", policy="hipaa")
if result.has_violations:
    print(result.violations)

intelligent_scan()

ML-based scan combining regex, embedding similarity, behavioral analysis, and organization baselines.

async def intelligent_scan(
    content: str,
    user_id: Optional[str] = None,
    org_id: Optional[str] = None,
    context: Optional[str] = None,
) -> IntelligentScanResult

result = await sf.intelligent_scan(
    "My SSN is 123-45-6789",
    user_id="user-123",
    org_id="acme-corp",
    context="customer_support",
)
print(f"{result.risk_level}: {result.recommended_action}")
# "high: Block"

redact_pii()

clean = await sf.redact_pii("My SSN is 123-45-6789, email me at john@example.com")
# "My SSN is [SSN], email me at [EMAIL]"

submit_compliance_feedback()

Submit a true/false-positive correction. Used to retrain the ML model.

async def submit_compliance_feedback(
    scan_id: str,
    is_false_positive: bool,
    user_id: Optional[str] = None,
    notes: Optional[str] = None,
) -> Dict[str, Any]

await sf.submit_compliance_feedback(
    scan_id="scan-xyz",
    is_false_positive=True,
    notes="This was a test phone number, not real PII",
)

ML Learning and Org Baseline Methods

Monitoring Methods

get_cache_stats()

stats = await sf.get_cache_stats()
print(f"Hit rate:     {stats.hit_rate:.1%}")
print(f"Tokens saved: {stats.tokens_saved:,}")
print(f"Cost saved:   ${stats.cost_saved_usd:.4f}")
print(f"L1/L2/L3:     {stats.l1_hits} / {stats.l2_hits} / {stats.l3_hits}")

health_comprehensive()

h = await sf.health_comprehensive()
print(h.overall_status)       # "healthy"
print(h.redis_connected)      # True
print(h.providers_available)  # ["openai", "anthropic", "google"]

Other monitoring methods

SmartflowAgent

Stateful agent with conversation memory and per-message compliance scanning.

async with SmartflowClient("https://smartflow.example.com", api_key="sk-...") as sf:
    agent = SmartflowAgent(
        client=sf,
        name="SupportBot",
        model="gpt-4o",
        system_prompt="You are a helpful customer support agent.",
        user_id="user-123",
        org_id="acme",
    )
    r1 = await agent.chat("How do I reset my password?")
    r2 = await agent.chat("What if I forgot my email too?")

    print(agent.message_count)
    agent.clear_history()

SmartflowWorkflow

Chain AI operations with branching and error handling.

workflow = SmartflowWorkflow(client, name="TicketFlow")

workflow \
    .add_step("classify", action="chat",
              config={"prompt": "Classify this ticket: {input}", "model": "gpt-4o-mini"}) \
    .add_step("check", action="compliance_check",
              config={"content": "{output}"}) \
    .add_step("route", action="condition",
              config={"field": "output", "cases": {"billing": "billing_step"}, "default": "general_step"})

result = await workflow.execute({"input": ticket_text})
print(result.output)
print(result.steps_executed)
print(result.execution_time_ms)

SyncSmartflowClient

Synchronous wrapper for scripts and Jupyter notebooks. Every async method is available without await.

from smartflow import SyncSmartflowClient

sf = SyncSmartflowClient("https://smartflow.example.com", api_key="sk-...")

reply      = sf.chat("Hello!")
emb        = sf.embeddings("Hello", model="text-embedding-3-small")
img        = sf.image_generation("A sunset", model="dall-e-3")
transcript = sf.audio_transcription(open("audio.mp3", "rb"))
audio      = sf.text_to_speech("Hello!", voice="nova")
ranked     = sf.rerank("What is the return policy?", ["doc1", "doc2"])

In Jupyter with an existing event loop: pip install nest_asyncio then nest_asyncio.apply().

OpenAI Drop-in Replacement

Any code targeting the OpenAI API works by pointing base_url at Smartflow. MetaCache, compliance scanning, VAS logging, and routing apply transparently.

from openai import OpenAI

client = OpenAI(
    api_key="sk-sf-your-virtual-key",
    base_url="https://smartflow.example.com/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

Response Types

AIResponse

CacheStats

ComplianceResult

IntelligentScanResult

Response Headers

Every proxied response includes these headers:

Environment Variables

Set on the Smartflow server; not used in client code.

Provider Keys

Feature Flags and Ports

Error Reference

HTTP Status Codes

SDK Exceptions

from smartflow import ComplianceError, RateLimitError
import asyncio

try:
    result = await sf.chat("sensitive message")
except ComplianceError as e:
    print(f"Blocked by policy: {e}")
except RateLimitError:
    await asyncio.sleep(60)
    # retry

Changelog

v3.0 (proxy) / v0.3.0 (SDK) — 2026

New in the proxy:

• Vector Store API (/v1/vector_stores/*) — Redis-backed, no external vector database required
• RAG Pipeline API (/v1/rag/ingest, /v1/rag/query) — document chunking, embedding, context retrieval
• A2A Agent Gateway (/a2a/*) — A2A protocol for inter-agent orchestration
• Webhook alerting — Slack, Teams, Discord for budget, failure, and latency events
• Model-name heuristic routing — claude-*, gemini-*, gpt-* detected automatically
• Anthropic API key injection for /anthropic/* passthrough
• Cost-based and latency-based routing strategies
• Prometheus metrics endpoint (/metrics)
• MCP access control — allowed_tools, disallowed_tools, guardrail_mode per server
• MCP cost tracking via Redis HINCRBYFLOAT

New in the SDK:

• image_generation() — multi-provider image generation
• audio_transcription() — multipart audio, Groq/Deepgram/Fireworks routing
• text_to_speech() — returns raw audio bytes
• stream_chat() — async SSE iterator
• rerank() — Cohere-compatible document reranking
• Extended embeddings() with encoding_format, dimensions, input_type

v2.0 (proxy) / v0.2.0 (SDK)

• MCP gateway with server registry, catalog, OAuth flow
• SmartflowAgent with compliance scanning and conversation memory
• SmartflowWorkflow for multi-step AI pipelines
• Maestro ML policy engine (intelligent compliance)

v1.0 (proxy) / v0.1.0 (SDK)

• OpenAI-compatible proxy, virtual keys, 3-tier semantic cache
• Initial SDK: chat, chat_completions, embeddings
• VAS audit logging, SyncSmartflowClient