# Provably

> Provably makes agentic workflows reliable. SourceryKit, its Python SDK (`sourcerykit` on PyPI, import as `provably`), adds a deterministic eval layer to any Python agent. It monkey-patches outbound HTTP calls (requests, httpx, aiohttp), records every interaction, enforces a trusted endpoint registry, and lets receiving agents evaluate handoff claims against cryptographically proven query records — so multi-agent workflows stay hallucination-free without model self-evaluation.

## Overview

- **SDK:** SourceryKit
- **Distribution:** `sourcerykit` (PyPI publish pending), import as `provably`
- **Python:** 3.11+
- **License:** Proprietary
- **Status:** v0.2
- **Source layout:** `src/provably/`
- **Website:** https://provably.ai
- **GitHub:** https://github.com/ProvablyAI/sourcerykit

Provably is built by a team of cryptographers and software engineers with deep expertise in cryptography, distributed systems, and databases. Fully remote, with anchors in Zurich, Warsaw, Northern Italy, and Lithuania.

## How It Works

SourceryKit operates through four stages in order:

1. **Intercept + Police** — Every outbound `requests` / `httpx` / `aiohttp` call goes through the SDK's monkey-patched HTTP path. Before the request leaves the process, the URL is checked against the `trusted_endpoints` table. If the URL is not registered, the call is killed with `RuntimeError("BLOCKED: ...")` and never reaches the network.

2. **Capture + Store** — If the endpoint is trusted, the request goes out, the response is captured (status + headers + raw body), canonicalized, and inserted into `provably_intercepts`. The agent only sees the response after the row is written.

3. **Hand off** — When an agent finishes its work, it builds a typed `HandoffPayload` (one `HandoffClaim` per external call, describing what the agent claims about that response) and ships it to the next agent or service via `post_handoff(...)`.

4. **Eval** — The receiving service runs `evaluate_handoff(payload)` (the SDK's verifiable-guardrails check — not to be confused with proof verification). For each claim, the evaluator pulls the corresponding query record from the Provably backend and runs one of four deterministic comparisons; the result is `PASS`, `CAUGHT`, or `ERROR` per claim.

Nothing in this loop relies on a model self-evaluating its own output.

## The Five Pillars

### init

```python
import provably

# One-call bootstrap (recommended)
provably.configure_indexing(True)   # bootstrap + interceptor + enable
provably.configure_indexing(False)  # interceptor only, recording off

# Step-by-step alternative
provably.initialize_runtime()   # one-time bootstrap; idempotent per process
provably.init_interceptor()     # install monkey-patches
provably.enable()               # turn recording on
```

`initialize_runtime()` registers a Provably middleware, onboards the configured Postgres database, ensures the `provably_intercepts` collection exists, and warms an in-memory cache with the integration API key.

### intercept

```python
provably.enable()                   # default after init_interceptor()
provably.disable()                  # stop recording (patch stays installed)
provably.is_enabled()               # bool

provably.set_interceptor_context(   # tag the next intercept rows
    agent_id="cluster_a",
    action_name="lookup_patient",
    intercept_index=0,
)
```

The interceptor records every successful `requests.get/post` and `httpx.get/post` into `provably_intercepts`. The original wire response is stored first; any hook only affects the object returned to application code, not the stored row.

### handoff

```python
from provably import HandoffPayload, HandoffClaim, post_handoff

payload = HandoffPayload(
    provably_org_id="my-org",
    integration_api_key="key",
    claims=[HandoffClaim(action_name="get", claimed_value=..., query_record_id="qr_1")],
)
post_handoff("https://my-eval-service.example", payload)
```

Convenience builders: `build_handoff_payload` assembles a `HandoffPayload` automatically from the interceptor's in-memory state. `claim_contract` generates system-prompt text that tells an LLM how to emit the correct `HandoffClaim` JSON shape. `default_instructions` and `field_descriptions` provide ready-made prompt text for receiving agents.

### eval

```python
from provably import evaluate_handoff

result = evaluate_handoff(payload, provably_base_url="https://api.provably.ai")
# {"outcome": "PASS" | "CAUGHT" | "ERROR", "per_claim": [...], "errors": [...]}
```

Outcome semantics:
- **PASS** — every claim's content matched its proven indexed value and every proof verified.
- **CAUGHT** — at least one claim disagreed with the indexed value or a proof failed.
- **ERROR** — the evaluator could not run (missing config, backend unreachable, transient error). Not evidence of tampering — the system was unhealthy, not the agent.

### trusted_endpoints

```python
from provably import is_trusted_endpoint, ensure_trusted_endpoints_table

conn = psycopg2.connect("...")
ensure_trusted_endpoints_table(conn)
ok = is_trusted_endpoint("https://api.example.com/v1/data", "my-org", conn)
```

The registry is a Postgres table. URLs are normalized (lowercase scheme + host, default ports collapsed, trailing slash dropped). Supports path-pattern entries with `{name}` (one segment) and `{name:path}` (subtree) placeholders.

## Verification Modes

| Mode | Comparison |
|------|-----------|
| `verbatim` | Canonical-JSON equality between `claimed_value` and the indexed payload (or `json_path` slice) |
| `field_extraction` | Equality on the value at `json_path` only |
| `schema_type` | `claimed_value` is ignored; the value at `json_path` is validated against `expected_json_schema` |
| `range_threshold` | Numeric `claimed_value` must equal the indexed numeric and lie in `[range_min, range_max]` |

## Framework Coverage

Transport patches cover `requests` (module-level + Session.send), `httpx` (module-level + Client.send + AsyncClient.send), and `aiohttp` (ClientSession._request, soft dependency).

Supported agent/LLM frameworks (all via transport-level patching):
- OpenAI SDK (httpx)
- Anthropic SDK (httpx)
- Pydantic AI (delegates to AsyncOpenAI / AsyncAnthropic)
- LangChain and LangGraph (delegates to provider SDKs)
- LlamaIndex (httpx via OpenAI SDK)
- AutoGen (AsyncOpenAI)
- Haystack (httpx)
- Phidata / Agno (AsyncOpenAI / httpx)
- OpenAI Agents SDK (httpx)
- Google GenAI (httpx + optional aiohttp)
- LiteLLM (aiohttp)
- DSPy (via LiteLLM)
- smolagents (OpenAI SDK / HF / LiteLLM)
- CrewAI (OpenAI/Anthropic ✅, LiteLLM ✅, Bedrock ❌)

Not yet supported: AWS Strands (boto3/botocore), MCP servers, in-process LLMs, gRPC, websockets.

## Configuration

| Variable | Used by | Required |
|----------|---------|----------|
| `PROVABLY_API_KEY` | `initialize_runtime`, integration cache | yes |
| `PROVABLY_ORG_ID` | `initialize_runtime`, intercept allow-list | yes |
| `PROVABLY_RUST_BE_URL` | `initialize_runtime`, evaluator | yes |
| `POSTGRES_URL` | intercept storage, trusted endpoints, handoff preprocess | yes |
| `PROVABLY_APP_UI_URL` | optional UI deep-links | no |
| `PROVABLY_QUERY_RESOLVE_MAX_WAIT_S` | max seconds to wait for a query record (default 15) | no |

### Getting Started

1. Sign up at app.provably.ai
2. Create an organisation — its id goes in `PROVABLY_ORG_ID`
3. In the left-side menu, go to Integrations and create one — the generated key is your `PROVABLY_API_KEY`

## Quick Start

```python
import provably
import requests

# One-call bootstrap
provably.configure_indexing(True)

response = requests.get("https://my-trusted-endpoint.example/data")
record = response.json()

payload = provably.HandoffPayload(
    provably_org_id="my-org",
    integration_api_key="...",
    task="discharge_summary",
    claims=[
        provably.HandoffClaim(
            action_name="lookup_patient",
            claimed_value=record,
            query_record_id="qr_123",
        ),
    ],
)
provably.post_handoff("https://my-eval-service.example", payload)
```

Eval side:

```python
import provably

result = provably.evaluate_handoff(
    payload,
    provably_base_url="https://api.provably.ai",
)
assert result["outcome"] in ("PASS", "CAUGHT", "ERROR")
```

## Public API

All public symbols are re-exported from the top-level `provably` namespace:

```python
from provably import (
    # init
    initialize_runtime, configure_indexing,
    # intercept
    init_interceptor, enable, disable, is_enabled,
    set_interceptor_context, set_intercept_body_hook,
    set_intercept_url_allowlist, take_last_intercept_row_id,
    # handoff types
    HandoffPayload, HandoffClaim, HandoffProofAction, HandoffProofBundle,
    BenchmarkRow, Outcome, VerificationMode,
    # handoff transport
    post_handoff,
    # handoff builders
    build_handoff_payload, DEFAULT_HANDOFF_TASK,
    claim_contract, default_instructions, field_descriptions,
    # eval
    evaluate_handoff, extract_indexed_from_query_record,
    outcome_from_trace, aggregate_outcome,
    # trusted endpoints
    is_trusted_endpoint, list_trusted_endpoints,
    check_claim_endpoints_are_trusted, normalize_url_for_trust,
    ensure_trusted_endpoints_table,
)
```

## Where Things Live

| Component | Hosted by | Notes |
|-----------|-----------|-------|
| `trusted_endpoints` table | You — your Postgres | SDK ships the schema and CRUD helpers |
| `provably_intercepts` table | You — same Postgres | Append-only, one row per outbound HTTP call |
| Eval service | You — any HTTP service calling `evaluate_handoff()` | SDK provides the function; you host it |
| Provably query record | Provably — fetched over HTTPS | Source of truth for claim comparison |

## Data Admin Platform

Provably also provides a Data Admin platform (self-hosted or SaaS) at app.provably.ai for managing data sources, verifiable data catalogues, network management, and access controls.

Features: data management (connect databases, create trusted data), verifiable data catalogues (discover and query provable datasets), network management (form and join data collaboration networks), access management (fine-grained permissions), verifier network (neutral integrity layer), and API/MCP readiness (serve verifiable data to apps and AI models).

### Provably V2 Middleware

V2 enables SQL queries on databases or tabular data to return verifiable results. Supports privacy-preserving aggregate queries that return results without revealing underlying data.

Core features: general SQL and aggregate analytics queries, SUM/COUNT/MIN/MAX aggregations, PostgreSQL support (SQLite and MySQL on request), custom blockchain integrations, and proofs under 5kb.

## Use Cases

**Customer Service Agents**: Order status, balances, plans, refunds, and support handoffs — all verifiable across systems of record.

**Healthcare Workflows**: Patient records, medication checks, discharge updates, and referrals verified before downstream agents act.

**Financial Operations**: Payments, account balances, approval states, reconciliations, and audit trails across ERP, banking, and payment APIs.

## Research

### QEDB Paper: Expressive and Modular Verifiable Databases

QEDB is a verifiable database infrastructure that compiles SQL queries directly over tabular data, producing fast, small proofs of correctness and integrity under 5kb — no SNARKs or specialised circuits required. It supports joins, aggregates, and snapshots, and sits as a third path beyond the two dominant lineages of verifiable databases: authenticated data structures and SNARK-based systems. The paper has been accepted into CCS 2026 (Conference on Computer and Communications Security). The practical outcome: remote machine-verifiable audit trails, where downstream systems can verify query results without re-running them or trusting the source.

ePrint: https://eprint.iacr.org/2025/1408
Full paper: https://provably.ai/papers/qedb-paper.pdf

## Frequently Asked Questions

### What can Provably prove and what failures can it stop?

Provably proves the endpoint, request, and result of every agent interaction with deterministic systems — application APIs, databases, developer tools, and internal services exposed through MCP servers, CLIs, APIs, or SDKs. That covers data retrievals, status checks, state updates, action payloads, handoffs, and logs. It stops data hallucinations, poisoned outputs, incomplete handoffs, and incorrect summaries built on unverifiable source data.

### How is Provably different from observability, evals, and LLM judges?

Provably is complementary to observability, evals, and LLM judges. Observability tools require trace collection across teams and networks, high-effort classification of agent outputs, and mostly cover subjective reasoning errors that need human review — and even then, the success rate of failure attribution is around 29%.

Provably takes a different path: agents prove the source, request, and result behind their answers deterministically and cryptographically, in real time, with no extra traces. Use observability for reasoning and model behavior; use Provably to verify source-backed data before agents act.

### How fast is setup and what does it work with?

Provably ships as a Python SDK that monkey-patches outbound HTTP calls (requests, httpx, aiohttp) — no trace collection or code rewrites required. It works with OpenAI Agents SDK, Anthropic SDK, LangChain, LangGraph, LlamaIndex, AutoGen, Pydantic AI, CrewAI, Google GenAI, LiteLLM, and DSPy via transport-level patching.

Downstream services verify proofs through a public verification API; the sender only passes proof metadata alongside the answer. Agents can also deploy Provably themselves using the provided templates and skills.

### Does Provably add latency or cost?

Verification runs in milliseconds and proofs are under 5kb. Downstream agents verify proof metadata without re-querying the source system, which cuts repeated API calls, latency, and permission overhead.

Provably is delivered as SaaS, but provers and verifiers can also run on-prem next to the source system, reducing latency further.

### How does the technology work?

Provably is a verifiable database middleware. It compiles SQL queries directly over tabular data and produces verifiable SQL proofs of data and query integrity — proofs are under 5kb, with no SNARKs or specialised circuits required. The underlying research, qedb: Expressive and Modular Verifiable Databases (without SNARKs) (https://eprint.iacr.org/2025/1408), has been accepted into CCS 2026 (https://www.sigsac.org/ccs/CCS2026/). You can use Provably as a verifiable registry, a verifiable log, or a middleware layer for verifiable data collaboration. Traditional ledgers and audit logs were built for human audits; Provably moves the industry toward remote machine-verifiable audit trails.

## How Provably Differs from Other Approaches

### vs. observability and tracing tools

Observability and tracing tools collect spans, logs, and metrics across an agent's execution path to help engineers debug after the fact. They require trace collection across teams and networks plus high-effort classification of agent outputs, and they mostly surface subjective reasoning errors that need human review — failure-attribution success rate sits around 29% even with that overhead. Provably is complementary: it adds a deterministic, cryptographic layer that verifies the source, request, and result of every external call in real time, with no trace collection needed.

### vs. evals frameworks

Evals frameworks score agent or model outputs against benchmarks, judge sets, or held-out test cases — useful for measuring quality during development. They mostly focus on probabilistic, subjective dimensions of reasoning. Provably operates at a different layer: it verifies the external data that flows into an agent's decisions, deterministically and cryptographically, before downstream agents act on it.

### vs. LLM-as-judge approaches

LLM-as-judge approaches use another model to evaluate the correctness or quality of an agent's output. They rely on the judging model's own probabilistic reasoning, which inherits hallucination risk and cannot independently confirm whether the underlying data was real. Provably removes the model from the verification path entirely: each claim is checked against a cryptographically proven query record, so the outcome (PASS, CAUGHT, or ERROR) depends on math, not another model's opinion.

### vs. existing guardrail libraries

Guardrail libraries typically intercept prompts and outputs to enforce content policies, format constraints, or safety rules. They check what the agent says, not what data the agent actually retrieved. Provably enforces a trusted-endpoint registry at the transport layer — outbound HTTP calls to unregistered URLs are blocked before they leave the process — and produces verifiable claims about the source, request, and result that any downstream service can independently check.

## Links

- Website: https://provably.ai
- Machine view (LLM-friendly homepage): https://provably.ai/ai
- Data Admin: https://app.provably.ai
- Developer Docs: https://provably.ai/docs
- API Reference: https://api.provably.ai/api/v1/swagger/
- GitHub: https://github.com/ProvablyAI/sourcerykit
- Blog: https://provably.ai/blogs
- Research: https://provably.ai/research
- Twitter/X: https://x.com/getprovably
- LinkedIn: https://www.linkedin.com/company/provably
- Telegram: https://t.me/ProvablyAI