Verify agent tool calls before errors spread
The state of the art Python SDK for runtime verification of agent tool calling. Run production grade agentic workflows with greater accuracy and reliability.
Setup — two options:
Harness: Read AGENTS.md at https://github.com/ProvablyAI/sourcerykit, summarize what it covers and wait for my instructions.
No need to fetch AGENTS.md separately — its full text is on this page, in the [Agent onboarding](#agent-onboarding) section below.
Python: $ pip install sourcerykit
SDK source: [SourceryKit](https://github.com/ProvablyAI/sourcerykit)
01 Prove the tool call happened
Agents create verifiable handoffs that answers came from the right source and weren't changed — increasing reliability and security.
02 100% error detection rate
Verifiable guardrails detect hallucinations, or wrong-source data without re-querying source systems. Perform retries. Heal workflows.
03 Ship reliable agents in minutes
Add the SourceryKit SDK in minutes and verify across agents, tools, and networks without complex tracing or cross-team setup.
Trusted By
Performance: From days to minutes
Traditional observability needs failures, traces, and classification before teams can improve agents. SourceryKit verifies handoffs in real time, catching errors earlier and accelerating learning.
| Approach | Result |
|---|---|
| SourceryKit + Observability tools | 3x times faster to build reliable agents. |
| Observability tools (Galileo, Arize Phoenix, LangFuse, LangSmith) | — |
- Deploy Faster — Deploy verifiable guardrails without waiting for complex trace setup.
- Catch failures earlier — Catch errors during execution, before users or downstream agents see them.
- Improve agents faster — Use verification failure data to improve evals, and agent behaviour.
Least privilege verification: prove answers without reopening the source
Production agents increasingly run with scoped access. One agent may query CRM, billing, databases, or internal tools, while downstream agents and orchestrators may not be allowed to access those same systems again.
SourceryKit lets source-facing agents attach proof metadata to tool-backed answers, so later agents, orchestrators, evaluators, or auditors can verify the result without re-querying restricted source systems or copying raw sensitive data into traces.
- Preserve least privilege access boundaries across multi-agent workflows.
- Reduce repeated source-system calls and duplicated permission checks.
- Verify deterministic claims with compact proofs instead of asking another model to judge plausibility.
Adversarial verification and LLM judges can spend more tokens reviewing an answer, but they still cannot prove whether a value came from the original source unless the source evidence is included in context. SourceryKit verifies deterministic claims against proof objects, reducing repeated prompts, repeated tool calls, and source re-querying while increasing the effectiveness of verification.
Integrations: Built for your existing tools
Connect SourceryKit to your existing stack and every agent interaction becomes verifiable. Every API call, database query, and MCP server interaction is intercepted and turned into a proof downstream agents can verify in milliseconds.
Anthropic, AWS, Cursor, GitHub, Gmail, HubSpot, Jira, Linear, MongoDB, Notion, Postgres, Redis, Salesforce, Slack, Snowflake, Stripe, Supabase, Vercel, VS Code, Sentry, Claude, Google Drive, 1000+ more
Use cases: Where errors are not an option
Customer service agents
Customers get answers they can actually trust. Order status, balances, plans, refunds, and support handoffs - all verifiable across systems of record.
Healthcare workflows
Protect complex care workflows that span multiple teams and networks. Patient records, medication checks, discharge updates, and referrals all verified before downstream agents act.
Financial operations
Power finance workflows with verifiable data. Payments, account balances, approval states, reconciliations and audit trails across ERP, banking, and payment APIs.
Accepted into CCS 2026: QEDB: Expressive and Modular Verifiable Databases
SourceryKit runs on QEDB, a verifiable database that proves query results are correct without SNARKs or circuits. Proofs stay around 1KB no matter the data size.
Tools: The verifiable data stack
- SDKs & MCP ready — Python SDKs and MCP support to get started quickly. More languages coming soon.
- Manage deployments — Deploy and monitor integrations across endpoints and workflows.
- Access management — Manage proving and verification rights across teams and organisation boundaries.
- Observability — Track proof activity and failed verifications. Use the data to learn faster.
- Verifiable data — Create verifiable events, logs, registries, and data responses with cryptographic proof.
- SaaS or on-prem — Get started in minutes with SaaS. Contact us for our upcoming on-prem release.
Frequently asked questions
What can SourceryKit prove and what failures can it stop?
SourceryKit verifies agent interactions with deterministic systems, including APIs, databases, RAG and retrieval pipelines, MCP servers, CLIs, SDKs, developer tools, and internal services. For each verified interaction, Provably proves the source endpoint, request, and result behind an agent answer or handoff. It helps stop data hallucinations, wrong source data, poisoned tool outputs, unsupported RAG answers, incomplete handoffs, and incorrect summaries based on unverifiable source data, so agents and orchestrators can block, retry, or repair before bad data reaches the next step.
How is SourceryKit different from observability, evals, and LLM judges?
SourceryKit is complementary to observability, evals, and LLM judges. Observability requires trace collection across teams and networks, plus high-effort review to find and classify errors in agent outputs. LLM judges can miss plausible-looking errors; in one benchmark, judges found only 29% of errors in logs because they could not reliably detect mistakes when the answer looked reasonable. SourceryKit lets agents prove their answers, so downstream agents and orchestrators can deterministically verify the source endpoint, request, and result without re-querying the original system. This verification data can feed observability and eval pipelines, helping teams build reliable agents faster.
How fast is setup and what does it work with?
SourceryKit ships as a Python SDK that can be added to agent workflows in minutes, without trace collection, large code rewrites, or framework-specific integrations. It works with production agent stacks including OpenAI Agents SDK, Anthropic SDK, LangChain, LangGraph, LlamaIndex, AutoGen, Pydantic AI, CrewAI, Google GenAI, LiteLLM, DSPy, MCP servers, internal APIs, databases, and RAG pipelines. Downstream agents and services verify proofs through a public verification API; the sender only passes proof metadata alongside the answer or handoff.
Does SourceryKit add latency or cost?
Proofs are generated in about 50ms and verified in about 100ms. In practice, when using Provably's hosted prover and verifier, proof generation takes around 1 second and verification takes around 1 second. Proofs and verifications run in parallel, so an agent can perform multiple proofs or verifications in the same workflow without latency increasing linearly with each call. Enterprise teams can also run the prover and verifier inside their own infrastructure for lower latency and stricter deployment requirements. [Talk to us](mailto:[email protected]) if you want to do this.
Do you use a blockchain or cryptographic signatures?
No. SourceryKit is off-chain by design, which makes it simpler to deploy in enterprise settings. Blockchains are too slow and expensive for high-volume, low-latency agent workflows. Signatures only prove that a message was signed, and are not well suited to multi-hop agent or network verification. SourceryKit uses [QEDB](https://provably.ai/papers/qedb-paper.pdf), a verifiable database built for agents. QEDB proves remote data and query results are correct and complete, with general SQL support. Importantly, it produces compact proof files around 1KB, with proof generation and verification fast enough for production agent workflows.
Haven't found your answer here? [Email us](mailto:[email protected])
Newsletter
Blogs
Agent onboarding
Read AGENTS.md at https://github.com/ProvablyAI/sourcerykit, summarize what it covers and wait for my instructions.
Full text of [AGENTS.md](https://github.com/ProvablyAI/sourcerykit/blob/main/AGENTS.md) follows, mirrored from the sourcerykit repository:
AGENTS.md
SourceryKit is the Python SDK for Provably: verifiable guardrails for AI agents — it records outbound HTTP calls, enforces endpoint policies, and checks an agent's claims against what those calls actually returned, so a hallucinated value is caught instead of shipped.
[!IMPORTANT] Setup is scriptable — prefer the non-interactive
sourcerykit initCLI over hand-writing config. Exactly one step needs a human: clicking the email-verification link. Full steps: docs/onboarding.md.
Where to go next
| Your situation | Do this |
|---|---|
| First time — no Provably credentials yet | Run the sourcerykit init setup (see the note above) → docs/onboarding.md. |
| Integrating SourceryKit into an agent | Open the closest cookbook (below), mirror it, then swap in your own tool and claims. |
| Want to see one full run first | docs/example.md — end-to-end walkthrough. |
Got an unexpected CAUGHT or ERROR | Read the outcomes below, then docs/handoff.md. |
| Need a signature, type, or CLI flag | docs/src/api.md · docs/cli.md |
| Record or inspect outbound HTTP calls | docs/intercept.md |
| Allow-list outbound endpoints | docs/trusted-endpoints.md |
| Understand how the pieces fit | docs/architecture.md |
Migrate from the old provably SDK | docs/migrations/v1_0/v1_0.md |
Cookbooks are the ground truth — mirror one, never hand-roll the claim. Everything else is supporting docs; load only what a task needs.
Cookbooks (runnable examples)
Each is a full runnable agent: python agent_run.py for a PASS; add --tamper to corrupt
a claim and watch evaluate_handoff return CAUGHT. Every claimed value carries a
sourcerykit_ref (copied from the tool's output) that maps it to the exact recorded call —
so the same tool can be called many times, and separate agents can each own a stage.
Framework not listed? The flow is identical everywhere — only the structured-output binding is framework-specific. Copy the closest cookbook and change just that binding; each cookbook's README covers its own wiring.
Single-agent — one agent fetches, claims, and verifies (weather):
| Cookbook | Framework | What you'll find |
|---|---|---|
| openai_agents | OpenAI Agents SDK | Structured output via output_type=SourceryKitAgentResponse |
| claude_agent | Claude Agent SDK | Structured output via output_format json_schema |
| langchain_agent | LangChain create_agent | Structured output via response_format=, claims read from result["structured_response"] |
Multi-agent / multi-tool — producer agents build claims; a separate verifier evaluates them:
| Cookbook | Framework | Pattern |
|---|---|---|
| openai_agents_multi_agent | OpenAI Agents SDK | Orchestrator → specialists → verify (customer support) |
| crewai_multi_agent | CrewAI | Specialist crew → build → evaluate → remediate (invoice audit) |
| langgraph_multi_agent | LangGraph | Fetcher → evaluator → healer on CAUGHT (flights) |
| claude_agent_multi_tool | Claude Agent SDK | Same tool called twice; sourcerykit_ref maps each claim (weather) |
The flow at a glance
bootstrap_system() # init: schema, handshake, HTTP interceptor — call first, once
insert_trusted_endpoint(url) # allow-list each outbound endpoint
async_intercept_context(...) # wrap the tool's HTTP call — records it
SourceryKitAgentResponse # the agent's structured output (reasoning + claimed_values)
build_handoff_payload(...) # compile claims; intercept_agent_id must match the agent_id above
evaluate_handoff(payload) # -> {"outcome": "PASS" | "CAUGHT" | "ERROR", "per_claim": [...]}
Outcomes:
PASS— every claim matched the recorded data.CAUGHT— a claim did not match the recorded data, or its endpoint was not trusted.ERROR— nothing was verified (for example, zero claims could be resolved). Verifying zero claims is alwaysERROR, never a pass.
Async throughout — await every SDK call. Recorded traffic: httpx, aiohttp, and
requests. Each claim maps to its recorded call by the sourcerykit_ref the tool returns
(copied into the claimed value), so the same tool or action_name can be called repeatedly.