PROVENANCE ENGINE · EARLY ACCESS

Trace any value
back to its origin.

Replay shows you what happened. Provenance answers the harder question: why. Select any value in a recorded execution and trace it backwards — through every transformation, function call, and assignment — to the input that caused it.

REAL-WORLD EXAMPLE

A Flask API returns the wrong price.
Provenance finds the root cause.

A customer reports they were charged £0.00 for an order. The execution was recorded. Here's how provenance traces the bug in three steps — no log searching, no guessing.

1
Start at the symptom

The API response contains total_price: 0.00. Open the replay in VS Code, navigate to the response handler, and select the total_price variable. Click "Trace provenance".

routes/orders.py:67 step 8,241
65 def create_order_response(order):
66 response = {
67 "total_price": order.total_price,
68 "currency": order.currency,
69 }
order.total_price = 0.00 order.currency = "GBP"
Where did order.total_price = 0.00 come from?
2
Provenance jumps to the calculation

The engine traces total_price back to line 34 of the pricing service, where it was computed. The unit price was correct (£49.99), but the discount was 100% — wiping the total to zero. Now select discount and trace again.

services/pricing.py:34 step 6,102
32 def calculate_total(items, discount):
33 subtotal = sum(i.price for i in items)
34 total = subtotal * (1 - discount)
35 return round(total, 2)
subtotal = 49.99 discount = 1.0 total = 0.00
Where did discount = 1.0 come from?
3
Root cause: the promo code lookup

The discount value came from a database lookup. The promo code STAFF100 — an internal testing code that gives 100% off — was left active in production. Bug found. The fix is a policy check, not a code change.

services/promotions.py:51 step 4,887
49 def get_discount(promo_code):
50 promo = db.query(Promotion).filter_by(
51 code=promo_code
52 ).first()
53 return promo.discount_pct / 100
promo_code = "STAFF100" promo.discount_pct = 100 ◉ External input — database read
3
Three provenance jumps. Root cause identified.
From a £0.00 charge in the API response to a test promo code left in the production database. No log searching. No manual code tracing. No reproducing the issue.
HOW IT WORKS

Inside the provenance engine.

Retrace's provenance engine is a custom Python bytecode interpreter written in C++ that replaces CPython's evaluation function. It executes every instruction while tracking exactly where each value came from.

01
Instruction counter
A monotonic counter increments with every Python bytecode instruction executed. This gives every point in the program execution a unique, addressable timestamp — like a frame number in a video. Any state can be recreated by replaying to that counter value.
02
Stack provenance
For every value on every stack frame, the engine records which instruction put it there. When you select a variable, the engine looks up its provenance counter, replays to that instruction, and shows you the exact source line, operation, and input values.
03
Creation & update counters
Every object is stamped with the instruction counter at creation time. Mutable objects are additionally stamped whenever they're modified. This means you can trace any value in the heap — not just stack variables — back through its complete mutation history.
04
Deterministic replay substrate
Provenance runs on top of Retrace's record-replay layer. All non-deterministic operations (database queries, API calls, system time) are replayed from the recorded trace. The engine can seek to any point and reconstruct the exact state — no production access needed.
ARCHITECTURE
Three-layer system
The provenance engine sits between the replay layer and the analysis interface. Each layer has a single responsibility.
Record Proxy layer captures external calls to trace.bin in production
Replay Stub types return recorded results, deterministic re-execution
Provenance Custom C++ interpreter tags every stack value with instruction counter
Analysis VS Code debugger, MCP server, or API — query the provenance data
USE CASES

What provenance makes possible.

Once every value carries its history, a new class of questions becomes answerable — by humans and by AI.

Early access partners are using provenance for:

Root cause analysis

Start at a wrong output and trace backwards through every transformation to find exactly where the bug was introduced. No hypothesis-and-test cycles.


"The API returned the wrong price — three provenance jumps showed a test discount code active in production."

Sensitive data tracing

Track how PII, credentials, or regulated data flows through your application. See every function that touched it, every transformation it went through, every output it reached.


"Show me every code path where the user's email address appears between input and API response."

AI-directed debugging

Expose recorded executions to AI agents via MCP. They can open a trace, inspect the crash state, trace provenance, and report the root cause — autonomously.


"Claude investigated the production error in 6 queries and identified the data validation gap without human guidance."

Compliance & audit

Prove exactly how a regulated calculation was performed. Provenance provides a complete, verifiable chain from inputs to outputs for any computation.


"Auditors need to verify how the risk score was calculated — provenance shows every input and formula."

ML pipeline debugging

When model outputs are unexpected, trace training data or inference inputs backwards through preprocessing to find where data was corrupted or mislabelled.


"The model predicted 'spam' with 99% confidence — provenance showed a preprocessing bug duplicated training labels."

Incident post-mortems

After an incident, replay the exact production execution and use provenance to build a precise timeline of how data flowed through the system. No more reconstructing from logs.


"The post-mortem took 20 minutes instead of 2 days — we had the exact execution and could trace every decision."

SECURITY MODEL

Your recordings are the most sensitive artifact in the system. We know.

A recording contains everything — source code behaviour, user data, secrets in memory, API responses. Every design decision follows from one principle: encrypt early, decrypt late, isolate completely.

Design principle: The production environment cannot read its own recordings back. Recordings are encrypted with a per-run public key before leaving the proxy. Even if the production host is compromised, historical recordings on S3 remain encrypted. The private key never touches production.

Encrypted at the boundary

The recording proxy encrypts with a per-run public key before anything leaves the production environment. Each recording gets its own key pair. If a private key is ever compromised, the blast radius is one recording — not all of them.

RSA/ECIES key encapsulation + AES-256-GCM for data. Public key widely distributed, private key in KMS/HSM only.

Replay is fully sandboxed

Your code runs during replay — but in a container with no network, no writable filesystem, no credentials, and no Linux capabilities. The only output is a fixed-size binary pipe carrying hashes and instruction counters. No strings, no repr(), no raw object contents.

--network=none --read-only --cap-drop=ALL --security-opt=no-new-privileges

Per-session ephemeral keys

Each debugging session gets a scoped, ephemeral decryption key that lives in agent memory only. When the session ends, the key is discarded and tmpfs is wiped. The agent never holds a master key. Compromise of one agent exposes one recording.

Session-scoped agents. Disposable EC2 instances. No persistent state.

TRUST BOUNDARIES
Production proxy
Encrypts with per-run public key → writes ciphertext to S3 → discards plaintext
✕ no private key ✕ can't read back
↓ encrypted bytes (one way, write only)
S3 storage
Encrypted at rest · IAM-enforced access · No component can delete · Lifecycle expiry only
✓ encrypted ✓ immutable
↓ session assignment (S3 URI + ephemeral key)
Retrace Agent
One recording · one ephemeral key · network: Retrace Server + S3 only · tmpfs wiped on end
✕ no master key ✓ session-scoped
↓ pipe pair (fixed binary protocol, no network)
Replay container
No network · no writes · no credentials · no capabilities · rootless Podman
✕ no key ✕ no network ✕ no writes

Replay is strictly safer than production

Your code already runs in production with full network access, AWS credentials, and database connections. In Retrace replay, the same code runs in a sealed container. The only viable attack is a kernel-level container escape.

Capability Production Retrace Replay
Network access Full — serves traffic, calls APIs None
Filesystem writes Yes — logs, temp files, uploads None — read-only
AWS credentials Yes — IAM role for the app None
Secrets in environment Yes — API keys, DB passwords None — only pipe FDs
Database / cache access Yes — live connections None — replayed from recording
Data exfiltration requires One HTTP request (trivial) Kernel exploit to escape container

Blast radius is bounded by design

If compromised What's exposed Scope
Production host Future recordings (proxy tampered) — but no decrypt key, can't read historical recordings 1 customer
S3 bucket Nothing — all recordings encrypted, useless without private key None
Replay container One decrypted recording — no key, no network, sandboxed agent host 1 recording
Retrace Agent One ephemeral key + one recording — no master key, network-sandboxed 1 recording
Per-run private key One recording decryptable — per-run key rotation limits blast radius 1 recording
AI INTEGRATION

Let AI agents investigate production issues.

Retrace exposes its provenance engine via MCP (Model Context Protocol) — the standard for connecting AI tools to external data. Any MCP-compatible agent can open a recorded trace, navigate the execution, and trace values to their origin.

This means AI coding assistants can debug production failures autonomously: open the trace, find the crash, trace the root cause, and report back — without human guidance.

open_trace Load a recorded execution for analysis
get_crash_state Get exception info and stack at crash point
inspect_stack Inspect local variables at any instruction
trace_provenance Trace a value back to its origin
Claude investigating via MCP
Claude Opening the recorded trace...
open_trace("./recordings/incident-2847")

Claude Checking the crash state...
get_crash_state(thread_id=0)
✓ PIILeakDetected at policy_gate.py:28

Claude Inspecting the frame at crash...
inspect_stack(step=4782, frame=0)
✓ email_field = "user@example.com"

Claude Tracing where email_field came from...
trace_provenance("email_field")
✓ Origin: db_handler.py:91 — step 4,510

Claude The PII leak occurred because
the email field was included in the
API response without passing through
the redaction filter at policy_gate.py.
Root cause: missing sanitize() call.

Patented technology

Retrace's value-level provenance mechanism — including instruction counters, creation counters, update counters, and stack provenance — is protected by granted patents in the UK, US, and EU.

GB2593858B US11880279B2 EP4100831B1
Early access includes.
  • Access to the provenance engine alongside your existing Retrace installation
  • Direct support from the Retrace team during onboarding
  • Input into the API and tooling roadmap
Start with record-replay.
Go deeper with provenance.
The open-source recorder is free. The provenance engine is in early access — we’re working with a small group of teams to refine the experience before general availability.