AI Provenance & Principal Attribution

Modern agentic systems do not merely answer single turn questions; an orchestrator agent delegates subtasks to planner, coder, and retriever subagents, which in turn invoke Model Context Protocol (MCP) servers, REST APIs, SQL databases, bash shells, and local filesystems. In standard observability tools, identity is lost at the first agent hop (every downstream call executes under the agent's shared service account) and data lineage stops at the vector database query.

Caber solves both problems simultaneously:

1. End to End AI Output Provenance: Every model completion is linked backward through tool calls, API requests, bash executions, and file reads all the way to the exact source document origin trees and 40 byte WRL Chunker spans.
2. Cross Chain Principal Attribution: Agent identity is cryptographically chained across multi agent delegations so that data use policy enforcement at every hop is evaluated against the initiating human Principal, the acting agent chain, and the attributes and context of the data.

Figure 1: End to End AI Output Provenance and Source Document Origin Tree

## Part 1: AI Output Provenance and Source Document Origin Trees

What Provenance Tracks and How It Works

When an LLM generates an output (gen_ai.chat), Caber constructs a directed acyclic Provenance Origin Tree stored in Neo4j and indexed in OpenSearch (csi-provenance-trees and csi-traces). The tree connects four layers of evidence:

1. AI Completion & Reasoning Layer:
   • Captures the model response span (gen_ai.chat), model identifier, token usage, temperature, and finish reasons via OpenTelemetry and OpenLit compatible hooks.
2. Tool, API, Bash, and File Execution Layer:
   • Traces every tool call invoked during the agent turn (gen_ai.execute_tool), including MCP tool invocations (mcp.tool.call), outbound HTTP/REST API requests (http.client.request), subprocess shell commands (process.command / bash execution hooks), and local or remote file reads (file.read / vfs.open).
   • Captures cryptographic SHA256 digests of tool inputs and outputs alongside execution exit codes and latency.
3. Subdocument Characterization Layer:
   • Runs caber_core::characterizer with the WRL Chunker on every tool output, file read payload, retrieved RAG chunk, and final LLM completion.
   • Identifies which specific byte ranges ([start_byte, end_byte]) of the tool output or RAG context actually survived into the model's final output (DERIVED_FROM_FRAGMENT edges).
4. Source Document Origin Layer:
   • Links each matched doc_id back through its ingestion and derivation history in Neo4j (DERIVED_FROM, CANONICAL_VERSION_OF, INGESTED_FROM_BUCKET) to the root enterprise artifact (for example, an S3 object version, a SharePoint DOCX, a Snowflake table row, or a Git commit).

Why Provenance Origin Trees Matter

• Hallucination vs. Grounded Fact Verification: Operators and downstream validators can see immediately which sentences in an AI completion are grounded in canonical enterprise documents (98.7% coverage down to 40 bytes) and which sentences were synthesized without source backing.
• Forensic Blast Radius Analysis: If a sensitive document or poisoned data source is discovered, security teams query Neo4j to find every tool call, agent session, and AI output ever derived from any 40 byte fragment of that file.

Figure 2: Cross Chain Agent Identity Propagation and Principal Based Policy Enforcement

## Part 2: Cross Chain Agent Identity & Principal Attribution

The Confused Deputy Problem in Multi Agent Systems

When Alice (Principal: alice@corp.com, role Marketing_Analyst) asks an Orchestrator Agent to summarize quarterly performance, and the Orchestrator delegates to a SQL/Retrieval Subagent running with an AWS IAM role or service token that has broad read access to Finance_Restricted, checking only the Subagent's credentials allows Alice to exfiltrate restricted financial data.

How Caber Propagates Principal Identity Across Call Chains

Caber implements the On Behalf Of (OBO) Principal Attribution Protocol across pycaber(auto), caber_core::identity, and Auth_Analyzer:

1. Ingress Principal Binding:
   • At the initial entry point (API gateway, web chat session, Slack bot, or CLI invocation), Caber extracts the authenticated human user's identity token (OIDC JWT from Okta, Microsoft Entra ID, Keycloak, or AWS IAM Identity Center) and resolves group memberships and security attributes via the Identity Connectors (Identity_Sources).
   • Caber mints an immutable PrincipalContext containing principal_id, tenant_id, roles, groups, clearance_level, and session_id.
2. Cryptographic W3C baggage & caber-obo-token Propagation:
   • Whenever an instrumented service or agent spawns a subagent, calls an HTTP endpoint, invokes an MCP tool, or publishes a message to RabbitMQ/Kafka, pycaber(auto) and caber_core inject W3C traceparent plus a HMAC SHA256 / Ed25519 signed caber-obo-token inside W3C baggage:
     • caber.principal.id: The original human Principal (alice@corp.com)
     • caber.agent.chain: Ordered list of delegated agent identities (["agent://orchestrator-v2", "agent://retrieval-worker-04", "mcp://finance-tools"])
     • caber.delegation.scope: Intersection of permissions granted along the chain
     • caber.signature: Cryptographic signature binding trace_id, principal_id, and agent.chain
3. Effective Permission Intersection at Policy Enforcement Points:
   • When the leaf agent or tool attempts to read a document or return a completion containing fragments characterized by the WRL Chunker, PolicyEngine computes effective authorization over the tuple:$$\text{AuthDecision}=\text{Eval}(\text{Principal},\bigcap _{a\in \text{AgentChain}}\text{Scope}(a),\text{DataAttributes},\text{DataContext})$$
   • Even if agent://retrieval-worker-04 has technical network access to a Finance_Restricted document, the policy evaluation fails closed because the initiating Principal (alice@corp.com) lacks Finance_Restricted clearance.

Supported Agent Frameworks & Propagation Transports

Framework / Transport	Hook Mechanism in pycaber(auto) & caber_core	Propagated Context Carrier
LangChain / LangGraph	BaseCallbackHandler + RunnableConfig metadata injection	In memory ContextVar + checkpoint state metadata
CrewAI / AutoGen / OpenAI Agents SDK	Handoff / task delegation wrapper via wrapt hooks	Agent message headers & shared context dictionary
Anthropic MCP (Stdio & SSE)	MCP client/server transport interceptor (params._meta)	JSON RPC _meta.caber_obo_token & traceparent
HTTP / gRPC Microservices	httpx, requests, aiohttp, grpcio, tonic middleware	HTTP headers traceparent, baggage, x-caber-obo-token
Subprocess / Bash Tool Calls	subprocess.Popen / os.execve environment wrapper	CABER_OBO_TOKEN, TRACEPARENT, CABER_PARENT_SPAN_ID env vars

6. Key Architecture Decisions & Design Rationale

• ADR 1: Cryptographic OBO Tokens vs Shared Agent Service Accounts: Propagating shared API keys or service account tokens across multi agent hops causes confused deputy vulnerabilities. Caber signs immutable caber obo tokens carrying the initiating human Principal identity and downscopes delegation permissions at every hop.
• ADR 2: Unified Provenance Trees in Neo4j: Storing tool executions and RAG chunk citations in disconnected log files prevents retroactive audit reconstruction. Caber persists unified Provenance Origin Trees in Neo4j, linking AI completions back to tool inputs, shell processes, and raw document chunks.
• ADR 3: Subdocument Derivation Mapping: Instead of coarse file level provenance, Caber links LLM completions to the exact 40 byte WRL Chunker spans detected in intermediate tool outputs and prompts via DERIVED_FROM_FRAGMENT graph edges.