Core Product Functionality Overview

1. What It Is

Caber is an active data governance, subdocument provenance, and runtime policy enforcement platform for enterprise AI, RAG pipelines, and multiagent systems. Rather than organizing documentation around individual repositories whose boundaries shift as services consolidate from Python (Caber_Platform / mvp) into Rust (PyCaber), this section organizes Caber into five invariant functional domains:

1. Connecting Policy to Data Fragments & The WRL Chunker: How Caber ingests documents, extracts structure and entities, fragments text with 100% byte coverage using the WRL Chunker, and binds access policies to subdocument fingerprints inside the Roaring Bitmap Index (.rbix).
2. Stale Source Detection & Two Stage IDF Clustering: How Caber groups duplicate, forked, and edited documents across heterogeneous storage containers into directed version lineage trees (EditCostTree DAGs) to distinguish authoritative sources from stale replicas.
3. Multisource Metapolicies & Active Runtime Governance: How Caber resolves policy conflicts when a single AI output or API payload synthesizes fragments from multiple upstream objects, and how compiled Rego policies execute inline stream mutations (allow, redact, deny).
4. AI Output Provenance, Call Chain Identity & Human Principal Attribution: How Caber traces tool calls, bash commands, file reads/writes, and model outputs into Source Document Origin Trees while attributing every agent action across a multiagent call chain to the initiating Human Principal (obo.user).
5. Operator Control Plane, Session Audit Traces & Visual Governance: How security operators visualize live session audit traces (TraceBalloonView), inspect OBO identity chains (OboChainBranch), explore the Neo4j Context Graph, manage stale clusters, and author Unified Policy Sets.

5. Key Architecture Decisions & Design Rationale

• ADR 1: Functional Invariance Over Repository Topography: Services migrate from Python to Rust, and repositories consolidate over time. Caber structures its architecture around invariant functional primitives (40 byte WRL characterization, Two Stage IDF freshness DAGs, declarative metapolicies, and cross chain principal attribution) that remain identical regardless of whether execution occurs in Python mvp or Rust PyCaber.
• ADR 2: Dual Mode Execution (Internal/Fold vs Internal/Split): Small deployments and edge agents cannot afford multi container network overhead, while large enterprise deployments require horizontal worker scaling. Caber uses the exact same six stage message envelopes across both bounded Tokio in memory channels (Fold mode) and distributed message queues (Split mode).
• ADR 3: Dual Operational Planes: Decoupling asynchronous corpus characterization from synchronous runtime interception ensures that heavy document parsing and graph clustering never block live user prompts or tool invocations.