Metapolicies & Multisource Governance

When an AI pipeline or RAG workflow synthesizes an answer, a single prompt or completion routinely combines fragments from documents owned by different departments, security classifications, and regulatory regimes. Evaluating each source policy in isolation is insufficient because sources often disagree. Caber introduces Metapolicies to govern how multiple source policies are combined and resolved into a single, deterministic authorization verdict.

Figure 1: Metapolicy Multisource Conflict Resolution Pipeline (Stage 1 per source policies, Stage 2 freshness filter, and Stage 3 metapolicy combinator)

## What Metapolicies Are and How They Fit Into the Bigger Picture

A Source Policy governs an individual document or data source (for example, finance_sec_10k_policy.rego or eng_architecture_policy.rego). A Metapolicy is a higher order policy that takes the vector of per source verdicts, freshness attributes, similarity scores, and principal context as inputs and computes the composite authorization decision (ALLOW, DENY, or WARN).

In the Caber functional pipeline:

1. Characterization & WRL Chunker identifies every source document (doc_id) and byte span present in the payload.
2. Freshness & Stale Clustering labels each matched document with its cluster status (CANONICAL, STALE_REPLICA, or SUPERSEDED) and similarity weight.
3. Metapolicy & Governance Engine evaluates each source document's policy against the initiating Principal and then applies the active Metapolicy to resolve conflicts across all matched sources before the payload leaves the process or gateway.

How Two Stage Policy and Metapolicy Evaluation Works

Both Python Auth_Analyzer (eval_policy.py and auth_analyzer.py) and Rust caber_core::governance::PolicyEngine execute a three phase resolution pipeline:

Phase 1: Per Source Policy Evaluation

For every matched document d_i in the characterization result:

• The engine resolves d_i's assigned source policy (policy_id) and metadata attributes (classification tier, department owner, PII/PHI entity tags from GLiNER2, and regulatory jurisdiction) from OpenSearch csi-document-db and Neo4j.
• The engine evaluates the Rego policy (Regorus in Rust or OPA REST in Python) against the input envelope containing the initiating Principal's attributes (principal_id, roles, groups, clearance_level, department), the action (prompt_assembly, completion_return, tool_invocation, or response_delivery), and the matched fragment context.
• Each source evaluation emits a structured SourceVerdict:
  • decision: ALLOW, DENY, or WARN
  • classification_rank: numeric lattice rank (0 = PUBLIC, 1 = internal, 2 = confidential, 3 = RESTRICTED, 4 = TOP_SECRET)
  • freshness_status: CANONICAL, STALE_REPLICA, or SUPERSEDED
  • similarity_score: overlap weight in [0.0, 1.0]
  • obligations: required post processing actions (REDACT_SPAN, MASK_PII, ATTACH_WATERMARK, REQUIRE_STEP_UP_MFA, LOG_COMPLIANCE_EVENT)

Phase 2: Freshness Filtering (freshness_filtered_override)

Before conflict resolution runs across the SourceVerdict vector, the metapolicy evaluator inspects the cluster lineage of all matched documents:

• If a document d_stale has freshness_status in {STALE_REPLICA, SUPERSEDED} and its canonical cluster leader d_canonical is either present in the matched set or verified in the csi-doc-similarity graph with an updated policy:
  • The stale replica's obsolete ACL or classification tag is suppressed (SUPPRESSED_STALE_ACL).
  • Only the canonical leader's current policy verdict participates in Phase 3.
• If d_stale is matched in a prompt or completion without its canonical leader (meaning the AI agent retrieved purely outdated text from a stale vector index), the metapolicy triggers a STALE_DATA_VIOLATION (DENY or WARN with replacement pointer to canonical_doc_id).

Phase 3: Metapolicy Combinator Resolution

Caber supports five configurable metapolicy combinators in Governance_Policies and Auth_Analyzer:

Metapolicy Combinator	Formal Resolution Rule	Primary Enterprise Use Case
deny_overrides (Default)	If any active (non stale) SourceVerdict is DENY, the composite verdict is DENY. Otherwise if any is WARN, composite is WARN; else ALLOW.	Strict zero trust enforcement across regulated multi tenant or cross department corpora.
most_restrictive_lattice	Computes the supremum (least upper bound) across all active sources on the sensitivity lattice (max(classification_rank)) and unions all obligations. Verifies Principal clearance against the supremum rank.	Defense, intelligence, and financial research where combining two internal sources from distinct compartments elevates sensitivity to confidential (compartment aggregation rule).
freshness_filtered_override	Filters out STALE_REPLICA and SUPERSEDED source verdicts when CANONICAL lineage is verified, then applies deny_overrides across the surviving canonical verdicts.	Enterprise wikis, S3 buckets, and SharePoint estates with frequent document revisions and uncollected shadow copies.
source_priority_weighted	Ranks sources by authoritative repository priority (authoritative_db > curated_kb > team_wiki > scratch_share) weighted by fragment similarity_score.	Hybrid RAG pipelines mixing authoritative system of record tables with informal Slack or wiki threads.
consensus_quorum	Requires a configurable weighted threshold (for example, >= 0.75 of matched fragment weight) to ALLOW, provided zero RESTRICTED sources emit DENY.	High recall exploratory search where minor boilerplate overlap with low weight documents should not block synthesis.

Why Metapolicies Matter

1. Eliminates the "Lowest Common Denominator" Trap: Without metapolicies, systems either fail open (allowing access if any single source is public) or fail closed on stale shadow copies.
2. Handles Compartment Aggregation: When an agent combines M&A target financials with internal HR headcount projections, most_restrictive_lattice automatically elevates the composite classification and enforces executive only clearance.
3. Sub Millisecond In Process Execution: By compiling Rego policies and metapolicies into the embedded Rust Regorus engine inside caber_core, two stage evaluation completes in < 0.5 ms without external network round trips.

6. Key Architecture Decisions & Design Rationale

• ADR 1: Embedded Regorus Engine vs Out of Process OPA Daemon: Calling an external Open Policy Agent daemon over HTTP or Unix domain sockets adds 5 to 15 milliseconds of latency to every token and tool call. Caber embeds Regorus directly into the caber_core Rust binary, evaluating Rego policy bundles in under 200 microseconds per span.
• ADR 2: Two Stage Policy and Metapolicy Decoupling: Evaluating every document with a monolithic global rule creates unmaintainable policy sprawl. Caber evaluates individual source policies in Stage 1, filters by freshness in Stage 2, and resolves multisource conflicts via declarative Metapolicy combinators (intersect, union, strict priority) in Stage 3.
• ADR 3: Dual Level Policy Hierarchy: Fine grained subdocument sensitivity labels (PHI, PII, Secret) from GLiNER2 take precedence over coarse document level classifications, preventing overly broad allow rules from leaking embedded secrets.