External Dependencies and Foundation Crates

Caber relies on a curated set of external infrastructure services, embedded storage engines, foundation Rust crates (base_utils, caber_auth, caber_storage, caber_service_log), and specialized ML and policy runtimes to deliver 40 byte WRL Chunker characterization, freshness clustering, metapolicy enforcement, and cross chain provenance. This reference explains what each dependency is, what role it plays in Caber, which functional subsystems read or write it, and how it is configured.

Figure 1: Caber Core Engine Connected to External Storage, Search, Graph, Messaging, and Identity Dependencies

## 1. Infrastructure Service Dependencies (`Dependencies` Config Section)

1.1 shared-storage (S3 Compatible Object Store)

• Technology: Any S3 compatible object store (AWS.s3MasterBucket = csi-mvp-master and AWS.s3NotificationBucket = csi-s3-notification), including Amazon S3, VersityGW, MinIO, Ceph RadosGW, Cloudflare R2, or Google Cloud Storage. MinIO is an optional local development target and is not a hard dependency of Caber.
• Default Ports: 9000 (S3 API gateway / MinIO / VersityGW), 443 (Amazon S3 HTTPS).
• What It Does: Serves as Caber's immutable blob and artifact backbone (gss CORI source prefix). Every raw ingested file, parsed structural element list (elements.json), self healing parser redirect ({sha}.redirect.json), serialized .rbix and .rbixb index sidecar, compiled governance bundle (customer/governance/<policy_digest>/ and current/), and VerdictSnapshot (body version 3) is stored here.
• How Caber Uses It:
  • Connectors and Ingest (pycaber, S3_Scanner): Stream raw enterprise objects into csi-mvp-master and emit notification events via csi-s3-notification (csi-s3event-q).
  • Document Parser: Reads raw objects, evaluates Caber runtime, writes {sha}.redirect.json on sparse deterministic parse fallback, and persists .rbix and .rbixb sidecars.
  • Unified Governance (pycaber, pycaber): Loads policy-set.json, data.json, and manifest.json from customer/governance/current/ and writes VerdictSnapshot (v3) artifacts referenced by audit.edge_verdict_snapshot_ref.
  • See Shared Storage Bucket Reference for the complete bucket key layout.

1.2 elastic-search (OpenSearch / Elasticsearch)

• Technology: OpenSearch 2.x (compatible with Elasticsearch 8.x).
• Default Ports: 9200 (REST API), 9600 (Performance Analyzer).
• What It Does: Provides full text search, document metadata indexing, pairwise similarity matrix storage, W3C OpenTelemetry trace persistence, API discovery cataloging (csi_api_map), service log storage (csi_service_log), and real time policy decision audit logging (csi_edge_auth) across 33 physical indices mapped via GLOBAL.dbTables.
• How Caber Uses It:
  • caber-core and caber-hub: Write csi_objects_map_rust, csi_link_data_rust, csi_doc_graph, csi_edge_auth, csi_api_events, csi_api_map (via single writer ApiMapMinter with a 5 attempt CAS loop driven by csi_api_map_merge_policy.json), and csi_service_log.
  • CaberUserConsole BFF (server/services/opensearch.ts): Queries OpenSearch to populate the Executive Dashboard, Unified Audit waterfall, and Compliance matrices.

1.3 neo4j (Neo4j Graph Database)

• Technology: Neo4j 5.x with APOC and Graph Data Science (GDS) plugins.
• Default Ports: 7687 (Bolt protocol), 7474 (HTTP Browser).
• What It Does: Stores Caber's unified Lineage and Context Graph constructed by pycaber (v3.9.2) across 10 node labels (Entity, Edge, Data, Region, Object, Bucket, Service, API, Host, and Redaction:OboRedaction) and 21 schema indexes (19 Range indexes and 2 FullText indexes: entityTextIndex and dataAttrIndex).
• How Caber Uses It:
  • Structural vs Behavioral Topology: Maintains both Service-[:OF]->Host<-[:OF]-API (structural residency) and Service-[:CALLS]->API (behavioral invocation) without collapsing them.
  • Hierarchical Region Trees: Reconstructs Document -> Section -> Paragraph :Region hierarchies and attaches Decision E7 NER rollup attributes (region.ner.labels, region.ner.entity_count).

1.4 postgres-db (PostgreSQL)

• Technology: PostgreSQL 16+.
• Default Port: 5432.
• What It Does: Provides transactional relational storage (ACID) for obo_db (v2, storing On Behalf Of delegation records, originator attribution, and token nonces) and dashboard_logs (CaberUserConsole RBAC accounts, policy studio drafts, and alert subscriptions).
• Note on Decommissioned Stores: Analytical DuckDB and ClickHouse stores are disabled and removed from the active runtime (Option<()>).

1.5 message-queue (SQS / RabbitMQ) and kafka-zookeeper (Apache Kafka)

• Technology: AWS SQS (or ElasticMQ / LocalStack in local split mode), RabbitMQ 3.13+ (5672 AMQP), and Apache Kafka (9092).
• What They Do: In Split deployment personalities (Internal and External), work queues (csi-parse-q, csi-link-q, csi-graph-q, csi-auth-q, csi-api-q, csi-s3event-q) carry Payload.Record envelopes with payload.record.v1 capability negotiation. In Standalone mode (caber-hub), in process tokio::sync::mpsc Fold channels replace external queues.

4. Production Resilience and Sizing Guidance

Dependency	Minimum Production Spec	Backup and Recovery Strategy	High Availability Architecture
OpenSearch	3 master, 4 data nodes (64 GB RAM, 16 vCPU)	Automated snapshot repository to S3 bucket daily	Multi AZ cluster with 1 primary and 1 replica shard per index
Neo4j	3 node Causal Cluster (32 GB RAM, 8 vCPU)	Continuous transaction log archiving and neo4j-admin backup	Raft based leader with read follower replicas
S3 Compatible Object Store	4 node distributed cluster or AWS S3 Standard	Object versioning enabled with S3 Cross Region Replication	Distributed erasure coding (EC:4) or cloud native replication
PostgreSQL	1 primary, 1 standby (16 GB RAM, 4 vCPU)	WAL archiving to S3 via pgBackRest with point in time recovery	Streaming replication with Patroni and etcd auto failover
RocksDB (roaring_index.db)	Local NVMe SSD mount (1 TB, IOPS 10000)	Periodic hot checkpoints (tar.zst) to shared-storage	Rebuilt deterministically from .rbix and .rbixb shards if destroyed