Configuration Overview and Orchestrator

This section is the definitive reference for configuring PyCaber and its central control plane orchestrator. Every parameter documented across this reference corresponds directly to a registered ConfigEntry in config/CONFIG_SCHEMA.json and is enforced by config/CONFIG_SCHEMA.json in caber-standards.

The configuration documentation is organized into four focused pages:

• Configuration Overview and Orchestrator (this page): Five tier runtime resolution, the pycaber two plane trust model, canonical operation hashing (operation-hash-v1 and operation-hash-v2), cryptographic approval grants (ApprovalGrant and GrantAuthority), revision store Compare And Swap (CAS) with the audit-canon-v1 tamper evident audit chain, six layer effective config provenance (ProvenanceLayer and RedactedValue), Operation Class RBAC with Separation of Duties (SoD), ConfigSource verification modes (LocalFile, RemotePlain, RemoteSigned), and compile time build feature gates.
• Core Engine and Dependencies: AWS, GLOBAL (including GLOBAL.sdk.* identity and instance registry parameters and GLOBAL.txnTokenConfig), Dependencies, WhitespaceRemapOnLeader, rbix, RKCQF, and Object_Sources endpoint vs URL scheme preservation.
• Services, SDKs, Ingest, and Policies: csi_genai_lib, DocParser (including DocParser.formatRouter and DocParser.nerEngine), ENV_Manager, Dashboard, S3_Scanner, API_Tap, ApiMap, Process_API, Link_Data (including Link_Data.separateClusterGroups), Auth_Analyzer, SQL_Scanner, Kafka_Scanner, Compliance_Policies, Identity_Sources, and Object_Sources.
• Environment Variables and Governance: Three bucket environment variable governance (config-selection, credential-source, bootstrap), the mechanical ConfigEntry.derived_env_var() rule (CSI_<SECTION_UPPER_SNAKE>_<KEY_UPPER_SNAKE>), the 7 kept legacy exceptions, and the complete Environment_Variables catalog.

Five Tier Configuration Precedence

Raw Nested JSON Case Preservation

When loading JSON configuration files (2026-08-20 parser invariant), sys_config normalizes top level section names and second level parameter keys for case insensitive registry lookup, while strictly preserving the exact casing of nested maps and object keys under raw_json (for example DocParser.formatRouter.routes, Dependencies.<service>.dbTables, Compliance_Policies, and Identity_Sources). Downstream consumers that inspect arbitrary user defined keys read the case preserved subtree directly from raw_json.

ConfigSource Verification Modes

At startup and during hot reload cycles, pycaber loads the active configuration snapshot through one of three ConfigSource variants:

• LocalFile: Reads a local JSON file from disk (selected via CSI_CONFIG_PATH or fallback config.json). Intended for local development, standalone CLI execution, and container deployments with volume mounted configs.
• RemotePlain: Fetches the JSON snapshot over HTTP/HTTPS from a control plane endpoint without signature verification. Permitted only in isolated development or test harnesses.
• RemoteSigned: Fetches a signed configuration envelope from sys_config_control and verifies its Ed25519 signature against the public key configured in CSI_CONFIG_SIGNING_PUBKEY before parsing or applying any setting. If signature verification fails or the revision number regresses below the last applied revision (rollback protection), sys_config rejects the payload and retains the current in memory configuration.

Local Prefix Resolution (get_global_output_directory)

When PyCaber runs in local filesystem storage mode (GLOBAL["localStorage"] = true) or writes intermediate staging artifacts, get_global_output_directory() computes the canonical base directory by joining:

1. GLOBAL["localPrefix"] (default "/var/tmp/"), and
2. Dependencies.shared-storage.bucket (falling back to GLOBAL.csiBucket, default "caber-data").

For example, with default settings (GLOBAL["localPrefix"] = "/var/tmp/" and Dependencies.shared-storage.bucket = "caber-data"), local object keys resolve under /var/tmp/caber-data/.