Ecosystem CLI Reference

This page catalogues all command line developer tools, build scripts, model service orchestrators, and governance utilities across the Caber ecosystem.

Figure 1: CLI Utilities Across Core Engine, Connectors, Demos, and Standards

## 1. Core Runtime & Ingestion CLIs

Transparently attaches Caber zero code auto instrumentation to Python AI applications, LLM agents, web frameworks, and background workers without modifying application source code.

• Syntax:

  caber-run <command> [args...]

• Positional Parameters:
  • <command> (string, required): The executable to run (for example python, uvicorn, gunicorn, pytest).
  • [args...] (list of strings, optional): Command line arguments forwarded directly to the target program.
• Execution Mechanism: Creates a temporary bootstrap directory containing sitecustomize.py which executes import pycaber.auto. Prepends this directory to PYTHONPATH and invokes the command via subprocess.run. Automatically instruments standard libraries (requests, httpx, urllib3, aiohttp, psycopg, redis) and frameworks before the application entry point executes.
• Environment Variables:
  • CSI_API_KEY: API key for Caber telemetry authentication.
  • CSI_CABER_HOST: Address of caber_core service (for example http://localhost:8080).
  • CSI_GLOBAL_REDACT_ENABLED: Global redaction kill switch (1 or 0).
• Exit Codes: Returns the exact exit code of the executed child command.

2. Build & Release CLIs (PyCaber/scripts/)

local_build.sh

Primary local compilation driver (1,939 lines) compiling native Rust crates into ABI3 Python wheels and building the caber_core container.

• Syntax:

  ./scripts/local_build.sh [options]

• Options & Parameters:
  • -v, --version <version> (string, optional): Specify semantic version string or auto increment current version.
  • -g, --package <pkg> (string, optional, default: all): Target package: pycaber_wrap, pycaber_parser, pycaber_connector, pycaber, caber_core, or all.
  • -x, --features <list> (string, optional, default: debug): Comma separated cargo features to add to configuration defaults.
  • -X, --override-features <list> (string, optional): Comma separated cargo features to replace configuration defaults entirely.
  • -p, --python <version> (string, optional): Specific Python version or space separated list (for example cp312 or cp39 cp312).
  • -d, --debug-build (flag, optional): Build with Cargo dev profile instead of release for lldb and gdb debugging.
  • -I, --with-image (flag, optional): Build native caber_core binary and local Docker container image.
  • --tag <tag> (string, optional, default: latest): Tag override for resulting Docker container.
  • -n, --no-push (flag, optional): Build wheels locally without publishing to artifact repository.
  • --no-stubs (flag, optional): Skip automatic wheel reinstallation and .pyi type stub generation.
  • -f, --force-clean (flag, optional): Purge Cargo target directories before starting build.
  • --timings (flag, optional): Generate Cargo HTML compilation timings report.
• Exit Codes: 0 on successful build, non zero on compilation or stub error.

install_wheels.sh

Reinstalls freshly compiled wheels from the local wheelhouse across all repository virtual environments.

• Syntax:

  ./scripts/install_wheels.sh

• Execution Mechanism: Executes .venv/bin/pip install --force-reinstall --find-links wheelhouse/macos-aarch64 sequentially across PyCaber, mvp, CaberDemos, and CaberEvalBenchmarks to guarantee all local workspaces run identical native binary versions.

prune_wheels.py

Cleans up historical native wheel artifacts from the wheelhouse directory while preserving the latest versions.

• Syntax:

  python scripts/prune_wheels.py [--wheelhouse <dir>] [-d] [-v]

• Options & Parameters:
  • --wheelhouse <dir> (path, optional, default: wheelhouse/): Path to wheelhouse directory.
  • -d, --dry-run (flag, optional): Show files that would be deleted without removing them.
  • -v, --verbose (flag, optional): Print detailed deletion decisions.
• Exit Codes: 0 on success.

ci_drift_check.py

Detects drift between Git repository interfaces, public exported symbols, and configuration schemas.

• Syntax:

  python scripts/ci_drift_check.py [--expected <path>] [--strict]

• Options & Parameters:
  • --expected <path> (path, optional): Path to baseline expected configuration schema file.
  • --strict (flag, optional): Treat warnings and benign interface additions as errors.
• Exit Codes: 0 if in sync, 1 if drift detected.

parser_service.py

Model service orchestrator managing local native GPU model servers for deep document parsing and named entity recognition.

• Syntax:

  python scripts/parser_service.py {pull|start|stop|status|check} [options]

• Subcommands & Options:
  • check: Evaluates host GPU memory and CUDA, MPS, or CPU availability.
  • pull: Downloads model weights for GLiNER2 or docling serve sidecar.
  • start: Launches the model server daemon.
    • --service {gliner2,docling}: Target model engine to launch.
    • --port <port>: Network port to bind.
    • --device {auto,mps,cuda,cpu}: Hardware compute acceleration target.
  • stop: Terminates the running model service daemon.
  • status: Reports daemon process state, memory consumption, and port binding.
• Exit Codes: 0 on success, 1 on error.

compile_proto_for_git_repo.py

Fetches Protocol Buffer definitions from a public Git repository, compiles binary descriptors with protoc, and converts protobuf messages to JSON.

• Syntax:

  python scripts/compile_proto_for_git_repo.py [options]

• Options & Parameters:
  • --repo <url> (string, required unless --descriptor-in is provided): Public Git repository URL.
  • --version <tag> (string, optional): Git tag, branch, or commit hash. Defaults to default branch.
  • --proto-root <dir> (string, optional): Subdirectory containing .proto files (for example proto).
  • --include-path <dir> (string, optional, repeatable): Additional protoc import include path.
  • --descriptor-out <path> (path, optional): Output compiled descriptor set file path.
  • --descriptor-in <path> (path, optional): Existing descriptor set file to load instead of compiling.
  • --message-type <type> (string, optional): Fully qualified message type (for example streamlit.ForwardMsg).
  • --message-file <path> (path, optional, repeatable): Binary protobuf message file to parse.
  • --json-out-dir <dir> (path, optional): Directory where converted JSON outputs are written.
  • --keep-repo <dir> (path, optional): Directory to preserve cloned repo rather than using a temp folder.
  • -v, --verbose (flag, optional): Enable debug level logging.
• Exit Codes: 0 on success, 1 on error.

3. Demo & Synthetic Data CLIs

Unified orchestrator for interactive evaluation, automated verification, video capture, and cloud deployment across all four demo scenarios.

• Full command options and flag reference available in Caber Demo CLI Reference.

4. Standards & Governance CLIs (caber-standards/scripts/)

validate_docs.py

Executes 10 automated compliance checks verifying metadata, link integrity, URI resolution, sync status, breaking alerts, mermaid syntax, marker balance, freshness, agent protocols, and negative lexicons.

• Syntax:

  python scripts/validate_docs.py [--registry <path>] [--strict] <repo_path>

• Parameters:
  • <repo_path> (path, positional, required): Path to repository documentation directory to validate.
  • --registry <path> (path, optional): Path to GLOBAL_REGISTRY.md for URI resolution.
  • --strict (flag, optional): Treat warnings as failures.
• Exit Codes: 0 if all checks pass, 1 if any check fails.

extract_schema_catalog.py

Extracts and normalizes CONFIG_SCHEMA.json into queryable parameters and markdown reference tables.

• Syntax:

  python scripts/extract_schema_catalog.py [--schema <path>] [--out <path>] [--pretty]

• Parameters:
  • --schema <path> (path, optional): Path to source CONFIG_SCHEMA.json.
  • --out <path> (path, optional): Output JSON destination path. Prints summary if omitted.
  • --pretty (flag, optional): Pretty print JSON formatting.

extract_live_interfaces.py

Harvests live Python AST symbols and Rust crate declarations across all Caber repositories to detect stale documentation.

• Syntax:

  python scripts/extract_live_interfaces.py [--root <dir>] [--out <path>] [--pretty]

• Parameters:
  • --root <dir> (path, optional): Root directory containing Caber repositories.
  • --out <path> (path, optional): Output JSON path.
  • --pretty (flag, optional): Pretty print JSON.

install_hooks.py

Installs pre commit Git hooks enforcing Caber documentation standards across workspace repositories.

• Syntax:

  python scripts/install_hooks.py <repo_path>

• Parameters:
  • <repo_path> (path, required): Target repository root where .git/hooks/pre-commit will be installed.
• Exit Codes: 0 on successful installation, 1 if path does not exist.