> ## Documentation Index > Fetch the complete documentation index at: https://internal.september.wtf/llms.txt > Use this file to discover all available pages before exploring further. # Components > C4 Level 3: the major components inside the Engine and what each one is responsible for. The Engine runs as one process, but inside that process several substantial components cooperate. This page is a tour. Each component is described in terms of its responsibility, its main classes, and what it talks to. For the higher-level view, see [Containers](./containers). For shared type contracts, see [Types](./types). ## Engine core The driving loop and the surrounding context machinery. **Path:** `src/engine_core/` **Responsibility:** Drive the agent loop. Manage task slots. Handle context assembly, tool dispatch, compaction, and channel state. **Main pieces:** * `coordinator.py` — top-level orchestrator. Owns task slots, routes incoming `/execute` calls, and handles cancellation. Lazy-loaded through `get_coordinator()`. * `agent_loop.py` — the inner LLM-call loop. Sends context to herald, parses the response, dispatches tool calls. * `events.py` — 32+ event types. The vocabulary the SSE stream and internal subscribers use. * `compaction_orchestrator.py` — context overflow recovery. Collapses earlier turns into summaries. * `channel_state.py` — execution checkpoints; lets `/execute/replay` reconnect mid-turn. * `cached_snip.py` — prompt caching heuristics. * `context_engineering.py` — message formatting and context assembly. * `streaming_tool_executor.py` — runs tools concurrently up to `CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY`. **Talks to:** herald (LLM), memory (read/write), asset\_directory (tool calls), utility\_directory (tool calls), sandbox (tool execution). ## Memory Long-term and working memory. Three durable stores plus an in-execution working memory. **Path:** `src/memory/` **Responsibility:** Store, search, and serve episodes, knowledge facts, social-graph entries, soul, and working memory. **Main pieces:** * `episodic_memory.py` — episodes (what happened), with vec0 + FTS5. * `knowledge_store.py` — knowledge facts (what's true), with confidence and validity windows. * `social_graph.py` — nodes (people) and edges (relationships). * `working_memory.py` (interface), `sqlite_working_memory.py`, `inmemory_working_memory.py` — working memory backends. * `memory_factory.py` — factory; reads `USE_INMEMORY_WM` to choose. * `embeddings.py` — hybrid scoring (cosine + keyword). **Talks to:** infra (SQLite pool), herald (for embedding calls). ## Herald Unified abstraction over LLM providers. **Path:** `src/herald/` **Responsibility:** Take a normalized request (messages, tools, model parameters) and a provider choice; return a normalized streaming response. Handle prompt caching, quota tracking, retries, token budgeting. **Main pieces:** * `base.py` — abstract LLM interface. * `router.py` — picks Anthropic / OpenAI / Gemini. * `providers/{anthropic,openai,gemini,openai_compatible}.py` — the per-provider implementations. * `cache.py` — prompt cache hit tracking. * `usage_tracker.py` — token and call accounting. * `quota.py` — rate-limit handling. * `token_budget_parser.py` — token budget continuation loop (gated by `TOKEN_BUDGET_ENABLED`). **Talks to:** the upstream LLM provider over HTTPS. ## Sandbox The execution boundary for tools that touch the system. **Path:** `src/sandbox/` **Responsibility:** Run tool subprocesses isolated from the host. Apply permission policy; surface dangerous operations to HITL. **Main pieces:** * `executor.py` — orchestrates `bwrap` + `seccomp` + `landlock`. Manages process tree and watchdog. * `permissions.py` — permission prompt generation and routing. * `bash_security.py` — semantic analysis of bash commands for dangerous patterns. * `bash_paths.py` — path extraction from a parsed bash AST. * `path_safety.py` — path validation (Windows-evasion detection, symlink traversal). * `seccomp.py` — BPF filter compiler and loader. * `landlock.py` — landlock filesystem ACL setup. **Talks to:** the host kernel (syscalls, namespaces); the agent loop (emits permission prompts as HITL events). ## Asset Directory MCP connector subsystem. **Path:** `src/asset_directory/` **Responsibility:** Hold the registry of MCP servers, the user's connections, encrypted credentials, and OAuth machinery. Dispatch tool calls to remote MCP servers. **Main pieces:** * `asset_directory.py` — main coordinator and public surface. * `connection_manager.py` — connection lifecycle (OAuth flow, token refresh, credential rotation). * `mcp_client.py` — MCP protocol client. * `credential_store.py` — Fernet-encrypted credential storage. * `circuit_breaker.py` — failure detection; degrades misbehaving servers. * `refresh_scheduler.py` — background refresh of long-lived credentials. **Talks to:** the brain (credentials and registry); external MCP servers over their chosen transport (HTTPS / WebSocket / stdio). ## Utility Directory Built-in tool registry. **Path:** `src/utility_directory/` **Responsibility:** Define and dispatch the platform tools — read, write, bash, etc. Manage subprocess environment scrubbing. **Main pieces:** * `utility.py` — tool execution interface. * `platform_registry.py` — built-in tool catalog. * `subprocess_env.py` — env scrubbing in three tiers (off / default / strict, controlled by `SUBPROCESS_SCRUB_MODE`). * `tool_runtime.py` — runtime invocation glue. * `ud_write.py` — file write operations. **Talks to:** sandbox (for execution); the brain (custom tool defs). ## Learning Centre Background batch that turns trajectories into long-term memory. **Path:** `src/learning_centre/` **Responsibility:** On a schedule, read recent trajectories, generate episodes, consolidate knowledge facts, update soul and social graph. **Main pieces:** * `pipeline.py` — the batch pipeline. * `scheduler.py` — interval scheduler. * `signals.py` — learning signals (corrections, gotchas, reinforcement). **Talks to:** memory (write), herald (embedding + completion calls). ## Infra Cross-cutting infrastructure pieces. **Path:** `src/infra/` **Responsibility:** Database connection pool, migrations, graceful shutdown, background task coordination. **Main pieces:** * `sqlite_pool.py` — async SQLite connection pool with sqlite-vec loaded. * `migrations.py` — migration runner. Reads `migrations/*.sql` on startup. * `graceful_shutdown.py` — drain callback registry. * `bg_tasks.py` — background task coordination. ## Raven Observability storage and query. **Path:** `src/raven/` **Responsibility:** Capture observability events into the brain; expose a query API for external observability tooling. **Main pieces:** * `store.py` — event storage. * `api.py` — observability API. * `query.py` — query DSL. ## Security Content security primitives. **Path:** `src/security/` **Responsibility:** Detect secrets in model output; sanitize Unicode; guard writes that could leak. **Main pieces:** * `secret_scanner.py` — regex-based secret detection. * `unicode_sanitizer.py` — Unicode normalization. * `write_guard.py` — guard layer over outbound content. ## Server The HTTP surface. **Path:** `src/server.py` **Responsibility:** FastAPI app, route registration, middleware, SSE streaming, request validation. **Notable bits:** * `CORSMiddleware` configured from `CORS_ORIGINS`. * `UnicodeSanitizationMiddleware` strips problematic sequences from request bodies. * Routes are grouped by category (execution, hitl, rpc, assets, memory, skills, admin, observability) and import their handlers from the matching module. ## Bootstrap and config **Paths:** `src/bootstrap.py`, `src/config.py` `config.py` is a Pydantic settings loader that reads `.env` and the OS environment. `bootstrap.py` is the engine startup sequence: load config, init logging, run migrations, initialize subsystems, register graceful shutdown callbacks. ## See also * [Containers](./containers) for the C4 L2 view this page sits inside of. * [Data flow](./data-flow) for what calls what during a single request. * [Types](./types) for the contracts between these components.