Skip to main content

Documentation Index

Fetch the complete documentation index at: https://septemberai.mintlify.app/llms.txt

Use this file to discover all available pages before exploring further.

The Engine’s brain is a single SQLite file. SQLite gets a lot of grief from people who haven’t run it in production at scale; in practice it handles single-user agentic workloads beautifully. This page covers operating the brain — backup, restore, vacuum, integrity checks, sizing.

​
What’s in the brain

20+ tables. The full schema lives in migrations/*.sql. The major ones:
TableWhat it holdsGrowth rate
episodes, episodes_vecEpisodic memory.Slow
knowledge_store, knowledge_store_vecKnowledge facts.Slow
social_graph_nodes, _edges, _vecPeople and relationships.Slow
working_memory_logWithin-task context. TTL-bound.Fast (but pruned)
trajectoriesPer-task execution traces.Fast
conversationsConversation threading.Slow
channel_state_snapshotsSSE checkpoints. TTL-bound.Fast (but pruned)
connections, server_registry, action_registryAsset Directory.Slow
learning_signalsUser feedback.Slow
observability_eventsTelemetry.Fast
ingestion_chunksLarge media chunks.Variable
*_vec tables hold 1536-dim float embeddings; they grow with episodes, knowledge facts, and social-graph nodes.

​
Sizing

A working brain for an actively-used agent typically grows at:
  • A few KB to a few MB per turn (working memory + observability events).
  • A few KB per episode (long-term, after Learning Centre processes trajectories).
  • A few hundred bytes per knowledge fact.
  • A few KB per MCP connection (encrypted credentials are small).
Order-of-magnitude after sustained use:
TimeBrain size (typical user)
First week5–20 MB
First month50–200 MB
First year500 MB – 2 GB
Multi-GB brains run fine. The bottleneck is your backup strategy, not SQLite.

​
Backup

Three options, in increasing rigor:

​
/memory/export

The Engine exposes an API endpoint that returns the entire memory as a portable JSON bundle: 
curl -fsS "$ENGINE_URL/memory/export" \
  -H "X-Engine-Key: $ENGINE_KEY" \
  > brain-export-$(date +%s).json
Restore via POST /memory/import. Useful for migrating between Engines or for application-level backups. Caveat: the export covers memory and connections. It does NOT cover:
  • Channel state (transient).
  • Observability events.
  • Trajectories not yet consolidated.
For a true full backup, use one of the next two.

​
SQLite .backup

SQLite’s online backup API takes a consistent snapshot without requiring downtime: 
docker compose exec engine sqlite3 /data/brain.sqlite \
  ".backup /data/brain-$(date +%s).db"
The backup file is itself a SQLite file you can copy out and store. To run on a schedule (cron / k8s CronJob): 
#!/bin/bash
TS=$(date +%Y%m%d-%H%M%S)
docker compose exec engine sqlite3 /data/brain.sqlite \
  ".backup /data/backups/brain-${TS}.db"
docker compose exec engine sh -c \
  'find /data/backups -mtime +7 -delete'  # 7-day retention

​
Volume snapshot

If your infrastructure provides volume snapshots (EBS, GCP persistent disks, k8s VolumeSnapshot): 
schedule: daily at 02:00
retain: 7 daily, 4 weekly, 12 monthly
Volume snapshots are the cheapest, fastest, and most complete backup. They include the entire data directory, not just the SQLite file. Recommended for production.

​
Restore

​
From /memory/export

curl -X POST "$ENGINE_URL/memory/import" \
  -H "X-Engine-Key: $ENGINE_KEY" \
  -H "Content-Type: application/json" \
  -d @brain-export.json
The import is additive — existing memory is preserved, new memory is merged in. To restore to a clean state, wipe the brain volume first.

​
From .backup

# 1. stop the engine
docker compose stop engine

# 2. restore
docker compose run --rm --entrypoint sh engine -c \
  'sqlite3 /data/brain.sqlite ".restore /data/backups/brain-20260420.db"'

# 3. start the engine
docker compose up -d engine

​
From volume snapshot

Detach the data volume; attach the snapshot; start the Engine. Your infrastructure’s documentation has the specific commands.

​
Integrity check

SQLite has a built-in integrity check: 
docker compose exec engine sqlite3 /data/brain.sqlite \
  'PRAGMA integrity_check;'
Should print ok. Run after any restore, after a hard kill, or on a schedule (weekly). If it returns errors, the brain is damaged. Restore from backup.

​
Vacuum

SQLite doesn’t reclaim disk space automatically. As rows are deleted (working memory entries expiring, old trajectories cleaned up), the file size doesn’t shrink — the deleted space sits as free pages. To reclaim: 
docker compose exec engine sqlite3 /data/brain.sqlite 'VACUUM;'
This requires temporary disk space equal to the brain’s current size (SQLite copies into a temp file). Run during low-traffic windows; the database is locked during the vacuum. For high-write workloads, consider auto-vacuum mode (set at table creation; the Engine’s migrations don’t enable it today). A reasonable schedule: vacuum monthly. The file size growth between vacuums isn’t dramatic in normal use.

​
Migrations

Migrations run automatically on Engine startup. Each migration file is a SQL script: 
migrations/
  001_memory_schema.sql
  002_utility_directory.sql
  ...
  014_token_issued_at.sql
The Engine tracks which migrations have been applied in a _migrations_applied table. On startup, it runs any unapplied migrations in order. Don’t edit applied migrations. Once a migration is in production, treat it as immutable. To change a schema, add a new migration with the change. Don’t run migrations manually. The Engine is the only thing that should write to _migrations_applied.

​
Cleanup

The Engine doesn’t auto-clean old data today. A reasonable retention policy: 
# trajectories older than 30 days
sqlite3 /data/brain.sqlite \
  'DELETE FROM trajectories WHERE created_at < datetime("now","-30 days");'

# observability events older than 90 days
sqlite3 /data/brain.sqlite \
  'DELETE FROM observability_events WHERE created_at < datetime("now","-90 days");'

# channel state snapshots are TTL-pruned automatically
Run weekly via cron / CronJob, then VACUUM monthly.

​
Performance tuning

For deployments with high write rates:
  • WAL mode. The Engine enables WAL by default. Verify: 
    PRAGMA journal_mode;
    Should return wal.
  • Synchronous mode. Default is NORMAL. For higher write throughput at the cost of slightly more durability risk on crash, use synchronous = NORMAL (already the default).
  • Cache size. The Engine sets a sensible default. Override only if profiling shows cache misses.

​
When the brain grows too large

A few GB is fine. Tens of GB is fine. Hundreds of GB starts to feel slow. If you’re approaching that scale on a single brain:
  1. Audit retention. Are you keeping data you don’t need?
  2. Move ingestion_chunks out. Large media chunks are the most common bloat. Consider a separate object store (S3, GCS) and store only references in the brain.
  3. Consider sharding. Run multiple Engines per user with topic- specific brains. Outside the design today, but possible.

​
See also