Deployment Guide¶

Docker Compose (Recommended)¶

The simplest way to run Crystal with persistent storage:

docker compose up -d

This starts Crystal on port 7540 with a named volume crystal_data for persistence.

To stop:

docker compose down

Data persists in the crystal_data volume across restarts. To destroy data:

docker compose down -v

Docker¶

Build the Image¶

docker build -t crystal .

Run with a Bind Mount¶

docker run -d \
  --name crystal \
  -p 7540:7540 \
  -v $(pwd)/data:/data \
  crystal

Run with a Named Volume¶

docker volume create crystal_data
docker run -d \
  --name crystal \
  -p 7540:7540 \
  -v crystal_data:/data \
  crystal

Binary¶

Build¶

go build -o crystal ./cmd/crystal

Run¶

./crystal \
  --host 0.0.0.0 \
  --port 7540 \
  --data-dir /var/lib/crystal \
  --storage disk

CLI Flags¶

Environment Variables¶

CLI flags can also be set via environment variables. Environment variables are overridden by CLI flags.

Config File¶

Create a JSON config file:

{
  "server": {
    "host": "0.0.0.0",
    "port": 7540,
    "read_timeout": 30000000000,
    "write_timeout": 30000000000,
    "read_header_timeout": 10000000000,
    "idle_timeout": 120000000000,
    "shutdown_timeout": 15000000000,
    "max_body_bytes": 4194304
  },
  "schema": {
    "vertices": [
      {
        "type_name": "Person",
        "constraints": [
          { "name": "name", "type": 0, "required": true }
        ]
      }
    ],
    "edges": []
  },
  "replication": {
    "primary_url": "",
    "graph": "",
    "token": "",
    "read_only": false,
    "retry_interval": 2000000000
  },
  "auth": {
    "enabled": true,
    "tokens": [
      { "name": "admin", "token": "REPLACE-ME", "scope": "write" },
      { "name": "dashboard", "token": "REPLACE-ME-TOO", "scope": "read" }
    ]
  },
  "tls": {
    "enabled": false,
    "cert_file": "/etc/crystal/tls/crystal.crt",
    "key_file": "/etc/crystal/tls/crystal.key"
  },
  "cors": {
    "allowed_origins": []
  },
  "storage": {
    "type": "disk",
    "data_dir": "/var/lib/crystal",
    "sync_on_write": false,
    "flush_interval": 30000000000,
    "max_wal_bytes": 67108864,
    "default_graph": "default",
    "max_graphs": 32
  }
}

Timeout values are in nanoseconds (Go time.Duration). 30000000000 = 30 seconds.

Security¶

Crystal refuses to start with authentication enabled and no tokens configured, which is the default. This is deliberate: the alternative is a graph database listening on 0.0.0.0 that anyone with network reach can read, rewrite or delete.

Generate a token and pass it in the environment rather than committing it:

export CRYSTAL_AUTH_TOKEN=$(openssl rand -hex 32)
./crystal --data-dir /var/lib/crystal

Clients (including crystal-cli, which reads CRYSTAL_TOKEN) send it as Authorization: Bearer <token>.

To run without authentication — only reasonable when bound to loopback behind another authenticating layer — set auth.enabled: false and bind to 127.0.0.1.

TLS¶

export CRYSTAL_TLS_CERT=/etc/crystal/tls/crystal.crt
export CRYSTAL_TLS_KEY=/etc/crystal/tls/crystal.key
./crystal

Crystal terminates TLS itself when configured. Behind a reverse proxy that already terminates TLS, leave it disabled and bind Crystal to loopback.

./crystal --config /etc/crystal/config.json

Scaling out¶

Crystal is RAM-resident (ADR 0001), so a single process is bounded by memory. Two mechanisms scale past one process, and they solve different problems.

Shard by graph — for capacity¶

One process serves several named graphs, and graphs are independent. Route each graph to whichever node has room:

curl -XPOST -H "Authorization: Bearer $TOKEN" \
  -d '{"name":"case-2024-07"}' http://node-a:7540/api/v1/graphs

This fits investigation and narrative workloads almost exactly, because those are naturally partitioned: one investigation, one case or one story is one graph. Capacity then scales with node count, under one constraint — a single graph must fit on a single node.

Set storage.max_graphs per node. Every graph is resident in full, so the memory cost of a process is the sum of its graphs.

Read replicas — for read throughput and availability¶

A follower consumes the primary's transaction stream and serves reads:

./crystal --data-dir /var/lib/crystal-replica \
          --replica-of https://primary:7540
export CRYSTAL_REPLICATION_TOKEN=<read-scoped token on the primary>

A follower refuses all writes with 403. Its data is defined entirely by the primary's stream, so a local write would either be silently overwritten by the next transaction touching the same entity, or survive as divergence that nothing ever reconciles.

What replication does and does not give you. It scales reads and gives you a warm copy on another machine. It does not raise capacity — every replica holds the whole graph — and it does not fail over automatically. Promotion is an operator action: stop the follower, remove --replica-of, restart, and repoint clients.

A follower that falls far enough behind that the primary has checkpointed away the transactions it needs will re-bootstrap from a full snapshot automatically. Raising max_wal_bytes and flush_interval on the primary widens that window.

Health Checks¶

Point restart logic at /livez and traffic routing at /readyz. Wiring both to the same endpoint defeats the split: a storage failure would restart the pod instead of depooling it, and restarting does not fix a disk.

Health Check¶

curl http://localhost:7540/health

Returns 200 OK with runtime info when the server is healthy.

Systemd¶

Example unit file at deployments/systemd/crystal.service:

[Unit]
Description=Crystal Graph Engine
After=network.target

[Service]
Type=simple
User=crystal
ExecStart=/usr/local/bin/crystal --data-dir /var/lib/crystal
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Data Backup¶

The disk backend stores data in a data directory with two files:

• graph.json -- full snapshot of the graph state
• wal.bin -- write-ahead log with mutations since the last snapshot

To backup, flush first to ensure a consistent snapshot, then copy:

curl -X POST http://localhost:7540/api/v1/graph/flush
cp /var/lib/crystal/graph.json /backup/crystal-$(date +%Y%m%d).json

To restore, stop Crystal, replace graph.json, remove wal.bin, and restart.

Crash Recovery¶

The disk backend uses a Write-Ahead Log (WAL) for crash safety. Every mutation is appended to wal.bin and fsynced before being applied in memory. On startup, Crystal:

1. Loads the last graph.json snapshot
2. Replays any WAL entries written after the snapshot
3. Writes a fresh snapshot and truncates the WAL

This means all acknowledged writes survive kill -9, OOM kills, and power loss. The only data that can be lost is a mutation that was interrupted mid-WAL-write (before fsync completed), which is at most one operation.

A background checkpointer writes a fresh snapshot and truncates the WAL at the configured flush_interval (default 30s). A second trigger fires inline once the WAL passes max_wal_bytes (default 64 MiB), so a burst of writes cannot outrun the timer. Together these bound WAL size and recovery time; setting both to zero removes the bound and the log grows until the disk fills.

Performance Considerations¶

• Every write does one fsync to the WAL file. On SSDs this is typically 0.1-1ms per mutation. For high-throughput batch loading, consider using the in-memory backend and exporting afterward.
• The checkpoint interval (flush_interval) controls the tradeoff between WAL size and checkpoint overhead. Shorter intervals mean faster recovery but more frequent full-graph serialization.
• max_wal_bytes bounds the same tradeoff by volume rather than by time, which is what protects a write-heavy burst. The write that crosses the threshold pays for the checkpoint inline, so a very small value turns ordinary writes into snapshot latency.
• The sync_on_write option triggers a full snapshot on every mutation in addition to the WAL, and truncates the WAL afterwards. This is rarely needed now that the WAL provides per-mutation durability, and it makes every write cost a full-graph serialization.
• The in-memory storage backend has no I/O overhead and is suitable for ephemeral workloads or testing.
• The current MVP stores the entire graph in memory regardless of backend. Plan for datasets that fit in RAM.