Deployment Guide¶

Docker Compose (Recommended)¶

The simplest way to run Crystal with persistent storage:

docker compose up -d

This starts Crystal on port 7540 with a named volume crystal_data for persistence.

To stop:

docker compose down

Data persists in the crystal_data volume across restarts. To destroy data:

docker compose down -v

Docker¶

Build the Image¶

docker build -t crystal .

Run with a Bind Mount¶

docker run -d \
  --name crystal \
  -p 7540:7540 \
  -v $(pwd)/data:/data \
  crystal

Run with a Named Volume¶

docker volume create crystal_data
docker run -d \
  --name crystal \
  -p 7540:7540 \
  -v crystal_data:/data \
  crystal

Binary¶

Build¶

go build -o crystal ./cmd/crystal

Run¶

./crystal \
  --host 0.0.0.0 \
  --port 7540 \
  --data-dir /var/lib/crystal \
  --storage disk

CLI Flags¶

Environment Variables¶

CLI flags can also be set via environment variables. Environment variables are overridden by CLI flags.

Config File¶

Create a JSON config file:

{
  "server": {
    "host": "0.0.0.0",
    "port": 7540,
    "read_timeout": 30000000000,
    "write_timeout": 30000000000
  },
  "storage": {
    "type": "disk",
    "data_dir": "/var/lib/crystal",
    "sync_on_write": false,
    "flush_interval": 30000000000
  }
}

Timeout values are in nanoseconds (Go time.Duration). 30000000000 = 30 seconds.

./crystal --config /etc/crystal/config.json

Health Check¶

curl http://localhost:7540/health

Returns 200 OK with runtime info when the server is healthy.

Systemd¶

Example unit file at deployments/systemd/crystal.service:

[Unit]
Description=Crystal Graph Engine
After=network.target

[Service]
Type=simple
User=crystal
ExecStart=/usr/local/bin/crystal --data-dir /var/lib/crystal
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Data Backup¶

The disk backend stores data in a data directory with two files:

• graph.json -- full snapshot of the graph state
• wal.bin -- write-ahead log with mutations since the last snapshot

To backup, flush first to ensure a consistent snapshot, then copy:

curl -X POST http://localhost:7540/api/v1/graph/flush
cp /var/lib/crystal/graph.json /backup/crystal-$(date +%Y%m%d).json

To restore, stop Crystal, replace graph.json, remove wal.bin, and restart.

Crash Recovery¶

The disk backend uses a Write-Ahead Log (WAL) for crash safety. Every mutation is appended to wal.bin and fsynced before being applied in memory. On startup, Crystal:

1. Loads the last graph.json snapshot
2. Replays any WAL entries written after the snapshot
3. Writes a fresh snapshot and truncates the WAL

This means all acknowledged writes survive kill -9, OOM kills, and power loss. The only data that can be lost is a mutation that was interrupted mid-WAL-write (before fsync completed), which is at most one operation.

A background checkpointer writes a fresh snapshot and truncates the WAL at the configured flush_interval (default 30s). This bounds WAL size and recovery time.

Performance Considerations¶

• Every write does one fsync to the WAL file. On SSDs this is typically 0.1-1ms per mutation. For high-throughput batch loading, consider using the in-memory backend and exporting afterward.
• The checkpoint interval (flush_interval) controls the tradeoff between WAL size and checkpoint overhead. Shorter intervals mean faster recovery but more frequent full-graph serialization.
• The sync_on_write option triggers a full snapshot on every mutation in addition to the WAL. This is rarely needed now that the WAL provides per-mutation durability.
• The in-memory storage backend has no I/O overhead and is suitable for ephemeral workloads or testing.
• The current MVP stores the entire graph in memory regardless of backend. Plan for datasets that fit in RAM.