Backup & Restore

Data Overview

Glean data is stored in the following locations:

Data Type	Storage Location	Importance
Database	PostgreSQL (postgres_data volume)	Critical
Cache	Redis (redis_data volume)	Optional
Logs	glean_logs volume	Optional
Vector Data	Milvus volumes (full version)	Important

Database Backup

Manual Backup

# Backup PostgreSQL database
docker exec glean-postgres pg_dump -U glean glean | gzip > glean_db_$(date +%Y%m%d_%H%M%S).sql.gz

Automated Backup Script

Create backup script backup-glean.sh:

#!/bin/bash

BACKUP_DIR="$HOME/glean-backups"
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p "$BACKUP_DIR"

# Backup PostgreSQL
docker exec glean-postgres pg_dump -U glean glean | gzip > "$BACKUP_DIR/glean_db_$DATE.sql.gz"

# Backup PostgreSQL data volume
docker run --rm \
  -v glean_postgres_data:/data \
  -v "$BACKUP_DIR":/backup \
  alpine tar czf /backup/postgres_data_$DATE.tar.gz -C /data .

# Backup Redis data volume
docker run --rm \
  -v glean_redis_data:/data \
  -v "$BACKUP_DIR":/backup \
  alpine tar czf /backup/redis_data_$DATE.tar.gz -C /data .

# Clean up backups older than 7 days
find "$BACKUP_DIR" -name "*.gz" -mtime +7 -delete

echo "Backup completed: $BACKUP_DIR"

Set execute permission and run:

chmod +x backup-glean.sh
./backup-glean.sh

Scheduled Backups

Set up daily backups with cron:

# Edit crontab
crontab -e

# Add daily backup at 2 AM
0 2 * * * /path/to/backup-glean.sh

Database Restore

Restore from SQL Backup

# Stop services
docker compose down

# Start database
docker compose up -d postgres

# Wait for database to be ready
sleep 10

# Restore data
gunzip -c glean_db_20250101_020000.sql.gz | docker exec -i glean-postgres psql -U glean -d glean

# Start all services
docker compose up -d

Restore from Volume Backup

# Stop services
docker compose down

# Remove old volume
docker volume rm glean_postgres_data

# Create new volume
docker volume create glean_postgres_data

# Restore data
docker run --rm \
  -v glean_postgres_data:/data \
  -v "$HOME/glean-backups":/backup \
  alpine tar xzf /backup/postgres_data_20250101_020000.tar.gz -C /data

# Start services
docker compose up -d

Full Backup (with Milvus)

Full version also needs to backup Milvus data:

#!/bin/bash

BACKUP_DIR="$HOME/glean-backups"
DATE=$(date +%Y%m%d_%H%M%S)

mkdir -p "$BACKUP_DIR"

# Backup PostgreSQL
docker exec glean-postgres pg_dump -U glean glean | gzip > "$BACKUP_DIR/glean_db_$DATE.sql.gz"

# Backup all data volumes
for volume in postgres_data redis_data milvus_etcd_data milvus_minio_data milvus_data; do
  docker run --rm \
    -v glean_${volume}:/data \
    -v "$BACKUP_DIR":/backup \
    alpine tar czf /backup/${volume}_$DATE.tar.gz -C /data .
done

echo "Full backup completed: $BACKUP_DIR"

Migrating to New Server

Export Data

On the old server:

# Backup database
docker exec glean-postgres pg_dump -U glean glean | gzip > glean_backup.sql.gz

# Export OPML (optional, user-level)
# Export via Web interface

Import Data

On the new server:

# Deploy Glean
curl -fsSL https://raw.githubusercontent.com/LeslieLeung/glean/main/docker-compose.yml -o docker-compose.yml
docker compose up -d

# Wait for database to be ready
sleep 30

# Restore data
gunzip -c glean_backup.sql.gz | docker exec -i glean-postgres psql -U glean -d glean

# Restart services
docker compose restart

Backup Strategy Recommendations

Strategy	Frequency	Retention	Description
Daily Backup	Daily	7 days	SQL backup
Weekly Backup	Weekly	4 weeks	Full volume backup
Monthly Backup	Monthly	12 months	Archive backup

Verifying Backups

Regularly verify backup integrity:

# Check backup files
gunzip -t glean_db_*.sql.gz

# Restore in test environment to verify
docker compose -f docker-compose.test.yml up -d
# Restore and verify data

Next Steps

• Update Guide - Update Glean version
• Troubleshooting - Common issue resolution