backupsidecar/TESTING.md

Backup Script Testing Guide

This document provides step-by-step instructions for testing the backup script locally using Docker Compose with a Restic server and PostgreSQL database.

Prerequisites

• Docker and Docker Compose installed
• The backup script (src/backup.sh) and its dependencies (restic, curl, jq, pg_dump, psql)
• Basic understanding of bash and PostgreSQL

Setup

1. Start the Test Environment

# Start the services
docker-compose up -d

# Wait for services to be ready (about 10-15 seconds)
sleep 15

# Verify services are running
docker-compose ps

2. Verify Services

# Check if restic server is accessible
curl -s http://localhost:8000/api/v1/repos

# Check if PostgreSQL is accessible
docker exec postgres-test psql -U testuser -d testdb -c "SELECT version();"

Directory Backup Testing

This section tests the directory backup and restore functionality.

1. Prepare Test Data

# Create a test directory with sample files
mkdir -p test_data
cd test_data

# Create various types of files
echo "This is a text file" > sample.txt
echo '{"name": "test", "value": 123}' > data.json
mkdir -p subdir
echo "Nested file content" > subdir/nested.txt
echo "Binary data" | base64 > binary.dat

# Create some larger files for testing
dd if=/dev/urandom of=large_file.bin bs=1M count=10 2>/dev/null

# Go back to project root
cd ..

2. Perform Directory Backup

# Set environment variables for directory backup
export OPERATION_MODE="backup"
export BACKUP_MODE="directory"
export RESTIC_PASSWORD="testpassword123"
export RESTIC_REPOSITORY="rest:http://localhost:8000/test-backup"
export SOURCEDIR="/home/tbehrendt/dev/backupsidecar/test_data"
export ENABLE_GOTIFY="false"

# Run the backup
./src/backup.sh

3. Verify Backup

# List snapshots
restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")

# Check backup contents
restic -r "rest:http://localhost:8000/test-backup" ls latest --password-file <(echo "testpassword123")

4. Perform Directory Restore

# Create restore directory
mkdir -p restored_data

# Set environment variables for directory restore
export OPERATION_MODE="restore"
export RESTOREDIR="/home/tbehrendt/dev/backupsidecar/restored_data"
export RESTORE_SNAPSHOT_ID="latest"

# Run the restore
./src/backup.sh

5. Verify Directory Restore

# Compare original and restored directories
diff -r test_data restored_data

# Check file contents
echo "Original file:"
cat test_data/sample.txt
echo "Restored file:"
cat restored_data/sample.txt

# Verify binary file integrity
md5sum test_data/large_file.bin
md5sum restored_data/large_file.bin

# Check directory structure
tree test_data
tree restored_data

6. Cleanup Directory Test

# Remove test directories
rm -rf test_data restored_data

PostgreSQL Backup Testing

This section tests the PostgreSQL database backup and restore functionality.

1. Generate Test Data

# Generate test data in PostgreSQL
./generate_test_data.sh

2. Verify Initial Data

# Check that data exists
docker exec postgres-test psql -U testuser -d testdb -c "
SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
UNION ALL
SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
"

3. Perform PostgreSQL Backup

# Set environment variables for PostgreSQL backup
export OPERATION_MODE="backup"
export BACKUP_MODE="postgres"
export RESTIC_PASSWORD="testpassword123"
export RESTIC_REPOSITORY="rest:http://localhost:8000/postgres-backup"
export PGHOST="localhost"
export PGPORT="5432"
export PGDATABASE="testdb"
export PGUSER="testuser"
export PGPASSWORD="testpass"
export ENABLE_GOTIFY="false"

# Run the backup
./src/backup.sh

4. Verify PostgreSQL Backup

# List snapshots
restic -r "rest:http://localhost:8000/postgres-backup" snapshots --password-file <(echo "testpassword123")

# Check backup contents
restic -r "rest:http://localhost:8000/postgres-backup" ls latest --password-file <(echo "testpassword123")

5. Clear the Database

# Drop all tables to simulate data loss
docker exec postgres-test psql -U testuser -d testdb -c "
DROP TABLE IF EXISTS orders CASCADE;
DROP TABLE IF EXISTS customers CASCADE;
"

# Verify database is empty
docker exec postgres-test psql -U testuser -d testdb -c "
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public';
"

6. Perform PostgreSQL Restore

# Set environment variables for PostgreSQL restore
export OPERATION_MODE="restore"
export RESTORE_SNAPSHOT_ID="latest"

# Run the restore
./src/backup.sh

7. Verify PostgreSQL Restore

# Check that data has been restored
docker exec postgres-test psql -U testuser -d testdb -c "
SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
UNION ALL
SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
"

# Verify data integrity
docker exec postgres-test psql -U testuser -d testdb -c "
SELECT
    c.name as customer_name,
    COUNT(o.id) as order_count,
    SUM(o.price * o.quantity) as total_spent
FROM customers c
LEFT JOIN orders o ON c.id = o.customer_id
GROUP BY c.id, c.name
ORDER BY total_spent DESC
LIMIT 5;
"

# Check foreign key constraints
docker exec postgres-test psql -U testuser -d testdb -c "
SELECT
    tc.constraint_name,
    tc.table_name,
    kcu.column_name,
    ccu.table_name AS foreign_table_name,
    ccu.column_name AS foreign_column_name
FROM information_schema.table_constraints AS tc
JOIN information_schema.key_column_usage AS kcu
    ON tc.constraint_name = kcu.constraint_name
    AND tc.table_schema = kcu.table_schema
JOIN information_schema.constraint_column_usage AS ccu
    ON ccu.constraint_name = tc.constraint_name
    AND ccu.table_schema = tc.table_schema
WHERE tc.constraint_type = 'FOREIGN KEY'
    AND tc.table_name IN ('customers', 'orders');
"

Advanced Testing Scenarios

1. Test with Different Snapshot IDs

# List all snapshots
restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")

# Restore a specific snapshot
export RESTORE_SNAPSHOT_ID="<snapshot-id>"
./src/backup.sh

2. Test Error Handling

# Test with invalid repository
export RESTIC_REPOSITORY="rest:http://localhost:8000/nonexistent"
./src/backup.sh

# Test with wrong password
export RESTIC_PASSWORD="wrongpassword"
./src/backup.sh

3. Test with Gotify Notifications (Optional)

# If you have a Gotify server running
export ENABLE_GOTIFY="true"
export GOTIFYHOST="http://your-gotify-server:80"
export GOTIFYTOKEN="your-token"
export GOTIFYTOPIC="Backup Test"
./src/backup.sh

Cleanup

1. Stop Services

# Stop and remove containers
docker-compose down

# Remove volumes (optional - this will delete all data)
docker-compose down -v

2. Clean Up Test Files

# Remove any remaining test files
rm -rf test_data restored_data

Troubleshooting

Common Issues

1. Connection refused to restic server: Wait a bit longer for the container to start up
2. PostgreSQL connection failed: Ensure the database container is fully initialized
3. Permission denied: Make sure the backup script is executable (chmod +x src/backup.sh)
4. Restic repository not found: Check that the repository URL is correct and the server is running

Debug Commands

# Check container logs
docker-compose logs restic-server
docker-compose logs postgres

# Test restic connectivity
restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")

# Test PostgreSQL connectivity
docker exec postgres-test psql -U testuser -d testdb -c "SELECT 1;"

Expected Results

Directory Backup Test

• ✅ Backup completes successfully
• ✅ Files are backed up to restic repository
• ✅ Restore completes successfully
• ✅ Restored files match original files exactly
• ✅ Directory structure is preserved

PostgreSQL Backup Test

• ✅ Database backup completes successfully
• ✅ Database dump is backed up to restic repository
• ✅ Database can be cleared successfully
• ✅ Database restore completes successfully
• ✅ All data is restored correctly
• ✅ Foreign key relationships are maintained
• ✅ Data integrity is preserved

This testing procedure ensures that both directory and PostgreSQL backup/restore functionality works correctly and can be used as a foundation for automated testing in CI/CD pipelines.