This commit is contained in:
348
TESTING.md
Normal file
348
TESTING.md
Normal file
@@ -0,0 +1,348 @@
|
||||
# Backup Script Testing Guide
|
||||
|
||||
This document provides step-by-step instructions for testing the backup script locally using Docker Compose with a Restic server and PostgreSQL database.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker and Docker Compose installed
|
||||
- The backup script (`src/backup.sh`) and its dependencies (restic, curl, jq, pg_dump, psql)
|
||||
- Basic understanding of bash and PostgreSQL
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Start the Test Environment
|
||||
|
||||
```bash
|
||||
# Start the services
|
||||
docker-compose up -d
|
||||
|
||||
# Wait for services to be ready (about 10-15 seconds)
|
||||
sleep 15
|
||||
|
||||
# Verify services are running
|
||||
docker-compose ps
|
||||
```
|
||||
|
||||
### 2. Verify Services
|
||||
|
||||
```bash
|
||||
# Check if restic server is accessible
|
||||
curl -s http://localhost:8000/api/v1/repos
|
||||
|
||||
# Check if PostgreSQL is accessible
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "SELECT version();"
|
||||
```
|
||||
|
||||
## Directory Backup Testing
|
||||
|
||||
This section tests the directory backup and restore functionality.
|
||||
|
||||
### 1. Prepare Test Data
|
||||
|
||||
```bash
|
||||
# Create a test directory with sample files
|
||||
mkdir -p test_data
|
||||
cd test_data
|
||||
|
||||
# Create various types of files
|
||||
echo "This is a text file" > sample.txt
|
||||
echo '{"name": "test", "value": 123}' > data.json
|
||||
mkdir -p subdir
|
||||
echo "Nested file content" > subdir/nested.txt
|
||||
echo "Binary data" | base64 > binary.dat
|
||||
|
||||
# Create some larger files for testing
|
||||
dd if=/dev/urandom of=large_file.bin bs=1M count=10 2>/dev/null
|
||||
|
||||
# Go back to project root
|
||||
cd ..
|
||||
```
|
||||
|
||||
### 2. Perform Directory Backup
|
||||
|
||||
```bash
|
||||
# Set environment variables for directory backup
|
||||
export OPERATION_MODE="backup"
|
||||
export BACKUP_MODE="directory"
|
||||
export RESTIC_PASSWORD="testpassword123"
|
||||
export RESTIC_REPOSITORY="rest:http://localhost:8000/test-backup"
|
||||
export SOURCEDIR="/home/tbehrendt/dev/backupsidecar/test_data"
|
||||
export ENABLE_GOTIFY="false"
|
||||
|
||||
# Run the backup
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
### 3. Verify Backup
|
||||
|
||||
```bash
|
||||
# List snapshots
|
||||
restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")
|
||||
|
||||
# Check backup contents
|
||||
restic -r "rest:http://localhost:8000/test-backup" ls latest --password-file <(echo "testpassword123")
|
||||
```
|
||||
|
||||
### 4. Perform Directory Restore
|
||||
|
||||
```bash
|
||||
# Create restore directory
|
||||
mkdir -p restored_data
|
||||
|
||||
# Set environment variables for directory restore
|
||||
export OPERATION_MODE="restore"
|
||||
export RESTOREDIR="/home/tbehrendt/dev/backupsidecar/restored_data"
|
||||
export RESTORE_SNAPSHOT_ID="latest"
|
||||
|
||||
# Run the restore
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
### 5. Verify Directory Restore
|
||||
|
||||
```bash
|
||||
# Compare original and restored directories
|
||||
diff -r test_data restored_data
|
||||
|
||||
# Check file contents
|
||||
echo "Original file:"
|
||||
cat test_data/sample.txt
|
||||
echo "Restored file:"
|
||||
cat restored_data/sample.txt
|
||||
|
||||
# Verify binary file integrity
|
||||
md5sum test_data/large_file.bin
|
||||
md5sum restored_data/large_file.bin
|
||||
|
||||
# Check directory structure
|
||||
tree test_data
|
||||
tree restored_data
|
||||
```
|
||||
|
||||
### 6. Cleanup Directory Test
|
||||
|
||||
```bash
|
||||
# Remove test directories
|
||||
rm -rf test_data restored_data
|
||||
```
|
||||
|
||||
## PostgreSQL Backup Testing
|
||||
|
||||
This section tests the PostgreSQL database backup and restore functionality.
|
||||
|
||||
### 1. Generate Test Data
|
||||
|
||||
```bash
|
||||
# Generate test data in PostgreSQL
|
||||
./generate_test_data.sh
|
||||
```
|
||||
|
||||
### 2. Verify Initial Data
|
||||
|
||||
```bash
|
||||
# Check that data exists
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "
|
||||
SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
|
||||
UNION ALL
|
||||
SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
|
||||
"
|
||||
```
|
||||
|
||||
### 3. Perform PostgreSQL Backup
|
||||
|
||||
```bash
|
||||
# Set environment variables for PostgreSQL backup
|
||||
export OPERATION_MODE="backup"
|
||||
export BACKUP_MODE="postgres"
|
||||
export RESTIC_PASSWORD="testpassword123"
|
||||
export RESTIC_REPOSITORY="rest:http://localhost:8000/postgres-backup"
|
||||
export PGHOST="localhost"
|
||||
export PGPORT="5432"
|
||||
export PGDATABASE="testdb"
|
||||
export PGUSER="testuser"
|
||||
export PGPASSWORD="testpass"
|
||||
export ENABLE_GOTIFY="false"
|
||||
|
||||
# Run the backup
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
### 4. Verify PostgreSQL Backup
|
||||
|
||||
```bash
|
||||
# List snapshots
|
||||
restic -r "rest:http://localhost:8000/postgres-backup" snapshots --password-file <(echo "testpassword123")
|
||||
|
||||
# Check backup contents
|
||||
restic -r "rest:http://localhost:8000/postgres-backup" ls latest --password-file <(echo "testpassword123")
|
||||
```
|
||||
|
||||
### 5. Clear the Database
|
||||
|
||||
```bash
|
||||
# Drop all tables to simulate data loss
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "
|
||||
DROP TABLE IF EXISTS orders CASCADE;
|
||||
DROP TABLE IF EXISTS customers CASCADE;
|
||||
"
|
||||
|
||||
# Verify database is empty
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "
|
||||
SELECT table_name FROM information_schema.tables
|
||||
WHERE table_schema = 'public';
|
||||
"
|
||||
```
|
||||
|
||||
### 6. Perform PostgreSQL Restore
|
||||
|
||||
```bash
|
||||
# Set environment variables for PostgreSQL restore
|
||||
export OPERATION_MODE="restore"
|
||||
export RESTORE_SNAPSHOT_ID="latest"
|
||||
|
||||
# Run the restore
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
### 7. Verify PostgreSQL Restore
|
||||
|
||||
```bash
|
||||
# Check that data has been restored
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "
|
||||
SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
|
||||
UNION ALL
|
||||
SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
|
||||
"
|
||||
|
||||
# Verify data integrity
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "
|
||||
SELECT
|
||||
c.name as customer_name,
|
||||
COUNT(o.id) as order_count,
|
||||
SUM(o.price * o.quantity) as total_spent
|
||||
FROM customers c
|
||||
LEFT JOIN orders o ON c.id = o.customer_id
|
||||
GROUP BY c.id, c.name
|
||||
ORDER BY total_spent DESC
|
||||
LIMIT 5;
|
||||
"
|
||||
|
||||
# Check foreign key constraints
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "
|
||||
SELECT
|
||||
tc.constraint_name,
|
||||
tc.table_name,
|
||||
kcu.column_name,
|
||||
ccu.table_name AS foreign_table_name,
|
||||
ccu.column_name AS foreign_column_name
|
||||
FROM information_schema.table_constraints AS tc
|
||||
JOIN information_schema.key_column_usage AS kcu
|
||||
ON tc.constraint_name = kcu.constraint_name
|
||||
AND tc.table_schema = kcu.table_schema
|
||||
JOIN information_schema.constraint_column_usage AS ccu
|
||||
ON ccu.constraint_name = tc.constraint_name
|
||||
AND ccu.table_schema = tc.table_schema
|
||||
WHERE tc.constraint_type = 'FOREIGN KEY'
|
||||
AND tc.table_name IN ('customers', 'orders');
|
||||
"
|
||||
```
|
||||
|
||||
## Advanced Testing Scenarios
|
||||
|
||||
### 1. Test with Different Snapshot IDs
|
||||
|
||||
```bash
|
||||
# List all snapshots
|
||||
restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")
|
||||
|
||||
# Restore a specific snapshot
|
||||
export RESTORE_SNAPSHOT_ID="<snapshot-id>"
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
### 2. Test Error Handling
|
||||
|
||||
```bash
|
||||
# Test with invalid repository
|
||||
export RESTIC_REPOSITORY="rest:http://localhost:8000/nonexistent"
|
||||
./src/backup.sh
|
||||
|
||||
# Test with wrong password
|
||||
export RESTIC_PASSWORD="wrongpassword"
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
### 3. Test with Gotify Notifications (Optional)
|
||||
|
||||
```bash
|
||||
# If you have a Gotify server running
|
||||
export ENABLE_GOTIFY="true"
|
||||
export GOTIFYHOST="http://your-gotify-server:80"
|
||||
export GOTIFYTOKEN="your-token"
|
||||
export GOTIFYTOPIC="Backup Test"
|
||||
./src/backup.sh
|
||||
```
|
||||
|
||||
## Cleanup
|
||||
|
||||
### 1. Stop Services
|
||||
|
||||
```bash
|
||||
# Stop and remove containers
|
||||
docker-compose down
|
||||
|
||||
# Remove volumes (optional - this will delete all data)
|
||||
docker-compose down -v
|
||||
```
|
||||
|
||||
### 2. Clean Up Test Files
|
||||
|
||||
```bash
|
||||
# Remove any remaining test files
|
||||
rm -rf test_data restored_data
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Connection refused to restic server**: Wait a bit longer for the container to start up
|
||||
2. **PostgreSQL connection failed**: Ensure the database container is fully initialized
|
||||
3. **Permission denied**: Make sure the backup script is executable (`chmod +x src/backup.sh`)
|
||||
4. **Restic repository not found**: Check that the repository URL is correct and the server is running
|
||||
|
||||
### Debug Commands
|
||||
|
||||
```bash
|
||||
# Check container logs
|
||||
docker-compose logs restic-server
|
||||
docker-compose logs postgres
|
||||
|
||||
# Test restic connectivity
|
||||
restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")
|
||||
|
||||
# Test PostgreSQL connectivity
|
||||
docker exec postgres-test psql -U testuser -d testdb -c "SELECT 1;"
|
||||
```
|
||||
|
||||
## Expected Results
|
||||
|
||||
### Directory Backup Test
|
||||
|
||||
- ✅ Backup completes successfully
|
||||
- ✅ Files are backed up to restic repository
|
||||
- ✅ Restore completes successfully
|
||||
- ✅ Restored files match original files exactly
|
||||
- ✅ Directory structure is preserved
|
||||
|
||||
### PostgreSQL Backup Test
|
||||
|
||||
- ✅ Database backup completes successfully
|
||||
- ✅ Database dump is backed up to restic repository
|
||||
- ✅ Database can be cleared successfully
|
||||
- ✅ Database restore completes successfully
|
||||
- ✅ All data is restored correctly
|
||||
- ✅ Foreign key relationships are maintained
|
||||
- ✅ Data integrity is preserved
|
||||
|
||||
This testing procedure ensures that both directory and PostgreSQL backup/restore functionality works correctly and can be used as a foundation for automated testing in CI/CD pipelines.
|
||||
Reference in New Issue
Block a user