sync

ci: add bash syntax check (#18 )
Reviewed-on: #18 Co-authored-by: Timo Behrendt <t.behrendt@t00n.de> Co-committed-by: Timo Behrendt <t.behrendt@t00n.de>
2025-09-06 08:24:06 +02:00 · 2025-09-04 20:48:13 +02:00 · 2025-09-04 20:36:40 +02:00
8 changed files with 248 additions and 551 deletions
--- a/.gitea/workflows/ci.yaml
+++ b/.gitea/workflows/ci.yaml
@@ -22,3 +22,12 @@ jobs:
          provenance: false
          tags: |
            backupsidecar:ci-test
  check-syntax:
    name: Check syntax
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - name: Check syntax
        run: |
          bash -n src/backup.sh
--- a/4
+++ b/4
@@ -7,6 +7,10 @@ RUN apk update && apk add --no-cache \
    postgresql-client \
    jq
 RUN curl -O https://dl.min.io/client/mc/release/linux-amd64/mc \
    && chmod +x mc \
    && mv mc /usr/local/bin/
 WORKDIR /app
 COPY src/backup.sh /app/backup.sh
--- a/README.md
+++ b/README.md
@@ -11,7 +11,7 @@ BackupSidecar is configured through environment variables. Below is a breakdown
 These variables apply to both backup and restore operations.
 - **`OPERATION_MODE`** _(optional)_ - Defines the operation type (`backup` or `restore`). Defaults to `backup`.
- **`BACKUP_MODE`** _(optional)_ - Defines the backup type (`directory` or `postgres`). Defaults to `directory`.
+- **`BACKUP_MODE`** _(optional)_ - Defines the backup type (`directory`, `postgres`, or `s3`). Defaults to `directory`.
 - **`RESTIC_PASSWORD`** _(required)_ - The encryption password for Restic.
 - **`RESTIC_REPOSITORY`** _(required)_ - The URI of the Restic repository (e.g., `rest:http://your-rest-server:8000/backup`).
 - **`RESTIC_REST_USERNAME`** _(optional)_ - The username for REST server authentication.
@@ -55,6 +55,22 @@ For `postgres` mode, the following database-related variables are required:
 - **`RESTORE_SNAPSHOT_ID`** _(optional)_ - The specific snapshot ID to restore (defaults to `latest`).
 - **`PSQL_ARGS`** _(optional)_ - Additional flags for `psql` (e.g., `--single-transaction`).
 ### S3 Operations
 For `s3` mode, the following S3-related variables are required:
 **Common Variables:**
 - **`S3_BUCKET`** _(required)_ - The name of the S3 bucket to backup/restore.
 - **`S3_ENDPOINT`** _(required)_ - The S3 endpoint URL (e.g., `http://minio:9000` for MinIO).
 - **`MINIO_ACCESS_KEY`** _(required)_ - The MinIO access key for S3 authentication.
 - **`MINIO_SECRET_KEY`** _(required)_ - The MinIO secret key for S3 authentication.
 - **`S3_PREFIX`** _(optional)_ - Optional path prefix within the bucket to backup/restore.
 **Restore-Specific Variables:**
 - **`RESTORE_SNAPSHOT_ID`** _(optional)_ - The specific snapshot ID to restore (defaults to `latest`).
 ## Dependencies
 Ensure the following commands are available in the container:
@@ -64,6 +80,7 @@ Ensure the following commands are available in the container:
 - `jq`
 - `pg_dump` _(only required for PostgreSQL backup operations)_
 - `psql` _(only required for PostgreSQL restore operations)_
 - `mc` _(only required for S3 operations)_
 ## Usage
@@ -236,6 +253,120 @@ spec:
              value: "Database Restore Notification"
 ```
 Example Kubernetes Job manifest for running BackupSidecar to backup an S3 bucket:
 ```yaml
 apiVersion: batch/v1
 kind: Job
 metadata:
  name: backupsidecar-s3-backup
  namespace: authentik
 spec:
  backoffLimit: 3
  activeDeadlineSeconds: 600
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: backupsidecar
          image: backupsidecar:latest
          env:
            - name: OPERATION_MODE
              value: "backup"
            - name: BACKUP_MODE
              value: "s3"
            - name: S3_BUCKET
              value: "my-bucket"
            - name: S3_ENDPOINT
              value: "http://minio:9000"
            - name: S3_PREFIX
              value: "data" # optional
            - name: MINIO_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: minio-secret
                  key: access_key
            - name: MINIO_SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: minio-secret
                  key: secret_key
            - name: RESTIC_REPOSITORY
              value: "rest:http://rest-server:8000/backup"
            - name: RESTIC_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: backupsidecar-secret
                  key: restic_password
            - name: GOTIFYHOST
              value: "http://gotify.example.com"
            - name: GOTIFYTOKEN
              valueFrom:
                secretKeyRef:
                  name: backupsidecar-secret
                  key: gotify_token
            - name: GOTIFYTOPIC
              value: "S3 Backup Notification"
 ```
 Example Kubernetes Job manifest for running BackupSidecar to restore an S3 bucket:
 ```yaml
 apiVersion: batch/v1
 kind: Job
 metadata:
  name: backupsidecar-s3-restore
  namespace: authentik
 spec:
  backoffLimit: 3
  activeDeadlineSeconds: 600
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: backupsidecar
          image: backupsidecar:latest
          env:
            - name: OPERATION_MODE
              value: "restore"
            - name: BACKUP_MODE
              value: "s3"
            - name: S3_BUCKET
              value: "my-bucket"
            - name: S3_ENDPOINT
              value: "http://minio:9000"
            - name: S3_PREFIX
              value: "data" # optional
            - name: RESTORE_SNAPSHOT_ID
              value: "abc123def456" # optional, defaults to latest
            - name: MINIO_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: minio-secret
                  key: access_key
            - name: MINIO_SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: minio-secret
                  key: secret_key
            - name: RESTIC_REPOSITORY
              value: "rest:http://rest-server:8000/backup"
            - name: RESTIC_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: backupsidecar-secret
                  key: restic_password
            - name: GOTIFYHOST
              value: "http://gotify.example.com"
            - name: GOTIFYTOKEN
              valueFrom:
                secretKeyRef:
                  name: backupsidecar-secret
                  key: gotify_token
            - name: GOTIFYTOPIC
              value: "S3 Restore Notification"
 ```
 ## Notifications
 The script can send success or failure notifications via Gotify when enabled. To enable notifications, set `ENABLE_GOTIFY=true` and provide the required Gotify configuration variables (`GOTIFYHOST`, `GOTIFYTOKEN`, `GOTIFYTOPIC`). When notifications are disabled, backup status messages are still logged to the console.
--- a/TESTING.md
+++ b/TESTING.md
@@ -1,354 +0,0 @@
 # Backup Script Testing Guide
 This document provides step-by-step instructions for testing the backup script locally using Docker Compose with a Restic server and PostgreSQL database.
 ## Prerequisites
 - Docker and Docker Compose installed
 - The backup script (`src/backup.sh`) and its dependencies (restic, curl, jq, pg_dump, psql)
 - Basic understanding of bash and PostgreSQL
 ## Setup
 ### 1. Start the Test Environment
 ```bash
 # Start the services
 docker-compose up -d
 # Wait for services to be ready (about 10-15 seconds)
 sleep 15
 # Verify services are running
 docker-compose ps
 ```
 ### 3. Initialize Restic Repositories
 Before running any backup operations, you need to initialize the restic repositories:
 ```bash
 # Initialize repository for directory backups
 RESTIC_PASSWORD="testpassword123" restic -r "rest:http://localhost:8000/test-backup" init
 # Initialize repository for PostgreSQL backups
 RESTIC_PASSWORD="testpassword123" restic -r "rest:http://localhost:8000/postgres-backup" init
 ```
 **Note**: The repositories only need to be initialized once. If you recreate the restic server container, you'll need to reinitialize the repositories.
 ## Directory Backup Testing
 This section tests the directory backup and restore functionality.
 ### 1. Prepare Test Data
 ```bash
 # Create a test directory with sample files
 mkdir -p test_data
 cd test_data
 # Create various types of files
 echo "This is a text file" > sample.txt
 echo '{"name": "test", "value": 123}' > data.json
 mkdir -p subdir
 echo "Nested file content" > subdir/nested.txt
 echo "Binary data" | base64 > binary.dat
 # Create some larger files for testing
 dd if=/dev/urandom of=large_file.bin bs=1M count=10 2>/dev/null
 # Go back to project root
 cd ..
 ```
 ### 2. Perform Directory Backup
 ```bash
 # Set environment variables for directory backup
 export OPERATION_MODE="backup"
 export BACKUP_MODE="directory"
 export RESTIC_PASSWORD="testpassword123"
 export RESTIC_REPOSITORY="rest:http://localhost:8000/test-backup"
 export SOURCEDIR="/tmp/test-data"
 export ENABLE_GOTIFY="false"
 # Run the backup
 ./src/backup.sh
 ```
 ### 3. Verify Backup
 ```bash
 # List snapshots
 restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")
 # Check backup contents
 restic -r "rest:http://localhost:8000/test-backup" ls latest --password-file <(echo "testpassword123")
 ```
 ### 4. Perform Directory Restore
 ```bash
 # Create restore directory
 RESTOR_DIR="/tmp/restored_data"
 mkdir -p $RESTOR_DIR
 # Set environment variables for directory restore
 export OPERATION_MODE="restore"
 export RESTOREDIR=$RESTOR_DIR
 export RESTORE_SNAPSHOT_ID="latest"
 # Run the restore
 ./src/backup.sh
 ```
 ### 5. Verify Directory Restore
 ```bash
 # Compare original and restored directories
 diff -r /tmp/test-data $RESTOR_DIR
 # Check file contents
 echo "Original file:"
 cat test_data/sample.txt
 echo "Restored file:"
 cat restored_data/sample.txt
 # Verify binary file integrity
 md5sum test_data/large_file.bin
 md5sum restored_data/large_file.bin
 # Check directory structure
 tree test_data
 tree restored_data
 ```
 ### 6. Cleanup Directory Test
 ```bash
 # Remove test directories
 rm -rf test_data restored_data
 ```
 ## PostgreSQL Backup Testing
 This section tests the PostgreSQL database backup and restore functionality.
 ### 1. Generate Test Data
 ```bash
 # Generate test data in PostgreSQL
 ./generate_test_data.sh
 ```
 ### 2. Verify Initial Data
 ```bash
 # Check that data exists
 docker exec postgres-test psql -U testuser -d testdb -c "
 SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
 UNION ALL
 SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
 "
 ```
 ### 3. Perform PostgreSQL Backup
 ```bash
 # Set environment variables for PostgreSQL backup
 export OPERATION_MODE="backup"
 export BACKUP_MODE="postgres"
 export RESTIC_PASSWORD="testpassword123"
 export RESTIC_REPOSITORY="rest:http://localhost:8000/postgres-backup"
 export PGHOST="localhost"
 export PGPORT="5432"
 export PGDATABASE="testdb"
 export PGUSER="testuser"
 export PGPASSWORD="testpass"
 export ENABLE_GOTIFY="false"
 # Run the backup
 ./src/backup.sh
 ```
 ### 4. Verify PostgreSQL Backup
 ```bash
 # List snapshots
 restic -r "rest:http://localhost:8000/postgres-backup" snapshots --password-file <(echo "testpassword123")
 # Check backup contents
 restic -r "rest:http://localhost:8000/postgres-backup" ls latest --password-file <(echo "testpassword123")
 ```
 ### 5. Clear the Database
 ```bash
 # Drop all tables to simulate data loss
 docker exec postgres-test psql -U testuser -d testdb -c "
 DROP TABLE IF EXISTS orders CASCADE;
 DROP TABLE IF EXISTS customers CASCADE;
 "
 # Verify database is empty
 docker exec postgres-test psql -U testuser -d testdb -c "
 SELECT table_name FROM information_schema.tables
 WHERE table_schema = 'public';
 "
 ```
 ### 6. Perform PostgreSQL Restore
 ```bash
 # Set environment variables for PostgreSQL restore
 export OPERATION_MODE="restore"
 export RESTORE_SNAPSHOT_ID="latest"
 # Run the restore
 ./src/backup.sh
 ```
 ### 7. Verify PostgreSQL Restore
 ```bash
 # Check that data has been restored
 docker exec postgres-test psql -U testuser -d testdb -c "
 SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
 UNION ALL
 SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
 "
 # Verify data integrity
 docker exec postgres-test psql -U testuser -d testdb -c "
 SELECT
    c.name as customer_name,
    COUNT(o.id) as order_count,
    SUM(o.price * o.quantity) as total_spent
 FROM customers c
 LEFT JOIN orders o ON c.id = o.customer_id
 GROUP BY c.id, c.name
 ORDER BY total_spent DESC
 LIMIT 5;
 "
 # Check foreign key constraints
 docker exec postgres-test psql -U testuser -d testdb -c "
 SELECT
    tc.constraint_name,
    tc.table_name,
    kcu.column_name,
    ccu.table_name AS foreign_table_name,
    ccu.column_name AS foreign_column_name
 FROM information_schema.table_constraints AS tc
 JOIN information_schema.key_column_usage AS kcu
    ON tc.constraint_name = kcu.constraint_name
    AND tc.table_schema = kcu.table_schema
 JOIN information_schema.constraint_column_usage AS ccu
    ON ccu.constraint_name = tc.constraint_name
    AND ccu.table_schema = tc.table_schema
 WHERE tc.constraint_type = 'FOREIGN KEY'
    AND tc.table_name IN ('customers', 'orders');
 "
 ```
 ## Advanced Testing Scenarios
 ### 1. Test with Different Snapshot IDs
 ```bash
 # List all snapshots
 restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")
 # Restore a specific snapshot
 export RESTORE_SNAPSHOT_ID="<snapshot-id>"
 ./src/backup.sh
 ```
 ### 2. Test Error Handling
 ```bash
 # Test with invalid repository
 export RESTIC_REPOSITORY="rest:http://localhost:8000/nonexistent"
 ./src/backup.sh
 # Test with wrong password
 export RESTIC_PASSWORD="wrongpassword"
 ./src/backup.sh
 ```
 ### 3. Test with Gotify Notifications (Optional)
 ```bash
 # If you have a Gotify server running
 export ENABLE_GOTIFY="true"
 export GOTIFYHOST="http://your-gotify-server:80"
 export GOTIFYTOKEN="your-token"
 export GOTIFYTOPIC="Backup Test"
 ./src/backup.sh
 ```
 ## Cleanup
 ### 1. Stop Services
 ```bash
 # Stop and remove containers
 docker-compose down
 # Remove volumes (optional - this will delete all data)
 docker-compose down -v
 ```
 ### 2. Clean Up Test Files
 ```bash
 # Remove any remaining test files
 rm -rf test_data restored_data
 ```
 ## Troubleshooting
 ### Common Issues
 1. **Connection refused to restic server**: Wait a bit longer for the container to start up
 2. **PostgreSQL connection failed**: Ensure the database container is fully initialized
 3. **Permission denied**: Make sure the backup script is executable (`chmod +x src/backup.sh`)
 4. **Restic repository not found**: Check that the repository URL is correct and the server is running
 5. **Script exits early with no output**: The restic repository hasn't been initialized yet. Run the initialization commands in step 3 above.
 ### Debug Commands
 ```bash
 # Check container logs
 docker-compose logs restic-server
 docker-compose logs postgres
 # Test restic connectivity
 restic -r "rest:http://localhost:8000/test-backup" snapshots --password-file <(echo "testpassword123")
 # Test PostgreSQL connectivity
 docker exec postgres-test psql -U testuser -d testdb -c "SELECT 1;"
 ```
 ## Expected Results
 ### Directory Backup Test
 - ✅ Backup completes successfully
 - ✅ Files are backed up to restic repository
 - ✅ Restore completes successfully
 - ✅ Restored files match original files exactly
 - ✅ Directory structure is preserved
 ### PostgreSQL Backup Test
 - ✅ Database backup completes successfully
 - ✅ Database dump is backed up to restic repository
 - ✅ Database can be cleared successfully
 - ✅ Database restore completes successfully
 - ✅ All data is restored correctly
 - ✅ Foreign key relationships are maintained
 - ✅ Data integrity is preserved
 This testing procedure ensures that both directory and PostgreSQL backup/restore functionality works correctly and can be used as a foundation for automated testing in CI/CD pipelines.
--- a/compose.yaml
+++ b/compose.yaml
@@ -1,19 +0,0 @@
 services:
  restic-server:
    image: restic/rest-server:latest
    container_name: restic-server
    ports:
      - "8000:8000"
    command: rest-server --no-auth --path /data
    restart: unless-stopped
  postgres:
    image: postgres:17
    container_name: postgres-test
    ports:
      - "5432:5432"
    environment:
      POSTGRES_DB: testdb
      POSTGRES_USER: testuser
      POSTGRES_PASSWORD: testpass
    restart: unless-stopped
--- a/generate_dir_test_data.sh
+++ b/generate_dir_test_data.sh
@@ -1,51 +0,0 @@
 #!/bin/bash
 set -euo pipefail
 # Script to generate test data for Directory backup testing
 # This script creates a few directories with a few files and directories in each and populates it with test data
 # Create base test directory
 TEST_DIR="/tmp/test-data"
 echo "Creating test directory structure in $TEST_DIR..."
 # Remove existing test directory if it exists
 rm -rf "$TEST_DIR"
 mkdir -p "$TEST_DIR"
 # Create various subdirectories
 mkdir -p "$TEST_DIR/documents/reports"
 mkdir -p "$TEST_DIR/documents/contracts" 
 mkdir -p "$TEST_DIR/data/logs"
 mkdir -p "$TEST_DIR/data/backups"
 # Create text files with content
 echo "This is the annual report for 2023" > "$TEST_DIR/documents/reports/annual_2023.txt"
 echo "Q4 financial summary" > "$TEST_DIR/documents/reports/q4_summary.txt"
 echo "Contract terms and conditions" > "$TEST_DIR/documents/contracts/agreement.txt"
 # Create JSON files
 cat << 'EOF' > "$TEST_DIR/data/config.json"
 {
  "app_name": "TestApp",
  "version": "1.0.0",
  "settings": {
    "debug": true,
    "max_retries": 3,
    "timeout": 30
  }
 }
 EOF
 # Create some log files
 for i in {1..3}; do
  echo "$(date) - Log entry $i" >> "$TEST_DIR/data/logs/app.log"
  echo "$(date) - Error $i: Sample error message" >> "$TEST_DIR/data/logs/error.log"
 done
 # Create symbolic links
 ln -s "../reports/annual_2023.txt" "$TEST_DIR/documents/contracts/report_link"
 ln -s "../../data/config.json" "$TEST_DIR/documents/reports/config_link"
 echo "Test data generation completed successfully!"
 echo "Created directory structure:"
 tree "$TEST_DIR"
--- a/generate_sql_test_data.sh
+++ b/generate_sql_test_data.sh
@@ -1,124 +0,0 @@
 #!/bin/bash
 set -euo pipefail
 # Script to generate test data for PostgreSQL backup testing
 # This script creates two tables with a foreign key relationship and populates them with test data
 # Database connection parameters
 PGHOST="${PGHOST:-localhost}"
 PGPORT="${PGPORT:-5432}"
 PGDATABASE="${PGDATABASE:-testdb}"
 PGUSER="${PGUSER:-testuser}"
 PGPASSWORD="${PGPASSWORD:-testpass}"
 # Export password for psql
 export PGPASSWORD
 echo "Generating test data for PostgreSQL database..."
 # Create tables
 echo "Creating tables..."
 psql -h "$PGHOST" -p "$PGPORT" -U "$PGUSER" -d "$PGDATABASE" << 'EOF'
 -- Drop tables if they exist
 DROP TABLE IF EXISTS orders CASCADE;
 DROP TABLE IF EXISTS customers CASCADE;
 -- Create customers table
 CREATE TABLE customers (
    id SERIAL PRIMARY KEY,
    name VARCHAR(100) NOT NULL,
    email VARCHAR(100) UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
 );
 -- Create orders table with foreign key to customers
 CREATE TABLE orders (
    id SERIAL PRIMARY KEY,
    customer_id INTEGER NOT NULL REFERENCES customers(id) ON DELETE CASCADE,
    product_name VARCHAR(100) NOT NULL,
    quantity INTEGER NOT NULL CHECK (quantity > 0),
    price DECIMAL(10,2) NOT NULL CHECK (price >= 0),
    order_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP
 );
 -- Create indexes for better performance
 CREATE INDEX idx_orders_customer_id ON orders(customer_id);
 CREATE INDEX idx_customers_email ON customers(email);
 CREATE INDEX idx_orders_order_date ON orders(order_date);
 EOF
 # Insert test data
 echo "Inserting test data..."
 psql -h "$PGHOST" -p "$PGPORT" -U "$PGUSER" -d "$PGDATABASE" << 'EOF'
 -- Insert customers
 INSERT INTO customers (name, email) VALUES
    ('John Doe', 'john.doe@example.com'),
    ('Jane Smith', 'jane.smith@example.com'),
    ('Bob Johnson', 'bob.johnson@example.com'),
    ('Alice Brown', 'alice.brown@example.com'),
    ('Charlie Wilson', 'charlie.wilson@example.com'),
    ('Diana Davis', 'diana.davis@example.com'),
    ('Eve Miller', 'eve.miller@example.com'),
    ('Frank Garcia', 'frank.garcia@example.com'),
    ('Grace Lee', 'grace.lee@example.com'),
    ('Henry Taylor', 'henry.taylor@example.com');
 -- Insert orders
 INSERT INTO orders (customer_id, product_name, quantity, price) VALUES
    (1, 'Laptop', 1, 999.99),
    (1, 'Mouse', 2, 25.50),
    (2, 'Keyboard', 1, 75.00),
    (2, 'Monitor', 1, 299.99),
    (3, 'Headphones', 1, 150.00),
    (3, 'Webcam', 1, 89.99),
    (4, 'Tablet', 1, 399.99),
    (4, 'Stylus', 1, 49.99),
    (5, 'Smartphone', 1, 699.99),
    (5, 'Case', 1, 19.99),
    (6, 'Desktop', 1, 1299.99),
    (6, 'RAM', 2, 79.99),
    (7, 'SSD', 1, 199.99),
    (7, 'Graphics Card', 1, 599.99),
    (8, 'Motherboard', 1, 199.99),
    (8, 'CPU', 1, 399.99),
    (9, 'Power Supply', 1, 149.99),
    (9, 'Cooling Fan', 2, 29.99),
    (10, 'Cable Set', 1, 39.99),
    (10, 'USB Hub', 1, 24.99);
 EOF
 # Verify data
 echo "Verifying test data..."
 psql -h "$PGHOST" -p "$PGPORT" -U "$PGUSER" -d "$PGDATABASE" << 'EOF'
 -- Show table counts
 SELECT 'customers' as table_name, COUNT(*) as row_count FROM customers
 UNION ALL
 SELECT 'orders' as table_name, COUNT(*) as row_count FROM orders;
 -- Show sample data
 SELECT 'Sample customers:' as info;
 SELECT id, name, email FROM customers LIMIT 5;
 SELECT 'Sample orders:' as info;
 SELECT o.id, c.name as customer_name, o.product_name, o.quantity, o.price 
 FROM orders o 
 JOIN customers c ON o.customer_id = c.id 
 LIMIT 5;
 -- Show foreign key relationship
 SELECT 'Foreign key relationship check:' as info;
 SELECT 
    c.name as customer_name,
    COUNT(o.id) as order_count,
    SUM(o.price * o.quantity) as total_spent
 FROM customers c
 LEFT JOIN orders o ON c.id = o.customer_id
 GROUP BY c.id, c.name
 ORDER BY total_spent DESC;
 EOF
 echo "Test data generation completed successfully!"
 echo "Database contains:"
 echo "- 10 customers"
 echo "- 20 orders with foreign key relationships"
 echo "- Various data types and constraints"
--- a/src/backup.sh
+++ b/src/backup.sh
@@ -24,7 +24,7 @@ OPERATION_MODE="${OPERATION_MODE:-backup}"
 #######################################
 # Determine backup mode from the environment only.
-# Valid values: "directory" or "postgres".
+# Valid values: "directory", "postgres", or "s3".
 # Default to "directory" if not provided.
 #######################################
 BACKUP_MODE="${BACKUP_MODE:-directory}"
@@ -39,6 +39,8 @@ if [ "$BACKUP_MODE" = "postgres" ]; then
    elif [ "$OPERATION_MODE" = "restore" ]; then
        REQUIRED_CMDS+=(psql)
    fi
 elif [ "$BACKUP_MODE" = "s3" ]; then
    REQUIRED_CMDS+=(mc)
 fi
 for cmd in "${REQUIRED_CMDS[@]}"; do
@@ -107,8 +109,16 @@ case "$BACKUP_MODE" in
            fi
        fi
        ;;
    s3)
        : "${S3_BUCKET:?Environment variable S3_BUCKET is not set (required for S3 mode)}"
        : "${S3_ENDPOINT:?Environment variable S3_ENDPOINT is not set (required for S3 mode)}"
        : "${MINIO_ACCESS_KEY:?Environment variable MINIO_ACCESS_KEY is not set (required for S3 mode)}"
        : "${MINIO_SECRET_KEY:?Environment variable MINIO_SECRET_KEY is not set (required for S3 mode)}"
        # Optional: S3 path prefix
        : "${S3_PREFIX:=}"
        ;;
    *)
-        echo "Error: Unknown backup mode '$BACKUP_MODE'. Valid modes are 'directory' and 'postgres'." >&2
+        echo "Error: Unknown backup mode '$BACKUP_MODE'. Valid modes are 'directory', 'postgres', and 's3'." >&2
        exit 1
        ;;
 esac
@@ -287,6 +297,91 @@ restore_postgres() {
    fi
 }
 #######################################
 # Backup an S3 bucket.
 # Syncs the S3 bucket to a temporary directory and then backs it up.
 #######################################
 backup_s3() {
    log "Starting S3 backup for bucket '${S3_BUCKET}' at endpoint '${S3_ENDPOINT}'"
    # Create a temporary directory for the S3 sync.
    TEMP_BACKUP_DIR=$(mktemp -d)
    log "Created temporary directory: ${TEMP_BACKUP_DIR}"
    # Configure MinIO Client alias
    local alias_name="backupsidecar"
    if ! mc alias set "${alias_name}" "${S3_ENDPOINT}" "${MINIO_ACCESS_KEY}" "${MINIO_SECRET_KEY}"; then
        local msg="Failed to configure MinIO client alias"
        log "$msg"
        send_notification "$msg"
        exit 1
    fi
    # Build S3 path
    local s3_path="${alias_name}/${S3_BUCKET}"
    if [ -n "${S3_PREFIX}" ]; then
        s3_path="${s3_path}/${S3_PREFIX}"
    fi
    log "Syncing S3 bucket from ${s3_path} to ${TEMP_BACKUP_DIR}..."
    if mc mirror "${s3_path}" "${TEMP_BACKUP_DIR}" --remove; then
        log "S3 sync completed successfully."
    else
        local exit_code=$?
        local msg="S3 sync failed with error code ${exit_code}"
        log "$msg"
        send_notification "$msg"
        exit "$exit_code"
    fi
    # Back up the directory containing the S3 content.
    run_restic_backup "${TEMP_BACKUP_DIR}"
 }
 #######################################
 # Restore an S3 bucket.
 # Restores the S3 content from the backup and syncs it back to S3.
 #######################################
 restore_s3() {
    local snapshot_id="${RESTORE_SNAPSHOT_ID:-latest}"
    log "Starting S3 restore for bucket '${S3_BUCKET}' at endpoint '${S3_ENDPOINT}'"
    # Create a temporary directory for the restore.
    TEMP_RESTORE_DIR=$(mktemp -d)
    log "Created temporary directory: ${TEMP_RESTORE_DIR}"
    # Restore the backup to the temporary directory
    run_restic_restore "${TEMP_RESTORE_DIR}" "${snapshot_id}"
    # Configure MinIO Client alias
    local alias_name="backupsidecar"
    if ! mc alias set "${alias_name}" "${S3_ENDPOINT}" "${MINIO_ACCESS_KEY}" "${MINIO_SECRET_KEY}"; then
        local msg="Failed to configure MinIO client alias"
        log "$msg"
        send_notification "$msg"
        exit 1
    fi
    # Build S3 path
    local s3_path="${alias_name}/${S3_BUCKET}"
    if [ -n "${S3_PREFIX}" ]; then
        s3_path="${s3_path}/${S3_PREFIX}"
    fi
    log "Syncing restored content from ${TEMP_RESTORE_DIR} to ${s3_path}..."
    if mc mirror "${TEMP_RESTORE_DIR}" "${s3_path}" --remove; then
        local msg="S3 restore completed successfully"
        log "$msg"
        send_notification "$msg"
    else
        local exit_code=$?
        local msg="S3 restore failed with error code ${exit_code}"
        log "$msg"
        send_notification "$msg"
        exit "$exit_code"
    fi
 }
 #######################################
 # Cleanup temporary resources.
 #######################################
@@ -315,6 +410,9 @@ main() {
                postgres)
                    backup_postgres
                    ;;
                s3)
                    backup_s3
                    ;;
            esac
            ;;
        restore)
@@ -325,6 +423,9 @@ main() {
                postgres)
                    restore_postgres
                    ;;
                s3)
                    restore_s3
                    ;;
            esac
            ;;
    esac
Author	SHA1	Message	Date
Timo Behrendt	5a25eca929	sync All checks were successful CI / Check syntax (pull_request) Successful in 10s Details CI / Build Docker image (pull_request) Successful in 21s Details	2025-09-06 08:24:06 +02:00
Timo Behrendt	afc568a4e9	ci: add bash syntax check (#18 ) All checks were successful CD / Check changes (push) Successful in 6s Details CD / Create tag (push) Successful in 14s Details CD / Build and push (amd64) (push) Successful in 23s Details CD / Build and push (arm64) (push) Successful in 1m14s Details CD / Create manifest (push) Successful in 8s Details Reviewed-on: #18 Co-authored-by: Timo Behrendt <t.behrendt@t00n.de> Co-committed-by: Timo Behrendt <t.behrendt@t00n.de>	2025-09-04 20:48:13 +02:00
Timo Behrendt	73ce57b122	feat: add restore functionality (#12 ) All checks were successful CD / Check changes (push) Successful in 14s Details CD / Create tag (push) Successful in 11s Details CD / Build and push (amd64) (push) Successful in 26s Details CD / Build and push (arm64) (push) Successful in 1m14s Details CI / Build Docker image (pull_request) Successful in 18s Details CD / Create manifest (push) Successful in 25s Details Reviewed-on: #12 Co-authored-by: Timo Behrendt <t.behrendt@t00n.de> Co-committed-by: Timo Behrendt <t.behrendt@t00n.de>	2025-09-04 20:36:40 +02:00