Shell Scripting for DevOps Engineers

Even in the era of advanced tools like Ansible and Terraform, shell scripting remains an essential superpower for every systems engineer. Whether writing custom hooks, parsing raw logs, preparing container runtimes, or orchestrating local automation, Bash is the glue that binds DevOps platforms.

Let's master the critical scripting patterns and constructs.

Automation Flow Architecture

A robust automation script is not just a list of commands. It must include input checks, environment verification, exit code tracking, error catching, and logging.

Essential Bash Scripting Patterns

Here is a robust script that runs disk check validations, creates database backups, rotates stale files, and outputs detailed logs:

#!/usr/bin/env bash

# Bash safety settings
set -euo pipefail
IFS=$'\n\t'

# Global variables
LOG_FILE="/var/log/db_backup.log"
BACKUP_DIR="/var/backups/db"
MAX_BACKUP_AGE_DAYS=7

log_message() {
    local -r msg="$1"
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $msg" | tee -a "$LOG_FILE"
}

check_disk_space() {
    local -r threshold=90
    local -r current_use=$(df / | awk 'NR==2 {print $5}' | sed 's/%//')

    if [ "$current_use" -gt "$threshold" ]; then
        log_message "ERROR: High disk usage detected ($current_use%). Aborting."
        exit 1
    fi
}

cleanup_old_backups() {
    log_message "Starting backup rotation..."
    find "$BACKUP_DIR" -type f -name "*.tar.gz" -mtime +$MAX_BACKUP_AGE_DAYS -delete
    log_message "Rotation complete."
}

main() {
    log_message "Initializing backup workflow..."
    mkdir -p "$BACKUP_DIR"
    
    check_disk_space
    
    # Simulate DB backup task
    local -r timestamp=$(date +%Y%m%d_%H%M%S)
    local -r backup_file="$BACKUP_DIR/backup_$timestamp.tar.gz"
    
    log_message "Creating database archive..."
    if touch "$backup_file"; then
        log_message "SUCCESS: Archive written to $backup_file"
        cleanup_old_backups
    else
        log_message "FAIL: Backup failed to write."
        exit 2
    fi
}

# Run the script
main

Crucial Bash Best Practices

1. Set Bash Shell Options: Always include set -euo pipefail at the top of your scripts:
   • -e: Immediately exit if a command fails.
   • -u: Treat unset variables as errors.
   • -o pipefail: Returns the exit status of the last command in a pipeline to fail.
2. Redirecting Outputs: Log outputs cleanly using descriptors:
   • &> /dev/null: Silence both stdout and stderr.
   • 2>&1: Redirect stderr into stdout.
3. Cron Automation: Automate script runs via crontab:

   # Run backup script every day at midnight (cron pattern)
   0 0 * * * /usr/local/bin/backup_db.sh > /var/log/cron_backup.log 2>&1