Cloudflare » Automatically Enable/Disable a Waiting Room Based on CPU Load

I’ve been using a modified version of the Cloudflare Under Attack Mode Automation script for some time now. In addition to using that to automate UAM, I’d forked the script and adapted it to enable/disable a queue-all Waiting Room.

While I could’ve added the Cloudflare API calls specific to the Waiting Room into the aforementioned UAM script, I decided to separate the Waiting Room stuff into its own script. By doing this, I was able to easily configure separate thresholds (i.e. CPU loads) for the Waiting Room independent of the UAM limits.

Take note though, this will only really work as intended if you aren’t already using waiting rooms. Cloudflare only gives one by default. I don’t know what the cost is for additional waiting rooms, but you can contact their sales department though to inquire about the cost if needed.

The reason that I wanted to do this in addition to the UAM automation is that UAM is a somewhat nuclear option that affects the entire zone. Even though I may only want it to affect a particular domain within that zone. While I won’t say my exact configuration and thresholds, I will say that I’ve configured the Waiting Room automation with a lower Upper CPU Limit than that of the UAM automation.

My thought process for this was that once the CPU threshold is reached for the Waiting Room, all visitors will immediately start queuing-up. Theoretically, this should start reducing the load on the origin. And once it drops below the Lower CPU Limit, the Waiting Room will be disabled, directing queued visitors along to the site. However, should the CPU load continue to rise even with the queue-all Waiting Room active, the UAM will activate once its Upper CPU Limit has been reached.

If the Waiting Room does its job, and the UAM isn’t needed, it seems as though it’s a more graceful approach than going straight to UAM. UAM interferes with other hostnames and services within the zone that I don’t want affected unless necessary. And, even then, only for the minimalist amount of time required.

Waiting Room

Within your Cloudflare dashboard, navigate to Traffic » Waiting Rooms.

  1. Click the Create button.
  2. Fill-in all the required fields. The important ones being Hostname and Path. For example:
    • Hostname » yourdomain.net
    • Path » /
  3. Click the Next button.
  4. Review your settings and click the Save button.
  5. You should find yourself back at the Traffic » Waiting Rooms overview page.
    • Click the toggle switch under Enabled for the Waiting Room you just created. You want it to be disabled for the moment. It will be automatically activated as needed by the CF Auto WR script on your server.
    • Click the toggle switch under Queue All for the Waiting Room you just created. While this isn’t strictly required to do now, as the CF Auto WR script will enable it via the API when needed, I enabled it from within the GUI all the same.
  6. Click Edit for the Waiting Room you just created.
  7. In the upper-right, you’ll see the Waiting Room ID. Click Copy to copy it to your clipboard. Save this in a text file or somewhere accessible. You’ll need it later-on to configure the CF Auto WR script.

API Token

While you could use modify the script to use the account e-mail, API key, etc. I much prefer API tokens. You can create an API token with only the required permissions granted to it rather than using, for example, a global API key that can do most anything.

Within your Cloudflare dashboard, navigate to My Profile » API Tokens.

  1. Click the Create Token button.
  2. Click the Get started button next to Create Custom Token.
  3. Name the token whatever you like.
  4. Under Permissions, you’ll want to configure two as below:
    • Zone » Waiting Rooms » Edit
    • Zone » Waiting Rooms » Read
  5. Configure the rest of the settings (IP filtering, etc.) as needed.
  6. Click the Continue to summary button.
  7. Review the settings and click the Create Token button.
  8. Click Copy to copy the API token to your clipboard. Save this in a text file or somewhere accessible. You’ll need it later-on to configure the CF Auto WR script.

CF Auto WR Script

You can install the script more-or-less wherever you want on your server. For this tutorial, we’ll assume it’s being installed within the home directory.

Just do a find-and-replace, within the script, to set the right directory/path within the script. I could’ve used a single variable to set the installation location, but honestly, was too lazy to do it at the time. Be sure to adjust the path of any relevant CLI commands as well.

#!/bin/bash
# Cloudflare Auto Waiting Room = CF Auto WR
# version 1.0.0

#config
debug_mode=0 # 1 = true, 0 = false, adds more logging & lets you edit vars to test the script
cf_apitoken=""
cf_zoneid=""
cf_roomid=""
upper_cpu_limit=35 # 10 = 10% load, 20 = 20% load.  Total load, taking into account # of cores
lower_cpu_limit=5
time_limit_before_revert=$((60 * 5)) # 5 minutes by default
#end config

# Functions

install() {
    mkdir /home/cfautowr &>/dev/null

    cat >/home/cfautowr/cfautowr.service <<EOF
[Unit]
Description=Automate Cloudflare Waiting Room

[Service]
ExecStart=/home/cfautowr/cfautowr.sh
User=root
EOF

  cat >/home/cfautowr/cfautowr.timer <<EOF
[Unit]
Description=Automate Cloudflare Waiting Room

[Timer]
AccuracySec=1
OnBootSec=60
OnUnitActiveSec=5

[Install]
WantedBy=timers.target
EOF

    chmod 644 /home/cfautowr/cfautowr.service

    systemctl enable /home/cfautowr/cfautowr.timer
    systemctl enable /home/cfautowr/cfautowr.service
    systemctl start cfautowr.timer

    echo "$(date) - cfautowr - Installed" >>/home/cfautowr/cfautowr.log

    exit
}

uninstall() {
    systemctl stop cfautowr.timer
    systemctl stop cfautowr.service
    systemctl disable cfautowr.timer
    systemctl disable cfautowr.service

    rm /home/cfautowr/cfstatus &>/dev/null
    rm /home/cfautowr/wrdisabledtime &>/dev/null
    rm /home/cfautowr/wrenabledtime &>/dev/null
    rm /home/cfautowr/cfautowr.timer
    rm /home/cfautowr/cfautowr.service

    echo "$(date) - cfautowr - Uninstalled" >>/home/cfautowr/cfautowr.log

    exit
}

disable_wr() {
    curl -X PATCH "https://api.cloudflare.com/client/v4/zones/$cf_zoneid/waiting_rooms/$cf_roomid" \
        -H "Authorization: Bearer $cf_apitoken" \
        -H "Content-Type: application/json" \
        --data '{"suspended":true,"queue_all":true}' &>/dev/null

    # log time
    date +%s >/home/cfautowr/wrdisabledtime

    echo "$(date) - cfautowr - CPU Load: $curr_load - Disabled WR" >>/home/cfautowr/cfautowr.log
}

enable_wr() {
    curl -X PATCH "https://api.cloudflare.com/client/v4/zones/$cf_zoneid/waiting_rooms/$cf_roomid" \
        -H "Authorization: Bearer $cf_apitoken" \
        -H "Content-Type: application/json" \
        --data '{"suspended":false,"queue_all":true}' &>/dev/null

    # log time
    date +%s >/home/cfautowr/wrenabledtime

    echo "$(date) - cfautowr - CPU Load: $curr_load - Enabled WR" >>/home/cfautowr/cfautowr.log
}

get_current_load() {
    currload=$(top -bn1 | grep "Cpu(s)" | sed "s/.*, *\([0-9.]*\)%* id.*/\1/" | awk '{print 100 - $1}')
    currload=$(echo "$currload/1" | bc)

    return $currload
}

get_room_status() {
    curl -X GET "https://api.cloudflare.com/client/v4/zones/$cf_zoneid/waiting_rooms/$cf_roomid" \
        -H "Authorization: Bearer $cf_apitoken" \
        -H "Content-Type: application/json" 2>/dev/null |
        awk -F":" '{ print $2 }' | sed -n 8p | tr -d ' ' | tr -d ',' | tr -d '\n' >/home/cfautowr/cfstatus

    room_status=$(cat /home/cfautowr/cfstatus)

    case $room_status in
    "false")
        return 1
        ;;
    "true")
        return 0
        ;;
    *)
        return 100 # error
        ;;
    esac
}

main() {
    # Get current protection level & load
    get_room_status
    curr_room_status=$?
    get_current_load
    curr_load=$?

    if [ $debug_mode == 1 ]; then
        debug_mode=1 # random inconsequential line needed to hide a dumb shellcheck error
        #edit vars here to debug the script
        #curr_load=5
        #time_limit_before_revert=15
    fi

    # If WR was recently enabled

    if [[ $curr_room_status == 1 ]]; then
        wr_enabled_time=$(cat /home/cfautowr/wrenabledtime)
        currenttime=$(date +%s)
        timediff=$((currenttime - wr_enabled_time))

        # If time limit has not passed do nothing
        if [[ $timediff -lt $time_limit_before_revert ]]; then
            if [ $debug_mode == 1 ]; then
                echo "$(date) - cfautowr - CPU Load: $curr_load - time limit has not passed regardless of CPU - do nothing" >>/home/cfautowr/cfautowr.log
            fi

            exit
        fi

        # If time limit has passed & cpu load has normalized, then disable WR
        if [[ $timediff -gt $time_limit_before_revert && $curr_load -lt $lower_cpu_limit ]]; then
            if [ $debug_mode == 1 ]; then
                echo "$(date) - cfautowr - CPU Load: $curr_load - time limit has passed - CPU Below threshhold" >>/home/cfautowr/cfautowr.log
            fi

            disable_wr

            exit
        fi

        # If time limit has passed & cpu load has not normalized, wait
        if [[ $timediff -gt $time_limit_before_revert && $curr_load -gt $lower_cpu_limit ]]; then
            if [ $debug_mode == 1 ]; then
                echo "$(date) - cfautowr - CPU Load: $curr_load - time limit has passed but CPU above threshhold, waiting out time limit" >>/home/cfautowr/cfautowr.log
            fi
        fi

        exit
    fi

    # If WR is not enabled, continue

    # Enable and Disable WR based on load

    #if load is higher than limit
    if [[ $curr_load -gt $upper_cpu_limit && $curr_room_status == 0 ]]; then
        enable_wr
    #else if load is lower than limit
    elif [[ $curr_load -lt $lower_cpu_limit && $curr_room_status == 1 ]]; then
        disable_wr
    else
        if [ $debug_mode == 1 ]; then
            echo "$(date) - cfautowr - CPU Load: $curr_load - no change necessary" >>/home/cfautowr/cfautowr.log
        fi
    fi
}

# End Functions

# Main -> command line arguments

if [ "$1" = '-install' ]; then
    install

    echo "$(date) - cfautowr - Installed" >>/home/cfautowr/cfautowr.log

    exit
elif [ "$1" = '-uninstall' ]; then
    uninstall

    echo "$(date) - cfautowr - Uninstalled" >>/home/cfautowr/cfautowr.log

    exit
elif [ "$1" = '-enable_wr' ]; then
    echo "$(date) - cfautowr - WR Manually Enabled" >>/home/cfautowr/cfautowr.log

    enable_wr

    exit
elif [ "$1" = '-disable_wr' ]; then
    echo "$(date) - cfautowr - WR Manually Disabled" >>/home/cfautowr/cfautowr.log

    disable_wr

    exit
elif [ -z "$1" ]; then
    main

    exit
else
    echo "cfautowr - Invalid argument"

    exit
fi

Create the above file. Make the necessary configuration (i.e. API token, Waiting Room ID, Zone ID, etc.) changes. And, run the following commands against it:

sudo chown -R root:root /home/cfautowr
sudo chmod 770 /home/cfautowr/cfautowr.sh

Once the script has been created and configured, you can proceed to install it as a service. Run the following command to install its services:

sudo /home/cfautowr/cfautowr.sh -install

Should you ever want to deactivate/uninstall its services, you can run the following command:

sudo /home/cfautowr/cfautowr.sh -uninstall

It can be reinstalled with the -install switch again anytime.

Assuming everything has been configured, installed, setup correctly, your queue-all Cloudflare Waiting Room should now be automagically enabled/disabled on-demand as per your configured CPU load thresholds.

While the script shown here is fully functional, it’s not the exact one that we’re using here. We’re using one with some further modifications and always have our Waiting Room active. Rather than having the Waiting Room activated/suspended via this script, we only have the script enable/disable the queue-all functionality of the Waiting Room.

Author

  • I tried to fix the world, but God wouldn't give me his source code.

    Formerly, CEO and lead developer of a technology company, focusing on the merchant services space. Formerly, of WHMCompleteSolution (WHMCS).

    An avid gamer.

Leave a comment

lexical-absolute
lexical-absolute
lexical-absolute
lexical-absolute
%d bloggers like this: