How to find Azure IP ranges automatically and add them to IPTables

The IP ranges used by Azure Public Cloud are updated weekly, and the URL to download the JSON file containing them changes too.

I found a couple of scripts online to automate this but they mostly relied on updating the URL manually every week, which I did not want to do. This meant that my first step was to programmatically determine the JSON URL from the download page.

The download page is https://www.microsoft.com/en-gb/download/details.aspx?id=56519 which when clicked through (via 'Download') takes you to https://www.microsoft.com/en-gb/download/confirmation.aspx?id=56519

On this page there is a link with the text "If your download does not start after 30 seconds, Click here to download manually" so all we need to do is pull the HTML and extract the link using a regular expression. Modifying the User-Agent sent is also required to stop the script from getting blocked.

import requests
import re

headers = {
	'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

response = requests.get('https://www.microsoft.com/en-us/download/confirmation.aspx?id=56519', headers=headers)
html = response.text
link = re.search(r'click here to download manually[^>]+href="([^"]+)"', html).group(1)

Python code to get the JSON URL

Once we have the link, we can simply download the JSON as normal and extract the IPs.

response = requests.get(link)
json = response.json()

if json:
    for value in json['values']:
    	for addressPrefix in value['properties']['addressPrefixes']:
        	# Do whatever you want with each IP here

Python code to extract the IPs from the JSON

In my case I wanted to allow the IPs through the firewall to MySQL, and clear the old IPs from the firewall without affecting whitelisted IPs. Rather than have the script modify the INPUT chain I decided to create a new chain called Azure and have that flushed then re-populated every run. I also decided to setup the INPUT rules manually to jump to the new chain.

# iptables -N Azure
# iptables -A INPUT -p tcp --dport 3306 -j Azure
# iptables -A INPUT -p tcp --dport 3306 -j REJECT
# ip6tables -N Azure
# ip6tables -A INPUT -p tcp --dport 3306 -j Azure
# ip6tables -A INPUT -p tcp --dport 3306 -j REJECT

Bash commands to configure IPTables ready for the script

I did not want to allow every Azure IP through - only the ones for the services I want to be able to access the server. In this example I've used AzureActiveDirectory.

import requests
import re
import subprocess

wanted_values = ["AzureActiveDirectory"]
chain = "Azure"

headers = {
	'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
}

response = requests.get('https://www.microsoft.com/en-us/download/confirmation.aspx?id=56519', headers=headers)
html = response.text
link = re.search(r'click here to download manually[^>]+href="([^"]+)"', html).group(1)

response = requests.get(link)
json = response.json()

if json:
    subprocess.run(["/sbin/iptables", f"-F {chain}"])
    subprocess.run(["/sbin/ip6tables", f"-F {chain}"])
    for value in json['values']:
        if value['name'] in wanted_values:
            for addressPrefix in value['properties']['addressPrefixes']:
                if re.search(r':', addressPrefix):
                    subprocess.run(f"/sbin/ip6tables -A {chain} -p tcp --source {addressPrefix} -j ACCEPT", shell=True)
                else:
                    subprocess.run(f"/sbin/iptables -A {chain} -p tcp --source {addressPrefix} -j ACCEPT", shell=True)

Full code in Python

The above code will work, though I would recommending adding extra error handling and alerting so that you know quickly if it stops working.