How to find Azure IP ranges automatically and add them to IPTables
The IP ranges used by Azure Public Cloud are updated weekly, and the URL to download the JSON file containing them changes too.
I found a couple of scripts online to automate this but they mostly relied on updating the URL manually every week, which I did not want to do. This meant that my first step was to programmatically determine the JSON URL from the download page.
On this page there is a link with the text "If your download does not start after 30 seconds, Click here to download manually" so all we need to do is pull the HTML and extract the link using a regular expression. Modifying the User-Agent sent is also required to stop the script from getting blocked.
Once we have the link, we can simply download the JSON as normal and extract the IPs.
In my case I wanted to allow the IPs through the firewall to MySQL, and clear the old IPs from the firewall without affecting whitelisted IPs. Rather than have the script modify the INPUT chain I decided to create a new chain called Azure and have that flushed then re-populated every run. I also decided to setup the INPUT rules manually to jump to the new chain.
I did not want to allow every Azure IP through - only the ones for the services I want to be able to access the server. In this example I've used AzureActiveDirectory.
The above code will work, though I would recommending adding extra error handling and alerting so that you know quickly if it stops working.