Dynamic DNS using a HTTP API Gateway and Lambda with Terraform and aws_apigatewayv2_api
Serverless is one of those technologies that sounds really simple, but can be quite tricky to actually implement.
There are so many different parts and sometimes the only error you'll see if you get something wrong is either a 404 or a ValidationException. I also found that a lot of the documentation was aimed towards using REST API, so here is a tutorial for using the HTTP API to implement a Lambda function entirely using Terraform.
The aims of this proof of concept were:
- Create a Dynamic DNS service which could potentially be shared across multiple servers, so using
aws-cliwasn't possible - IAM could only restrict to the entire hosted zone (domain) and not to individual records; - Implement the entire solution without any servers (apart from the ones we pretend aren't running serverless);
- Implement the entire solution without the need to login to AWS Management Console (ie. using Infrastructure as Code from a repository);
- The API should be behind a provided domain name, and not one generated by AWS (as this will change if the infrastructure is ever recreated). The assumption here is that you already have a domain name with a hosted zone on Route53;
- The service should be accessible over HTTPS using a GET request;
- The service should not be accessible on the bare domain unless the Lambda function name is passed (ie. we won't use $default).
Dynamic DNS
I've never liked the term Dynamic DNS - all DNS is dynamic. However, generally a DNS record is updated rarely and we're in control of (or know in advance about) the new value. Really Dynamic DNS means DNS for Dynamic IPs.
There are cases where IP address changes can happen at any time and we cannot know from the outside what the new IP address is, for example in the case of a server behind a home broadband connection, or perhaps an IoT device which only comes online intermittently. These devices will connect to an external Dynamic DNS service to update an A record to their current IP address.
Implementation
Once again, serverless seems really simple in theory but there are quite a lot of separate resources needed to be configured for this:
- The dynamic DNS code itself, written in Python in this case. The majority of this Python code was written by Caleb Hoffman;
- The Lambda function to execute the code;
- The HTTP API Gateway to allow the Lambda code to be accessed over HTTPS;
- AWS Certificate Manager to provide a TLS certificate for my custom domain, for use on the API Gateway above;
- Route53 for the dynamic DNS implementation itself, and for DNS verification of TLS certificates and redirection to the API gateway;
- The IAM roles and policies to allow access between all the above services.
Additionally, the current version has authentication based on hardcoded keys (passed as a variable from Terraform) so adding a new client would require a redeployment. This is fine for a tiny number of users, but the next step would be to create an external key/value store in a database - probably DynamoDB.
The entirely of the implementation, including the Python code, is stored in one Git repository and deployed using Terraform.
To start with, configure a few Terraform variables and set the provider and region:
provider "aws" {
region = "eu-west-1"
}
variable "ddns-hosted-zone" {
type = string
description = "The hosted zone for the DynamicDNS entries"
default = "ENTER_YOUR_HOSTED_ZONE_HERE"
}
variable "ddns-subdomain" {
type = string
description = "The subdomain for DynamicDNS entries, with the leading ."
default = ".mydomain.com"
}
variable "api-hosted-zone" {
type = string
description = "The hosted zone for the API hostname. Might be the same as ddns-hosted-zone"
default = "ENTER_YOUR_HOSTED_ZONE_HERE"
}
variable "api-hostname" {
type = string
description = "The hostname to point to the API gateway"
default = "apigw.mydomain.com"
}Next, we need some IAM permissions to allow everything to connect together:
#####################
# IAM Configuration #
#####################
# The first IAM role is to allow access to Lambda.
# Subsequent permissions need to be attached using aws_iam_role_policy_attachment
resource "aws_iam_role" "ddns_exec" {
name = "lambda-dynamic-dns"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
# Allowing access to Route53 and also to Cloudwatch for writing logs.
resource "aws_iam_policy" "route53_access" {
name = "Route53Access"
policy = <<POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"route53:GetHostedZone",
"route53:ListResourceRecordSets",
"route53:ListHostedZones",
"route53:ChangeResourceRecordSets",
"route53:ListResourceRecordSets",
"route53:GetHostedZoneCount",
"route53:ListHostedZonesByName"
],
"Resource": "arn:aws:route53:::hostedzone/${var.ddns-hosted-zone}"
}
]
}
POLICY
}
resource "aws_iam_role_policy_attachment" "route53_attach" {
role = aws_iam_role.ddns_exec.name
policy_arn = aws_iam_policy.route53_access.arn
}Next, I configured a Lambda function. This takes a template Python file lambda.py.template, injects three variables hostedZone, subdomain, and keys, then zips the injected lambda.py to lambda.zip:
########################
# Lambda Configuration #
########################
# First we need to create the lambda.py from a template file
# This essentially just injects variables from Terraform
resource "local_file" "config_lambda" {
content = templatefile("src/lambda.py.template", {
hostedZone = var.ddns-hosted-zone
subdomain = var.ddns-subdomain
keys = "{'test': '123445', 'test2': '123446'}"
}
)
filename = "src/lambda.py"
}
# Next, zip the file up ready for Lambda.
data "archive_file" "ddns" {
type = "zip"
source_file = "src/lambda.py"
output_path = "src/lambda.zip"
depends_on = [local_file.config_lambda]
}
# Upload it to Lambda as a function
resource "aws_lambda_function" "lambda_ddns" {
function_name = "DynamicDNS"
role = aws_iam_role.ddns_exec.arn
runtime = "python3.8"
filename = "src/lambda.zip"
handler = "lambda.lambda_handler"
# Use the output from archive_file to force regeneration
source_code_hash = data.archive_file.ddns.output_base64sha256
}The Python code is stored in src/lambda.py.template and contains some variables which will be filled in by Python. It takes three parameters hostname, ipaddress (optional - the client IP will be used), and key. If the key matches the expected key it will concatenate hostname with subdomain and UPSERT that Route53 A record.
import boto3
import json
def lambda_handler(event, context):
print(event)
keys = ${keys}
params = event['queryStringParameters']
subdomain = "${subdomain}"
hostname = params.get('hostname')
ipaddress = params.get('ipaddress') or event['requestContext']['identity']['sourceIp']
key = params.get('key')
r53 = boto3.client('route53')
hostedZone = "${hostedZone}"
if not key or key != keys.get(hostname):
return {
'statusCode': 403,
'body': "403 Forbidden"
}
else:
updateRecord(r53, hostname + subdomain, ipaddress, hostedZone)
return {
'statusCode': 200,
'body': "200 OK"
}
def updateRecord(client, hostname, ipaddress, hostedZoneId):
client.change_resource_record_sets(
HostedZoneId = hostedZoneId,
ChangeBatch={
"Comment": "Update record to reflect new IP address",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": hostname,
"Type": "A",
"TTL": 300,
"ResourceRecords": [
{
"Value": ipaddress
}
]
}
}
]
}
)Next, I configure a TLS certificate (along with Route53) to allow my custom domain to be attached to an API Gateway:
#############################################
# Configure an SSL certificate for a domain #
# to attach to the API gateway #
#############################################
resource "aws_acm_certificate" "api" {
domain_name = var.api-hostname
validation_method = "DNS"
}
resource "aws_route53_record" "cert_validation" {
allow_overwrite = true
name = tolist(aws_acm_certificate.api.domain_validation_options)[0].resource_record_name
records = [ tolist(aws_acm_certificate.api.domain_validation_options)[0].resource_record_value ]
type = tolist(aws_acm_certificate.api.domain_validation_options)[0].resource_record_type
zone_id = var.api-hosted-zone
ttl = 60
}
resource "aws_acm_certificate_validation" "cert" {
certificate_arn = aws_acm_certificate.api.arn
validation_record_fqdns = [ aws_route53_record.cert_validation.fqdn ]
}Next is the really fiddly part - configuring an API Gateway behind a custom domain to point to the Lambda function, and also pass on URL parameters without a default route.
First, configure the API Gateway itself with the custom domain name:
resource "aws_apigatewayv2_api" "api" {
name = "ddns"
protocol_type = "HTTP"
}
resource "aws_lambda_permission" "apigw" {
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.lambda_ddns.arn
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api.execution_arn}/*/*"
}
resource "aws_apigatewayv2_domain_name" "api" {
domain_name = var.api-hostname
domain_name_configuration {
certificate_arn = aws_acm_certificate_validation.cert.certificate_arn
endpoint_type = "REGIONAL"
security_policy = "TLS_1_2"
}
}Next, create the stage, route and integration to allow a URL on the above domain to route to the Lambda function:
resource "aws_apigatewayv2_stage" "ddns" {
api_id = aws_apigatewayv2_api.api.id
name = "ddns"
auto_deploy = true
}
resource "aws_apigatewayv2_api_mapping" "ddns" {
api_id = aws_apigatewayv2_api.api.id
domain_name = aws_apigatewayv2_domain_name.api.id
stage = aws_apigatewayv2_stage.ddns.id
}
resource "aws_apigatewayv2_integration" "ddns" {
api_id = aws_apigatewayv2_api.api.id
integration_uri = aws_lambda_function.lambda_ddns.invoke_arn
integration_type = "AWS_PROXY"
integration_method = "POST"
}
resource "aws_apigatewayv2_route" "ddns" {
api_id = aws_apigatewayv2_api.api.id
route_key = "GET /ddns"
target = "integrations/${aws_apigatewayv2_integration.ddns.id}"
}Finally, set the DNS for the custom to domain so that it points to the API Gateway:
resource "aws_route53_record" "api-endpoint" {
name = aws_apigatewayv2_domain_name.api.domain_name
type = "A"
zone_id = var.api-hosted-zone
alias {
name = aws_apigatewayv2_domain_name.api.domain_name_configuration[0].target_domain_name
zone_id = aws_apigatewayv2_domain_name.api.domain_name_configuration[0].hosted_zone_id
evaluate_target_health = false
}
}And that's it. Run terraform init and terraform apply and you should have a functioning dynamic DNS service running on Lambda.
To access it, if your custom API domain is apigw.mydomain.com, with the above keys you'd use: curl -s "https://apigw.mydomain.com/ddns?hostname=test&key=123445"
All code can be accessed via Gitlab here.