Setup AWS WAF with terraform

In my team at Aula, we recently decided to enhance our security configuration so we set up a web application firewall to monitor the requests that we receive through our API gateway. In order to be able to understand the requests better, we are storing the request logs that pass through WAF in an S3 bucket, and then using AWS Athena we query the logs and build reports. This helps us understand better where the requests are coming from, which are the IPs that are hitting our WAF, if bots are scraping our platform and if DDoS attacks happen. By having all this knowledge beforehand we can act and react faster by blocking malicious requests and bots and stopping DDoS attacks that are in progress. We also raise alerts in OpsGenie in case the request limit boundaries are exceeded so that we are informed as soon as something out of the ordinary is happening on our platform.

WAF allows you to add rules that are applied to incoming requests and can block, allow or count those requests. WAF allows you to add its own rules and to build custom rules in addition to what it already provides. We started to look into AWS core rule set and AWS IP reputation list rule group and we understood that they offer a wide number of rules that we would benefit from. The AWS core rule set contains rules that provide protection against the exploitation of a wide range of vulnerabilities, including OWASP such as Cross Site Scripting or Local File Inclusion. The AWS IP reputation list rule group contains rules that are used to block IP addresses associated with bots or other threats.

In our project, we use terraform to create our resources so we will need to write some terraform code to create our WAF instance in AWS, but we decided on a reverse engineering approach where we first build and setup the WAF directly in the AWS interface and then we write the terraform code and applied it.

In this article, I will first describe how to set up WAF directly in the AWS Console and then how to create the instance with terraform.

Setup WAF from the AWS Console

In order to create a new WAF resource from the AWS console you will need to do the following steps:

1. Log in to the AWS Console and go to Services
2. Choose WAF & Shield
3. Go to Web ACLs
4. Choose your region and Create web ACL
5. You will need to go through all the steps to create a new instance for WAF

6. Associate the WAF with the API Gateway: Go to Services -> API Gateway -> select the stage -> Settings -> WAF -> and apply the newly created WAF instance

Once the WAF is created you will already be able to add the rules to it. Also, when applying these rules in production make sure to take some time to count the requests and understand better who is hitting your website and not directly enforce these rules to block.

Now, let’s do all these changes via terraform.

Setup WAF with terraform

1. Create WAF resource:

resource "aws_wafv2_web_acl" "api-waf" {
  name        = "my-api-waf"
  description = "WAF for APIs"
  scope       = "REGIONAL"

  default_action {
    allow {}
  }

  visibility_config {
    cloudwatch_metrics_enabled = true
    metric_name                = "my-api-waf"
    sampled_requests_enabled   = true
  }
}

2. Add AWS-managed rules to WAF resource with action set to count:

resource "aws_wafv2_web_acl" "api-waf" {
  ...
  rule {
    name     = "my-api-aws-managed-rules-common-rule-set"
    priority = 0

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesCommonRuleSet"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "my-api-aws-managed-rules-common-rule-set"
      sampled_requests_enabled   = true
    }
  }
}

3. Add an anonymous IP rule and IP reputation list with action set to allow/block/count:

resource "aws_wafv2_web_acl" "api-waf" {
  ...

  rule {
    name     = "my-api-aws-managed-rules-anonymous-ip-list"
    priority = 1

    override_action {
      count {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesAnonymousIpList"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "my-api-aws-managed-rules-anonymous-ip-list"
      sampled_requests_enabled   = true
    }
  }

  rule {
    name     = "my-api-aws-managed-rules-ip-reputation-list"
    priority = 2

    override_action {
      none {}
    }

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesAmazonIpReputationList"
        vendor_name = "AWS"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "my-api-aws-managed-rules-ip-reputation-list"
      sampled_requests_enabled   = true
    }
  }
}

4. Associate API gateway with WAF:

resource "aws_api_gateway_rest_api" "my-api-gateway" {
  ...
  name = "my-api-gateway"
}

resource "aws_api_gateway_deployment" "my-api-gateway" {
  rest_api_id = aws_api_gateway_rest_api.my-api-gateway.id

  triggers = {
    redeployment = sha1(jsonencode(aws_api_gateway_rest_api.my-api-gateway.body))
  }

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_api_gateway_stage" "test" {
  rest_api_id   = aws_api_gateway_rest_api.my-api-gateway.id
  deployment_id = aws_api_gateway_deployment.my-api-gateway.id
  stage_name    = "test"
}

resource "aws_wafv2_web_acl_association" "api-waf" {
  resource_arn = aws_api_gateway_stage.test.arn
  web_acl_arn  = aws_wafv2_web_acl.api-waf.arn
}

5. Setup Athena:

Athena is an interactive query service that allows you to run SQL queries on data stored in an S3 bucket.

# We need to replace '-' with '_' as '-' is an invalid character for database naming
resource "aws_athena_database" "my_athena" {
  name   = "my_api_athena_wafrules"
  bucket = aws_s3_bucket.aws-waf-logs-bucket.bucket
}

6. Create glue data catalog table:

After setting up Athena we will need to create an AWS Glue Data Catalog table first to store metadata for the AWS logs coming from the S3 bucket that will be used for our queries. Those logs are processed and transformed into metadata and stored in this table. Athena will query the table metadata and create query responses.

resource "aws_glue_catalog_table" "aws_glue_waf_table" {
  name          = "waftable"
  database_name = aws_athena_database.my_athena.id

  table_type = "EXTERNAL_TABLE"

  storage_descriptor {
    location      = "s3://${aws_s3_bucket.aws-waf-logs-bucket.id}/AWSLogs/"
    input_format  = "org.apache.hadoop.mapred.TextInputFormat"
    output_format = "org.apache.hadoop.hive.ql.io.IgnoreKeyTextOutputFormat"

    ser_de_info {
      name                  = "waf-stream"
      serialization_library = "org.openx.data.jsonserde.JsonSerDe"

      parameters = {
        "serialization.format" = 1
      }
    }

    columns {
      name = "action"
      type = "string"
    }

    columns {
      name = "timestamp"
      type = "float"
    }

    columns {
      name = "rulegrouplist"
      type = "array<string>"
    }

    columns {
      name = "httprequest"
      type = "struct<clientip:string,country:string,uri:string,httpmethod:string,requestid:string>"
    }
    
    ...
  }
}

7. Create your own workgroup:

One workgroup enables you to isolate queries from other queries in the same account.

resource "aws_athena_workgroup" "waf-workgroup" {
  name = "my-api-waf-workgroup"

  configuration {
    enforce_workgroup_configuration    = true
    publish_cloudwatch_metrics_enabled = true

    result_configuration {
      output_location = "s3://${aws_s3_bucket.aws-waf-logs-bucket.id}/Queries/"
    }
  }
}

8. Create your own queries and add them to previously created workgroup:

resource "aws_athena_named_query" "requests" {
  name      = "query-1"
  workgroup = aws_athena_workgroup.waf-workgroup.name
  database  = aws_athena_database.my_athena.name
  query     = "SELECT httprequest.clientip, COUNT(*) AS count FROM ${aws_athena_database.my_athena.name}.waftable WHERE from_unixtime(timestamp/1000) > (current_timestamp - interval '3' month) GROUP BY httprequest.clientip, FLOOR(timestamp/(1000*60*5)) ORDER BY count DESC LIMIT 100; "
}

Now when you run this query you will be able to see all the IPs and the number of hits into the WAF in the past 3 months.

Prevent a DDoS attack by enforcing a rate-limit rule

If you want to stop a DDoS attack you will need to set up a rule with a rate-based statement rule where you set the aggregate key type IP and the limit per IP. This rule will allow a limited number of requests per IP in a certain period of time (e.g. 5 minutes) and block everything that’s coming in after that limit from that specific IP that initiated previous requests:

rule {
    name     = "my-api-blanket-rate-limit"

    action {
      block {}
    }

    statement {
      rate_based_statement {
        limit              = 100000
        aggregate_key_type = "IP"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "my-api-blanket-rate-limit"
      sampled_requests_enabled   = true
    }
  }

Before setting up WAF in your project keep in mind that this is a paid service and has a payment plan.

Hope this article will help you set up WAF easier by using terraform.

You can view the GitHub repository that sets up WAF here.

Let me know what you think in the comments below.