Creating CodeBuild workflows for Terraform on GitHub PRs

Ever find yourself being restricted to not be able to use GitHub Actions on simple Terraform workflows? CodeBuild to the rescue.

Not all organizations can afford to get the whole company to be able to run on GitHub Actions as the CI tool of choice. Here are some of the pros and cons:

Pros:

• Developer Happiness — CI steps are easy to write and maintain
• Reusable templates — It is easy to write reusable workflows that can be shared internally throughout the organization
• When using default GitHub runners for simple jobs, the runners require almost zero maintenance from DevOps team

Cons:

• GitHub Actions runners jobs are charged per minute, rounded from the nearest second up
• It can get expensive for long running jobs, especially with applications or workloads that require a lot of pre-installs
• Restricted to certain size of runners offered by GitHub
• Non-trivial amount of complexity if using self-hosted GitHub Actions runners, including Kubernetes clusters maintenance if to follow the model GitHub is using for using dynamic runners

CodeBuild to GitHub PRs

The key benefit and advantage in using CodeBuild as the CI runner is its cost. You are only charged exactly on what you consume and able to use your own AMIs (if required) to run the workloads.

However, it requires much customization on the buildspec files to get the CI workflow right. This also introduces complexity on the Developer’s perspective to understand how CodeBuild works out of the GitHub ecosystem.

Setting up CodeBuild pipelines configured to the GitHub webhooks requires some effort especially in setting the correct permissions given to the CodeBuild to display the CI results back to the PR as a decoration.

Unique Scenarios

In some organizations, not all developers are given access to sensitive AWS accounts and unable to read the raw CodeBuild logs. Hence it is important that the CodeBuild pipeline itself should send important information back to GitHub PRs as decorations to update Developers on CI status. In certain scenarios these workflows should also send alerts to Slack in terms of CodeBuild pipelines not working up to expectations.

Introducing Terraform Module for setting the above up:

GitHub - thepoppingone/terraform-aws-codebuild-tf-pipelines: Reusable Terraform module for creating…

Example usage:

module "awsacc_env_codebuild" {
   source = "github.com/<user_or_org>/terraform-aws-codebuild-tf-pipelines"

   codebuild_project_name = "test-infra"
   github_tf_apply_path   = "envs/dev/*"

   buildspec_file_repo_git_url = "https://github.com/<user_or_org>/test-infra.git"
   buildspec_tfapply_filepath  = "envs/uat/buildspec-tfapply.yml"
   buildspec_tfplan_filepath   = "envs/uat/buildspec-tfplan.yml"

   tags = {
     "Platform"    = ""
     "Environment" = ""
   }
 }

The above code segment would set up the following:

1. Terraform Plan Pipeline
2. Terraform Apply Pipeline
3. (Optional): Terraform Trivy pipeline

Terraform Plan Resources:

• Creates a CodeBuild pipeline that runs with PowerUser and IAMReadOnlyAccess role [Allows custom created IAM role to be passed]
• Triggers upon PR creation on GitHub
• Webhooks are created to link CodeBuild ot GitHub PR
• For more specific terraform plancommands, a buildspec file is required in the targeted repository path

Terraform Apply Resources:

• Creates a CodeBuild pipeline that runs with Administrator role [Allows custom created IAM role to be passed]
• Triggers upon push to the main branch in GitHub Repo (Configurable in a future version)
• Webhooks are created to link CodeBuild ot GitHub push events
• For more specific terraform applycommands, a buildspec file is required in the targeted repository path

Terraform Trivy Resources:

• Creates a CodeBuild pipeline that runs with AWSCodeBuildDeveloperAccess and CloudWatchLogsFullAccess role [Allows custom created IAM role to be passed]
• Triggers upon PR creation on GitHub
• Webhooks are created to link CodeBuild ot GitHub PR
• For more specific trivy commands, a buildspec file is required in the targeted repository path

More on custom buildspec files in the next article.

This article focuses on the infrastructure aspect on how the CodeBuild and its required related resources for this workflow to work.