GitLab Runner is an application that works with GitLab CI/CD Continues Integration/Continuous Deployment method. Once GitLab Runner is installed and registered, it can run jobs in a pipeline. It’s an open source and written in Go language.
For more information about GitLab Runner, refer to GitLab’s official documentations.
AWS Fargate is a serverless technology that you can use with Amazon ECS to run containers without managing servers or clusters of AWS EC2 instances.
For more information about AWS Fargate, refer to AWS’s official documentations.
If you have a team of engineers working on several projects and they are heavily utilizing your GitLab runner, there might be limitations with the numbers of the jobs running concurrently. Majority of the time, engineers would have to wait in line for the runner to complete a task prior to executing the next task in the pipeline. AWS Fargate custom executer driver for GitLab Runner comes very handy to resolve this dilemma. It will automatically autoscale and launch containers on AWS ECS to execute each GitLab task without negatively affect the performance. This solution generates an efficient and cost effective scaling capacity.
GitLab has a well documented process of using and implementing the AWS Fargate custom executor driver. The purpose of this tutorial is to simplify the process.
Note, GitLab Runner provides several options for different types of executors based on different scenarios. For this tutorial, I have chosen to work with the AWS Fargate executor driver; however, you may find a different executor that suites your needs best.
As of writing of this tutorial (2/27/22), AWS Fargate does not support running containers in privileged mode. As a result, customers can’t build container images inside Fargate containers as AWS imposes security best practices. To read more about this topic, refer to AWS’s Building container images on Amazon ECS on AWS Fargate article, dated 31st of March, 2021.
Kaniko is a tool to build container images from a Dockerfile inside a container without the need to have privileged mode. Kaniko resolves the AWS Fargate constrained environment of building a container image inside Fargate containers.