Building a Serverless CI/CD Pipeline on AWS.

IT automation, also known as Infrastructure as Code (IaC), is an intrinsic part of the DevOps culture and best practices. The goal is to guarantee the same environment is created every time the code is executed. This is pivotal to the implementation of Continuous Integration / Continuous Deployment (CI/CD). The purpose of the CI/CD pipeline is to enable teams to release a constant flow of software updates into production to quicken release cycles, lower costs, and reduce the risks associated with development.

This tutorial walks you through the creation of a serverless CI/CD pipeline using AWS CodePipeline and AWS Fargate. Also, Ansible is used for automation and a sample Python application is available to demonstrate the process from start to finish.

Architecture Diagram

Figure 1 shows the high-level architecture to be deployed:

Figure 1 — AWS Serverless Architecture

1. Updates are made to the CodeCommit repository.
2. Automatically triggers the CodeBuild project.
3. The project creates a Docker image in Elastic Container Registry.
4. After the image is created CodePipeline triggers the deploy to ECS Fargate.

Requirements

• The scripts were tested onmacOS Catalinawith Ansible 2.10 and Python 3.8
• AWS CLI installed and configured.
• Export AWS ACCESS KEY and AWS SECRET KEY as environment variables:

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=

• Configure an SSH Key in your IAM credentials. See For SSH Connections on Linux, macOS, or Unix or For SSH Connections on Windows for detailed instructions.
• Install Python library for AWS — boto3— as it’s used to deregister the ECS Task Definitions. Command to install: pip install boto3. See the pip documentation for more information.

Caveats

• Due to issues with Ansible module ecs_taskdefinition I’m using the AWS CLI command to register the task definition and a python script to deregister all task definitions inroles/fargate/tasks/main.yml .

Ansible Repository

You can find the Ansible and Python code in this repository to follow along: https://github.com/rafaelmnatali/aws-cicd-pipeline-python

Pipeline Creation

• Create the CI/CD pipeline with the following command:

ansible-playbook -i inventory cicd-fargate-pipeline.yml — tags create_fargate

The playbook cicd-fargate-pipeline.yml with the create_fargate tag will provision the following resources in AWS:

• AWSCodeCommit git-like repository for source code.
• AWS Elastic Container Registry for Docker images.
• AWS CodeBuild project to create the Docker image.
• IAM Roles and Policies for AWS CodeBuild, AWS CodePipeline, and ECS Fargate.
• ECS Cluster, ECS Service, and ECS Fargate Task Definition.
• CodePipeline to orchestrate the pipeline.

Python Application deployment

After the infrastructure is created, we need to upload the sample code to the AWS CodeCommit repository (Source):

• Clone the repository with the following command:

git clone ssh://git-codecommit.us-west-2.amazonaws.com/v1/repos/python-sample-app

• Copy the source code to the cloned repository:

cp ../python-sample-code/* python-sample-app

• Push the code to the CodeCommit repository:

cd python-sample-app
git add .
git commit -m "initial commit"
git push

The git push will trigger the pipeline to begin. Open the AWS CodePipeline console to follow the execution:

Figure 3 — AWS CodePipeline execution

As soon as the Deploy step is succeeded open the AWS ECS console to retrieve the public IP assigned for the task:

Figure 4 — ECS Task info

Finally, copy the IP in the browser and using the port 8080 and you should see the Hello World message:

Figure 5 — Hello World

Clean Up

• Destroy the CI/CD pipeline with the following command:

ansible-playbook -i inventory cicd-fargate-pipeline.yml --tags destroy_fargate