Deploying containers with the AWS Copilot CLI - Part 1

Introduction

In the current Kubernetes-focused world AWS recently released a preview of the ECS CLI v2 and renamed it to AWS Copilot CLI. Developers that don’t want to be bothered with managing and maintaining a complex container orchestration solution can use the CLI to deploy a containerized workload to ECS Fargate quickly. The AWS Copilot CLI allows you to deploy, monitor and troubleshoot your application, all from within the comforts of your favorite terminal. Under-the-hood the CLI uses AWS Cloudformation to deploy the resources, however all is abstracted away in simple YAML manifest files. At the time of writing the CLI is at version v0.3.0.

In this post, I’ll show you how to deploy an application using this new CLI. I’m going to use the good old Docker voting example app, which consists of multiple services and is perfect for this tutorial.

The AWS Copilot CLI is a tool for developers to create, release and manage production ready containerized applications on Amazon ECS and AWS Fargate. From getting started, pushing to a test environment and releasing to production, Copilot helps you through the entire life of your app development.

The Docker Vote App consist of the following services:

Step 1 - Download & Install

The AWS Copilot CLI is currently available for both macOS and Linux. You can find the download links and installation instructions on the Github project page. If you are on a Mac you can use Homebrew to install the CLI:

brew install aws/tap/copilot-cli

Also make sure you have Docker and the AWS CLI installed. Copilot will use Docker to build and package the application we are going to deploy.

Step 2 - Creating the Application

I start with creating an application. An application in the AWS Copilot CLI is a group of environments (test/prod), services (frontend/backend) and deployment pipelines. For information about the different constructs have a look at the official documentation.

From the example-voting-app folder run the following command to create the application:

$ copilot app init voting-app

Step 3 - Creating the test environment

Next, I create a test environment for the application by running:

$ copilot env init -n test --profile default --default-config

Copilot use the same credentials as the AWS CLI. For simplicity, I’ll use the default profile, but you can also use other profiles. When specifying --default-config the Copilot CLI will create a new VPC with public and private subnets across two availability zones, an Application Load Balancer and an ECS cluster to hold the application services. Also, a Cloud Map Namespace is created, which I will use later on for service discovery.

If you already have a VPC which you would like to use you can import the subnets using the following command:

$ copilot env init -n test --import-vpc-id vpc-099c6112b98cdcf47 \`
  `--import-public-subnets subnet-ef48fb2d2b616161f,subnet-014661ebb7ab8681a \`
  `--import-private-subnets subnet-055fafef48fb3c547,subnet-00c9e76f288363e7f`

Step 4 - Creating and deploying the database service

When deploying services using the AWS Copilot CLI you can choose between two service types, a Load Balanced Web Service or a Backend Service. In short, a Load Balanced Web Service will be internet-facing, and a Backend Service will only be accessible from other services.

You are now ready to create the first service. I start with the backend database service, which uses PostgreSQL to store the votes. In a production-like environment, this is probably not what you want, but that’s for another post. As AWS Copilot CLI requires a Dockerfile to be present, I create a simple definition using the following commands:

$ mkdir db
$ echo "FROM postgres:9.4" > ./db/Dockerfile

Next, run the following command to create the db service. As the Dockerfile does not expose a port, you need to specify the port manually:

$ copilot svc init --name db --svc-type "Backend Service" --dockerfile ./db/Dockerfile --port 5432

Copilot has now created a manifest.yml file for the service in the copilot\db folder, in which you can customize deployment options. First you need to make sure we have access to the database. You can do this by setting the POSTGRES_USER and POSTGRES_PASSWORD environment variables. At the time of writing, the AWS Copilot CLI doesn’t support the creation of secrets, so you have to create a secret in the SSM Parameter Store manually. Make sure to set copilot-environment and copilot-application tags as Copilot uses these to limit access to the secret.

$ aws ssm put-parameter --name /voting-app/test/POSTGRES_PASSWORD --value supersecretpassword --type SecureString --tags Key=copilot-environment,Value=test Key=copilot-application,Value=voting-app
{
    "Version": 1,
    "Tier": "Standard"
}

Next update the copilot\db\manifest.yml file to set the POSTGRES_USER and POSTGRES_PASSWORD environment variable.

variables:
  POSTGRES_USER: postgres

secrets:
  POSTGRES_PASSWORD: /voting-app/test/POSTGRES_PASSWORD  

Note that I add POSTGRES_PASSWORD to the secrets section, where the key is the name of the environment variable and the value is the name of the SSM parameter. You can now deploy the database service by running:

$ copilot svc deploy --name db --env test

After deployment, you can get more details about the status of the service by running:

$ copilot svc status db

You can also view the logs generated by the service by running:

$ copilot svc logs db

Step 5 - Creating and deploying the Redis service

You can now repeat the above steps for the other services. First, create the Redis service. As with the database service, create a simple Dockerfile:

$ mkdir redis
$ echo "FROM redis:alpine" > ./db/Dockerfile

After creating the Dockerfile, initialize and deploy the Redis service, running on port 6379.

$ copilot svc init --name redis --svc-type "Backend Service" --dockerfile ./redis/Dockerfile --port 6379

$ copilot svc deploy --name redis --env test

Step 6 - Creating and deploying the Worker service

You can now create and deploy the worker service, which will fetch votes from Redis and store results in the database. As you might have noticed, the Redis and Database service are available on the following endpoints db.voting-app.local:5432 and redis.voting-app.local:6379. Copilot uses ECS Service Discovery, which in turn uses the AWS Cloud Map to manage the DNS namespace. All Copilot services have access to an environment variable COPILOT_SERVICE_DISCOVERY_ENDPOINT, which the service can use to find other services.

$ copilot svc init --name worker --svc-type "Backend Service" --dockerfile ./worker/Dockerfile --port 8080

The Docker Voting Example App is configured with hardcoded hostnames and connection strings to connect between services, so you need to make some small modifications to the Vote, Worker and Result services. First, update the code for the Worker service, developed in C#. Add the references to the required environment variables in worker/src/Worker/Program.cs and update the connection strings throughout the file. I’ve created a fork of the original repository holding all changes.

See this commit for all changes to the Program.cs file. You also need to update copilot/worker/manifest.yml, so that the service can fetch the PostgreSQL credentials.

variables:
  POSTGRES_USER: postgres

secrets:
  POSTGRES_PASSWORD: /voting-app/test/POSTGRES_PASSWORD  

Once the file is updated, the service is ready for deployment.

$ copilot svc deploy --name worker --env test

Step 7 - Creating and deploying the Vote service

The Vote service will be the first frontend or Load Balanced Web Service. Start with the service initialization. You don’t need to specify a port as vote/Dockerfile already contains the EXPOSE statement, which will automatically be detected by the AWS Copilot CLI.

$ copilot svc init --name vote --svc-type "Load Balanced Web Service" --dockerfile ./vote/Dockerfile

After initialization, update the vote/app.py file so that the service can connect to the Redis service. Refer to this commit for the required changes. You can now deploy the Vote service.

$ copilot svc deploy --name vote --env test

As you can see in the output of the deployment command, the Vote service is now reachable on a public endpoint. Browse to the URL and cast your vote.

Step 8 - Creating and deploying the Result service

To see the results of our poll, create the Result service.

$ copilot svc init --name result --svc-type "Load Balanced Web Service" --dockerfile ./result/Dockerfile

Note that because you already have a Load Balanced Web Service deployed, Copilot will automatically update the path on which the service will be listening.

http:
  # Requests to this path will be forwarded to your service. 
  # To match all requests you can use the "/" path. 
  path: 'result'
  # You can specify a custom health check path. The default is "/"
  # healthcheck: '/'

As Application Load Balancers currently do not support rewrite rules, you have to update the Result service to listen on /result as by default its listens on /. But first, update the copilot/result/manifest.yml so that it contains the database credential environment variables. Also make sure to set the healtcheck parameter in the http section to /result as you will update the service next.

http:
  # Requests to this path will be forwarded to your service. 
  # To match all requests you can use the "/" path. 
  path: 'result'
  # You can specify a custom health check path. The default is "/"
  healthcheck: '/result'

variables:
  POSTGRES_USER: postgres

secrets:
  POSTGRES_PASSWORD: /voting-app/test/POSTGRES_PASSWORD

Update the result/server.js, result/views/app.js and result/views/index.html files to reflect the changes as shown in this commit. I’ve updated the service so that it serves the results from /result.

After all files are updated, deploy the final service.

$ copilot svc deploy --name result --env test

When you now browse to the shown URL (with /result) you should see the results of the poll.

Conclusion

With the Result service deployed, this post comes to an end. All the updated files can found in the copilot/part-1 branch of this repository on Github.

In this post we’ve deployed the Docker Example Vote App to AWS. While still in active development, the AWS Copilot CLI is an excellent tool to quickly and simply deploy existing Docker-based applications. In an upcoming post, I will explore how to integrate other AWS services and use the CLI to create a CI/CD pipelines to deploy our application.

Thank you for reading! I hope you enjoyed this post.

P.S.: If you want to clean up the deployed resources from your AWS account, simply run the following command and replace default with your AWS CLI profile name:

$ copilot app delete --yes --env-profiles test=default

Photo by Franz Harvin Aceituna on Unsplash

comments powered by Disqus