Clusters of Docker Containers in AWS with GitLab

If you're just starting off with Docker deployment in AWS, read on to learn how to use GitLab to deal with clusters of Docker containers in AWS.

This is meant to be a quick how-to reference for people who are new to Docker deployment in AWS.

Before starting, have a certificate that allows admin access to AWS. This is the *.pem that the web browser offers to download when a key-pair in EC2 is created.

Familiarity with the concepts of AWS/ECS, Docker, and Pipelines is assumed.

It’s also assumed that the app is already in a Docker image.

Without further ado, let’s dive right in.

AWS: Create a Load Balancer

The load balancer will provide a single URL to reach the cluster where containers can be added or removed. This is a manual step that is only necessary to be done once.

Screen of AWS 1 of 2 Screen of AWS 2 of 2

Click on Create Target Group

All containers deployed under this Load Balancer will be associated directly with this Target Group. Create Target Group AWS screen

Fill in this form below. The important thing to note here is the health check settings section: the health check endpoint must actually exist, otherwise, the container will be considered unhealthy and ECS will keep killing and creating it forever. Create Target Group Detail Page

Create the actual Load Balancer

Navigate to Services > EC2 > Load Balancers Load Balancer AWS Page

Fill in the name and port where the Load Balancer will listen to and… Load Balancer Detail Page 1 of 2 …select the availability zones. Load Balancer Detail Page 2 of 2

On the second step of the wizard, there will be a warning about the lack of HTTPS. Just skip it.

Select the default security group. Security Group AWS Page

In Target Group, select Existing Target Group and in Name, type MyTargetGroup, which was created above. Configure Routing AWS Page

Go to the end of the wizard and confirm/save the new Load Balancer.

The address to access the application is under DNS name. Basic Configuration AWS Page

AWS: ECS Initial State

Navigate to Services > EC2 Container Service. Container Service AWS Page

The page should look either this… Wizard AWS Page

…or this: Cluster AWS Page

At this point, it is expected to have no containers running.

GitLab: Configure the Pipelines

The docker-compose.yml referred to in the .gitlab-ci.yml below:

version: '2'
    image: ""
      - "80:8080"

Create the .gitlab-ci.yml File

The environment variables seen below can be configured in Settings > CI/CD Pipelines.

image: docker:latest
  - docker:dind
  - build
  - dockerise
  - deploy
  stage: build
  image: maven:3-jdk-8-alpine
    - cd api
    - mvn clean verify
##### 'artifacts' is the way artifacts can be passed around to the next pipelines #####
      - api/target/api.jar
##### This simply builds and pushes the Docker image #####
  stage: dockerise
    - docker version
    - docker build -t$CI_JOB_ID api/
    - docker build -t api/
##### $DOCKER_LOGIN_KEY can be obtained in GitLab as a Personal Access Token #####
    - docker login -u my_username -p $DOCKER_LOGIN_KEY
    - docker images
    - docker push$CI_JOB_ID
    - docker push
  stage: deploy
  image: python:3-alpine
    AWS_DEFAULT_REGION: "us-east-2"
##### Install the AWS ECS-CLI #####
    - apk add --update curl
    - curl -o /usr/local/bin/ecs-cli
    - chmod +x /usr/local/bin/ecs-cli
##### Configure ECS-CLI, run the container and scale to 2 #####
    - ecs-cli configure --region $AWS_DEFAULT_REGION --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --cluster CI-MyCluster-API
    - ecs-cli up --keypair $AWS_KEY_PAIR --capability-iam --size 2 --instance-type t2.micro --vpc vpc-xxxxxxx --subnets subnet-123abc,subnet-321cba
##### This docker-compose.yml is the one described above #####
    - ecs-cli compose --file api/docker-compose.yml service up --target-group-arn $PROD_TARGET_GROUP_ARN --container-name api --container-port 8080 --role ecsServiceRole
    - ecs-cli compose --file api/docker-compose.yml service scale 2
    name: ci
##### This is the URL seen under 'DNS name' when the LB was created #####
    - master

When the file above is committed to master in GitLab, the pipelines should be automatically triggered: GitLab Pipeline Overview Page

…and by clicking on the pipeline, the console should show the logs. The example below is the CI_deploy logs. CI_deploy logs

Notice the last message from the CLI: “ECS Service has reached a stable state.” In the .gitlab-ci.yml, the requested count was 2.

AWS: ECS Final State

The dashboard will now show some stats about the new cluster. ECS containers overview AWS page

Testing the Cluster

Notice the request is distributed evenly across the available containers in the cluster (in this case 2) by hitting the Load Balancer URL several times. Browser page with container ID 1 of 2 Browser page with container ID 2 of 2

Post migrated from

comments powered by Disqus