An AWS CDK Adventure Part 2: Experimenting with ECS Patterns

Finally, I’ve managed to find some time to write about my experience of further playing with AWS CDK. Christmas time is supposed to be the time to relax and spend with the family, but since there isn’t that much to do in another lockdown, and to get a break from looking after Andre, I’ve decided to share more stuff about AWS CDK. Besides, I’ve now registered to do a lightning talk on it at .NET Milton Keynes, and this post will serve as part of the preparation process.

I’ve been looking at AWS CDK ECS Patterns.

These are architectural models using AWS Elastic Container Service. They include constructs for creating, for instance, Network or Application Load Balanced EC2 and Fargate services, Scheduled Tasks and Cron Jobs. Since I was a little bit familiar with Fargate, I experimented with setting up an Application Load Balanced Fargate Service. I must say, it was quite easy to use the constructs needed to set up this serverless service programmatically. I found the unfamiliarity with tasks and definitions a bit intimidating when doing it from the console; however, writing it in C# gave me more confidence, probably because I was using a familiar programming language.

Some terms useful to know when working with AWS ECS:

Task definition: specifies how containers will run

Cluster: a grouping of tasks

The task I set out to do: deploy a container using an existing image (into this image I baked in my favourite super simple random number generating application).

The only controller in the application looks like this:

I also created a simple Docker file:

I created an ECR Repository:

aws ecr create-repository --repository-name number-generator 

I then navigated into the RandomNumberApp directory and ran the following commands:

docker build -t number-generator .
and
docker run --rm -it -p 8080:80 number-generator (to check that the app runs OK locally in a container)
aws ecr get-login-password | docker login --username AWS --password-stdin {AWS_ACCOUNT_ID}.dkr.ecr.us-east-1.amazonaws.com/number-generator (to log in to the registry 'number-generator' I created earlier)
docker tag {imageID} {AWS_ACCOUNT_ID}.dkr.ecr.{AWS_REGION}.amazonaws.com/number-generator (the whole string after the imageID is the URI of the Elastic Container Registry)
docker push {AWS_ACCOUNT_ID}.dkr.ecr.{AWS_REGION}.amazonaws.com/number-generator

Here is the whole stack to launch a Fargate service:

Some explanation of the above:

I created a new VPC which the ECS cluster would use. I also specified a task definition, and I referenced the repository I created earlier, by using a static “FromRepository” method of the Repository construct. An interesting thing is that while most of the constructs I have worked with have a property of Name in the Props class, a task definition uses the Family property to set the task name, and it is used for multiple revisions of task definitions.

For mapping container and host port, the values have to be equal otherwise when the app is synthesized, it throws the error “Host port must be left out or equal to container port 80 for network mode awsvpc”. awsvpc is the recommended network mode for ECS, and it simplifies container networking and makes the system more secure; when using this mode each task is given the same networking properties as EC2s. More information on Fargate networking here: Fargate task networking – Amazon ECS.

Another issue I had was when there was more than one image in the ECR repository. The line

var repo = Repository.FromRepositoryName(this,  RandomNumGeneratorECR", "number-generator")

picks up the image with the “latest” tag. But what if I wanted to use a different tag of the image, for example v1? Leaving it for myself to investigate…

In addition, I used the AwsLogDriver construct: it sends information from containers to Cloudwatch logs, and I set the stream prefix, which is optional for the log configuration for the containers of the Fargate launch type. Note how required properties for logging are automatically added in the CloudFormation template:

cdk synth, cdk deploy and the service is launched (it takes a while though, because there are a lot of resources to create)!

And when the service is launched, the logs from the container are available:

I also wanted to see if I could deploy into an existing VPC. Theoretically you can by using the line below:

Vpc.FromLookup(this, "MyExistingVPC", new VpcLookupOptions { IsDefault = true });

FromLookup method imports an existing VPC; it uses context from cdk.context.json where some values such as data about your VPCs and Availability Zones are cached.

I had an existing VPC and tried to use that, but I got an error on the ECS task:

At first I thought it was something to do with the container image having been built incorrectly, but the AWS documentation suggests that this error is caused by some missing elements of the VPC for the Fargate service: Resolve the “CannotPullContainerError” in Amazon ECS. So I’m leaving this for now.

Another thing that can facilitate creating a Fargate Service with AWS CDK is importing the Docker image from the file system. For that, you create a folder (on the same level as the src folder with the stack) with the app you’d like to bake into the image, and a Docker file:

And then, you can remove the Repository construct and change the Image property in the ContainerDefinition like so:

This code will create an ECR Repository, create a container image from the Dockerfile and the solution, tag the image “latest” and push the created image into the repository.

I think it’s pretty cool to launch quite a complex service that has a lot of resources behind it just with a few lines of code (the CloudFormation template generated for the example has over 1000 lines). However, in my opinion, you also need to understand what is going to be created by these high-level constructs: the example above creates over 40 resources and some of them are chargeable.

To sum up, I have enjoyed experimenting with this specific ECS pattern, and I hope to do more of it in the future.

References: AWS re:Invent 2019: Infrastructure as .NET with the AWS CDK (WIN310) – YouTube