This is a long overdue post about how my experiments with CDK Pipelines are going. I hope that there will be a few blog posts about my experience with it, depending on how much time I have spare. This particular post is focused on how to set up a Github repo as the code source via CDK Pipelines.
What is CDK Pipelines?
It’s a high level construct library that allows you to build CI/CD pipeline, and it has AWS CodePipeline under the hood. CDK Pipelines is still in developer preview at the time of writing this post, which means that it might not very stable, and breaking changes might be released and which probably explains the issues I ran into while experimenting with CDK Pipelines.
What I tried to do:
I have a simple API where you pass in a date, and it tells you what day of the week it was/will be. I also have a couple of simple unit tests. The code is in a Github repo. I wanted to create an AWS CI/CD pipeline where the code is picked up from this repository.
To build the pipeline, I followed this tutorial: https://cdkworkshop.com/40-dotnet/70-advanced-topics/100-pipelines/4000-build-stage.html, combined with the documentation on CDK Pipelines: https://docs.aws.amazon.com/cdk/latest/guide/cdk_pipeline.html
The first obstacle I hit was that GitHubSourceAction class defines version 1 of GitHub Actions (token-based), which is not recommended for new projects, and if used to create the stack, requires a manual change to GitHub Actions version 2. Firstly, I changed it manually after the initial deployment of the stack, but then realised that every time I deployed the changes and the stack was updated, I needed to make the same change manually again. In addition, after creation it was generating an error:
To resolve this issue, I had to manually add correct permissions to the CodePipeline service role. You might think – I’m not using CodeStar, what does Codestar have to do with this project? Interestingly, CodeStar is used under the hood of GitHub Actions Version 2. A CodeStar connection is needed by CodePipeline to connect to external source providers like GitHub. CodeStar connection is not available as a Level 2 construct, so I thought I could create it using a low-level CloudFormation construct:
This bit me because although the connection was created after running cdk deploy, it couldn’t be used. What I learnt later was that when a CodeStar connection is created via CloudFormation, it’s not automatically updated from pending to available state – you have to change it manually. So, it turned out to be easier to create the connection in the console before deploying the stack and then reference the ARN in the stack. Connections can be created in the https://console.aws.amazon.com/codesuite screen where you can view Developer Tools, under the Settings/Connections menu, or using AWS CLI.
In order to ensure that CodePipeline can access my GitHub repo, a policy had to be added to the CodePipeline service role with like so:
where connectionArn is the ARN of the manually created CodeStar connection. It’s also possible to add the policy programmatically:
After performing these steps, I was able to get the code from my Github repo, and kick off the CodePipeline on repository state change.
Next, instead of GitHubSourceActions I had to use a BitBucketSourceAction construct to create a source to pull code from – it’s used as one of BitBucketSourceActionProps:
What is also worth noting here is that when you create a repository in GitHub, the default name of the branch is ‘main’ rather than ‘master’. However, it hasn’t been changed yet in the CDK, so you have to specify the name if it’s not ‘master’, otherwise CodePipeline will complain that it cannot find the branch:
First impression of CDK Pipelines: there might not be sufficient number of examples just yet as it’s still quite new, and there are some issues that might arise when following available tutorials. However, with a bit of research and experimentation, defining a GitHub repo as the code source using CDK Pipelines is not too hard, and a bonus for me was that I learnt a few interesting bits about underlying AWS services. More to come soon (hopefully) about this powerful construct library!