AWS CodeStar: creating automated CodeBuild test reports

You may have guessed by now that I am so in love with AWS. The next declaration of love is exploring AWS Continuous Integration and Delivery services. Well, actually, there is another reason in addition to being enamoured – I need to have a look at those services in preparation for my AWS Developer Associate Exam, and we are not using any of them at work.

I started with checking out AWS CodeStar; it hasn’t come up that much in practice exams but I decided to explore it anyway. Codestar is a service that provides a mechanism for setting up CI/CD pipeline really quickly, and it also enables you to have an overview of all development activities (i.e. managing a repository, writing code etc). AWS has its own repository service called CodeCommit, but I opted to use a Github repo for this example. I must say it literally took a couple of minutes to set up the delivery chain. CodeStar has a number of templates depending on what type of application you would like to develop. I created a project using a .NET Core Web Service template. A Github repo was automatically created by AWS, with all the files necessary for deployment; it also already included an .NET Core API project and an XUnit test project.

I wrote a super simple API Controller which returns a random number using a service (probably a bit of an overkill for this example though, as it’s possible to write the number-generating code straight in the controller):

Besides, I added a couple of simple unit tests for the controller:

I committed the code into the repo, sat back and watched the deployment of my little app via CodeStar.

CodeStar uses several other AWS Services, such as CodeBuild, CodeDeploy and CodePipeline. Next on my list was to inspect the CodeBuild part of the CodeStar project. AWS CodeBuild compiles code, runs tests and creates deployable packages. As a big fan of testing, I decided to try out something related to that, and set my eyes on the Report group feature of CodeBuild. Basically, it allows you to create reports based on tests run during the build process. To set it up, I added the highlighted bits to the buildspec.yml file (this is a file that tells CodeBuild what to do during the build process):

However, when I committed the changes to my Github repo, I had the following error from CodeBuild:

The CodeStar Toolchain Worker role creates resources for other AWS services, and this role was missing permissions required to run the test report. So I looked for this role in the IAM section of the AWS Console, removed the permission boundaries, created the following policy and attached it to the role:

After that I was able to successfully run the tests and view the test report from the CodeBuild console:

And to test it further, I broke the second test by changing the expected status code from 200 to 400 in the Assert statement, and committed it into the repo. The build failed and the test report showed the following:

Finally, I fixed the tests back to how they were and restricted the policy to only allow working with the specific report group (you can get its ARN from the configuration section of the report group screen):

To sum up, my initial opinion of the AWS Codestar is that it’s very convenient to be able to set up continuous delivery pipeline so fast, but it’s still necessary to understand the underlying services that it uses.