GitHub Actions Workflow Reusability

8 min readMar 22, 2021

Since their inception in 2018, I have been a huge fan of GitHub Actions. What I enjoyed the most about them was the simplicity of their syntax and the fact it was directly embedded into my version control platform of choice for my personal projects.

Until very recently, I didn’t really have the chance to push their capacities to the limits with complex workflows. This was about to change. Over the last week, since I was the most used to GitHub Actions on the project, I have been asked to implement a CI/CD workflow for a new frontend project. In the following article, I will share the outcomes of my experience of using GitHub Actions in a professional context with multiple environments and the challenges of reusability I faced.

Please take note that this article makes abstraction of whether the desired workflow is good or not, the pertinence of it, and the quality of any technical choices made outside of the CI/CD pipeline implementation itself.

The Context

At my daily workplace, I have been assigned to a new project. Actually, the project itself was going on for couple of months already, but the terms of the contract just drastically imploded. Initially, the project consisted of a backend interfacing multiple third party integrations such as Freebees, Mailchimp, Ueat, and others. The frontend part of the application was developed by another business specialized in SEO. None of this is really relevant, except for the fact that the client just changed its mind and now wanted to have control over the whole stack instead. Thus, we ended up with the whole contract since the client’s new vision was out of the expertise field of the other business. With this boosted contract, my company had to mobilize more resources into the project as a response to the short delivery delay ahead of us.

Before my arrival on the project, the team already set up multiple environments (local, development, and staging). The production environment does not exist yet, but should make its appearance sooner or later. All the work done for the frontend was judged non recuperable (was built using WordPress CMS) and we spin up two brand new React Apps that we labeled admin and client. We used Yarn Workspaces to ease the sharing of reusable parts between both applications. Moreover, we decided to use a monorepo approach by moving the backend into its own subfolder. We also had to extract the infra project (simple Typescript/AWS-CDK project for our IaC) from the backend to the root of the repository.

After some tweaks of the .circleci/config.yml file we succeed to scavenge the CI/CD pipelines already in place for both backend and infra projects. We were finally ready to address the automation of our frontend applications’ integration and deployment.

The Mandate

In our Continuous Integration (CI) pipeline we want to perform the following:

Set up correct node environment
Install dependencies
License compliance
Lint
Test
Build

Likewise for the Continuous Deployment (CD) pipeline:

Retrieve CI workflow’s builds
Fetch secrets from AWS-SSM
Upload build into S3 bucket
Invalidate CloudFront cache

Other requirements are that these steps have to be performed for both admin and client applications and we also want to deploy to both dev and staging environments on the merge of a pull request into the main branch. The development environment has to be fully deployed before we move on to deploy anything on the staging environment. The CI pipeline should be triggered on every push to a pull request based on the main branch. We also want the CI pipeline to be executed before the CD pipeline on merge of a pull request. On top of that, we want more than anything else to respect the Don’t Repeat Yourself (DRY) principle. Last but not least, it is important that these two pipelines be triggered only when needed to avoid loosing time and resources to verify and deploy things that haven’t changed.

Obtained Solution

No matter how simple it is to use GitHub Actions normally, we had a hard time before getting everything on point. One thing that did not help was the lack of concrete examples on the web and the quality of the documentation we went through. GitHub Actions have been evolving really quickly since the last few years and it is not rare that we stumble upon outdated pieces of information.

Nevertheless, we managed to create two workflows satisfying our needs using the following GitHub Actions’ features:

In the following, I included screenshots of both workflows for you to follow along my explanations.

CI Workflow

We first begin by specifying that the CI pipeline should be executed whenever a push is made onto the main branch or on a pull request based on the main branch. We also indicate that this pipeline should only be triggered when the admin and/or the client applications are modified. Then, we leverage the dorny/paths-filter action to build an array of modified package names and set this array as the output of the detect-changes job. We then output this array to a detected-changes.json file and upload it using actions/upload-artifact action to be shared with the CD workflow.

In the lint-test-build job, we first start by telling GitHub that this job depends on the detect-changes job and hence have to be executed sequentially using the needs attribute. We then reuse the outputted array from the detect-changes job to build our strategy matrix. We also take care of specifying the default working directory to be used to execute the subsequent steps. This allow us to only specify once the working directory and override it in the few steps that needs to be executed from somewhere else such as Read .nvmrc and Install Dependencies. Thereafter, we simply checkout the code, set up node environment using the node version specified in the .nvmrc file and try to restore cached dependencies of a previous build based on the hash of the yarn.lock file if any exists or install the whole thing otherwise.

The rest of the workflow only consists in performing the different checks we wanted to as we would normally find in a traditional GitHub Actions CI Workflow (License compliance, Lint, Test, and Build). Take note that we upload both dev and staging builds as workflow artifacts as we did for the detected changes to be shared with our CD workflow.

CD Workflow

In the case of the CD workflow, the trigger consist in the workflow_run webhook event which allows us to make our CD workflow depends on the completion of our CI workflow. Also, we specify that this should only happen on the master branch.

The detect-changes job has the same behaviour than its homonym in the CI workflow, but we fulfill it by downloading the changes detected by the CI. This approach has the advantage of being faster, more error proof to inconsistencies between both workflows, and is also a nice workaround for the limitations of the dorny/paths-filter action which rely on the before attribute of the pull_request event payload which is not present in the workflow_run event. I would like to bring your attention on the fact that we used dawidd6/action-download-artifact instead of actions/download-artifact because the latter does not support cross workflow artifact sharing.

We then have the deploy job, which similarly to the lint-test-build job depends on the detect-changes job. Here again, we used the strategy matrix approach to reuse the same job for multiple environments (dev and staging) and applications (admin and client). The app matrix argument is still provided by the detect-changes job. One important thing to know here is that matrix arguments are always processed in the same order. Since our goal is to fully deploy development environment for both applications before starting to deploy anything in staging, we need to define the env argument before app to obtain the correct execution order (dev-admin, dev-client, staging-admin, and staging-client instead of admin-dev, admin-staging, client-dev, and client-staging). We also limit the parallelization to one to make sure the dev environment deployment will be completed successfully before moving on to the deployment in staging environment. At last, we also define the default working directory as we did in the CI workflow.

The rest of the workflow is not really relevant and is more tailored to our specific needs of deploying our applications into S3 buckets fronted by AWS CloudFront and the way we programmatically store infrastructure parameters into AWS SSM. I still present it for the sake of fulfilling our initial statement.

Conclusion

In conclusion, we had some hard time putting all the pieces of the puzzle together with the overflow of documentation on each individual features of GitHub Actions in addition to the predominant presence of outdated information in the forums we consulted. At the end of the day, I do think that GitHub Actions are mature enough to be used in a professional context and can reproduce most if not all the common features present in other workflow automation tools such as CircleCI.

During the process, I did discover a really promising feature which will dramatically help to increase the reusability of workflows: Composite Run Steps released in August 7, 2020. For the moment, this feature is still really embryonic since it only supports pure shell run commands (see syntax for more details). Nevertheless, there is an opened issue on the topic aiming at making composite run steps fully functional. I can’t wait for this feature to finally supports the uses syntax 😱 🎉 ⏳. Those will definitely give a lot more freedom on the division of workflows into multiple files, annihilate the need for uploading/downloading artifacts, and allow for simple design of private project specific actions. Furthermore, composite run steps actions will also open doors to a new way of creating GitHub Actions by combining different actions already available on the market to accomplish something even more complex at a certain cost of performance and flexibility undoubtedly.