Conducting trunk-based development and CI/CD practices using GitHub actions

Conducting trunk-based development and CI/CD practices using GitHub actions

Symphony Logo
Symphony
May 22nd, 2023

A relatively young approach to development ā€“ trunk-based development (TBD) ā€“ has come out as a result of seven years of research and data from over 32,000 professionals worldwide, conducted by DORA State of DevOps. It is the longest-running academically rigorous research investigation of its kind, providing an independent view into the practices and capabilities that help drive high-performance delivery and ultimately organizational outcomes.Ā  The research uses behavioral science to identify the most effective and efficient ways to develop and deliver software. It proved to be a great approach and engineers seem to love it. Many teams have adopted TBD so far. Just to name a well-known few, there is a GitHub team that works on github.com, Visual Studio Team Services, Google, and Netflix.

TBD: Overview

By definition, trunk-based development is a Git branching strategy that emphasizes the use of a single, shared code repository or "trunk" where all developers work on the same codebase. Rather than branching off separate copies of the code for each feature or bug fix, developers make changes directly to the main trunk branch, which is continuously integrated (CI) and continuously deployed (CD) to production.

646c7dfec286ed16b8164861

TBD vs Gitflow

Letā€™s make a brief comparison with the all-timer branching strategy ā€“ Gitflow. It uses at least two long-lived branches, main and develop. Feature branches are created off develop and merged back when completed, and they usually arenā€™t deleted after the merge.Ā 

When bugs or any kind of other issues inevitably show up, it can become increasingly difficult to figure out where those issues are exactly originating from by looking at the project history as developers can get lost in a sea of commits. All of this can slow down the development process and the release cycle. In that sense, Gitflow is not an efficient approach for teams wanting to implement CI/CD.Ā 

Unlike Gitflow, TBD uses one and one only long-lived branch, usually called main, of which short-lived feature branches are created. After the work is done on a feature branch, commits are squashed and merged as a single commit and the feature branch is deleted afterwards. This enforces linear commit history. Giflow provides more structure than TBD but can be complex to manage and sometimes can lead to merge hell.

TBD: Pros and Cons

TBD is designed to promote collaboration, reduce complexity, and accelerate development cycles by eliminating the overhead of managing multiple branches and merges. By working directly on the trunk, developers can quickly integrate and test changes in a shared environment, which can help catch and resolve issues early in the development process.

However, there are a few unwanted risks that come along with using the TBD. For instance, there is a risk of introducing breaking changes. Because all developers are working on the same branch, changes made to the trunk can potentially break existing code or cause conflicts with other changes. Nevertheless, this risk can be avoided by simply configuring an appropriate set of branch protection rules. The one thatā€™s especially useful in TBD is the one that prevents merging branches that are not up to date with the main branch.

Secondly ā€“ there is the need for strong communication. Technically this isnā€™t a risk, but rather a ā€œdownsideā€ if you look at communication as an obstacle. With everyone working on the same trunk, it's crucial that teams communicate well and work collaboratively to avoid conflicts and ensure that changes are integrated smoothly. This can be challenging, especially for larger or distributed teams.

Practices that define TBD

At first glance, TBD seems to be a straightforward process. Thatā€™s not far from the truth, but the perks of TBD are determined by a set of practices that have to be conducted properly in order to make the development process easier and conserve the beauty of straightforwardness.

  • Strict coding standards

  • Frequent code reviews

  • CI/CD with notifications

  • Automation

Depending on the project, the CD abbreviation either stands for continuous delivery or continuous deployment. The key difference is a well-known agile ceremony ā€“ The Release Day ā€“ where continuous deployment excludes the need for that step because all changes are automatically deployed to production. However, the latter requires feature flags to be set in place.

What Iā€™ve noticed during my career is that every team that uses TBD has a slightly different approach to CI/CD. And that is totally legit. After all, this is an organizational tool that should be adjusted to teamsā€™ needs.

For example, I have orchestrated a set of pipelines for one of our projects in a manner that excludes additional test runners in the CD part to save on GitHub minutes and also put a version tag before the build step. It may look a bit inconvenient, but in fact, this setting perfectly suits my teamā€™s needs.

646c7efdc286ed16b81649b0

These practices help to ensure that the code on the main branch is always in a stable, releasable state and that changes are thoroughly reviewed and tested before they are integrated. By following these practices, teams have a greater chance to produce high-quality code and ship changes more quickly to the test environment and their clients.

Building CI/CD pipelines using GitHub Actions

GitHub actions use workflows ā€“ Yaml files ā€“ located in the `.github/workflows/` root directory. There are lots of open-source GitHub actions that you can use in your workflow and drastically speed up the development of your pipeline. For example, widely used `actions/checkout@v3`, checks out your repository under `$GITHUB_WORKSPACE`, so your workflow can access it. And there are many more, some of which weā€™re going to use in our workflows.

Continuous Integration (CI)

In the CI part, we are going to make sure that no ā€œsmellyā€ or untested code can be merged. For that to work, we will define a pipeline that will test our code along the way and notify us if there is something wrong with it upon pull request (PR) creation.

646c7f75c286ed16b81649ee

Besides the four PR checks visible in the graph, the workflow includes a few more steps that round up the CI. I have put all steps in a single job named Quality Check, set `ubuntu` as my job runner, and configured a trigger to run this workflow upon PR creation.Ā 

First things first, check out your repo. Then set the environment variables in a separate step by running shell commands that extract engine versions from `package.json` ā€“ in this case node and pnpm ā€“ making them easily accessible within the job by simply calling `${{ env.NODE_VERSION }}`. That way, weā€™re using package.json as a single source of truth regarding the engine versions. This means we donā€™t need to update them manually through our workflows after updating them in the project.

Afterward, initiate node and pnpm with extracted versions, install dependencies and perform noted checks: lint code, Typescript check, unit tests are passing, line coverage is above 80%.

# pull-request.yml

name: PULL REQUEST on: Ā  pull_request: Ā  Ā  branches: Ā  Ā  Ā  - main jobs: Ā  quality: Ā  Ā  name: Quality Check Ā  Ā  runs-on: ubuntu-latest Ā  Ā  steps: Ā  Ā  Ā  Ā  uses: actions/checkout@v3 Ā  Ā  Ā  - name: Set ENV variables Ā  Ā  Ā  Ā  id: env_vars Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  echo "NODE_VERSION=$(jq -r .engines.node ./package.json)" >> $GITHUB_ENV Ā  Ā  Ā  Ā  Ā  echo "PNPM_VERSION=$(jq -r .engines.pnpm ./package.json)" >> $GITHUB_ENV Ā  Ā  Ā  - name: Use pnpm (${{ env.PNPM_VERSION }}) Ā  Ā  Ā  Ā  uses: pnpm/action-setup@v2 Ā  Ā  Ā  Ā  with: Ā  Ā  Ā  Ā  Ā  version: ${{ env.PNPM_VERSION }} Ā  Ā  Ā  Ā  Ā  run_install: false Ā  Ā  Ā  - name: Use Node.js (${{ env.NODE_VERSION }}) Ā  Ā  Ā  Ā  uses: actions/setup-node@v3 Ā  Ā  Ā  Ā  with: Ā  Ā  Ā  Ā  Ā  node-version: ${{ env.NODE_VERSION }} Ā  Ā  Ā  Ā  Ā  cache: pnpm Ā  Ā  Ā  - name: Install dependencies Ā  Ā  Ā  Ā  run: pnpm install Ā  Ā  Ā  - name: Lint Ā  Ā  Ā  Ā  id: lint Ā  Ā  Ā  Ā  run: pnpm run lint Ā  Ā  Ā  - name: TypeScript check Ā  Ā  Ā  Ā  id: tsc Ā  Ā  Ā  Ā  run: pnpm tsc Ā  Ā  Ā  - name: Unit tests Ā  Ā  Ā  Ā  id: test Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  pnpm run test:coverage > test_result.txt Ā  Ā  Ā  Ā  Ā  cat test_result.txt Ā  Ā  Ā  Ā  Ā  echo "coverage_value=$(cat test_result.txt | tail -2 | head -1 | sed 's/.*: \([0-9]*\)\..*/\1/')" >> $GITHUB_OUTPUT Ā  Ā  Ā  - name: Unit test coverage Ā  Ā  Ā  Ā  id: coverage Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  if [[ ${{ steps.test.outputs.coverage_value }} -le 79 ]]; then Ā  Ā  Ā  Ā  Ā  Ā  echo "Code coverage is less than 80%. Try to cover more lines with unit tests." Ā  Ā  Ā  Ā  Ā  Ā  exit 1 Ā  Ā  Ā  Ā  Ā  fi

In our project, we use Vitest for running unit tests and C8 for collecting coverage. The last step check is adjusted to these specific tools. So if you try to reuse this workflow, make sure that you collect coverage correctly as per your toolsā€™ docs.

Now, we should set up a system to notify the developers of the job outcome since without notifications all of this fuss somehow loses the point. And by notifying, I mean passing the message through GitHubā€™s already built-in emailing system and creating a code coverage badge that can be displayed in the projectā€™s readme file.

# pull-request.yml

# ...Ā 

Ā Ā Ā Ā Ā Ā - name: Create code coverage badge Ā  Ā  Ā  Ā  if: success() Ā  Ā  Ā  Ā  uses: schneegans/dynamic-badges-action@v1.6.0 Ā  Ā  Ā  Ā  with: Ā  Ā  Ā  Ā  Ā  auth: ${{ secrets.GIST_TOKEN }} Ā  Ā  Ā  Ā  Ā  gistID: <your_gist_id> Ā  Ā  Ā  Ā  Ā  filename: unit-test-coverage.json Ā  Ā  Ā  Ā  Ā  label: Coverage Ā  Ā  Ā  Ā  Ā  message: ${{ steps.test.outputs.coverage_value }}% Ā  Ā  Ā  Ā  Ā  valColorRange: ${{ steps.test.outputs.coverage_value }} Ā  Ā  Ā  Ā  Ā  maxColorRange: 90 Ā  Ā  Ā  Ā  Ā  minColorRange: 50 Ā  Ā  Ā  - name: Comment the Issue (email content) Ā  Ā  Ā  Ā  if: ${{ always() }} Ā  Ā  Ā  Ā  uses: actions/github-script@v6 Ā  Ā  Ā  Ā  env: Ā  Ā  Ā  Ā  Ā  BRANCH_NAME: ${{ github.event.pull_request.head.ref }} Ā  Ā  Ā  Ā  with: Ā  Ā  Ā  Ā  Ā  script: | Ā  Ā  Ā  Ā  Ā  Ā  github.rest.issues.createComment({ Ā  Ā  Ā  Ā  Ā  Ā  Ā  issue_number: context.issue.number, Ā  Ā  Ā  Ā  Ā  Ā  Ā  owner: context.repo.owner, Ā  Ā  Ā  Ā  Ā  Ā  Ā  repo: context.repo.repo, Ā  Ā  Ā  Ā  Ā  Ā  Ā  body: `Your HTML/Markdown content with job result.` Ā  Ā  Ā  Ā  Ā  Ā  })

And thatā€™s it! If the job outcome is successful, GitHub will let you merge this PR; if not, wellā€¦ youā€™re gonna need to fix that error and cover a few more lines with unit tests. Other than that ā€“ easy peasy.

Branch protection rules

Most importantly, the main trunk must be protected from pushing directly to the branch or merging outdated branches. In the GitHub repo settings, you can find a section dedicated to protecting branches. Pick the main branch and add the following rules:

  1. Require a pull request before merging

    • Dismiss stale pull request approvals when new commits are pushed

  2. Require status checks to pass before merging

    • Require branches to be up to date before merging

    • Quality Check (GitHub Actions)

  3. Require linear history

  4. Restrict who can push to matching branches

    • Restrict pushes that create matching branches

    • Admin account (GitHub Actions)

  5. Allow deletionsĀ 

Rules I would especially like to discuss are the 2nd and the 3rd rule. The second rule prevents merging changes if the quality check has failed and the third one ensures that new commits are either squashed before merging or rebased onto the protected (main) branch. These two keep the Git history clean.

646ca384c286ed16b81667f6

Continuous Delivery (CD)

Following the same logic, with a slight advancement regarding workflow reusability, continuous delivery can be accomplished as well. Unlike the CI workflow, which handles everything in a single job, CD will have separate jobs that will depend on each otherā€™s outcome.

646c80b1c286ed16b8164aa6

Starting with bumping the version in `package.json` and creating a git tag for that version, then building and deploying the tagged code to development and production environments. Depending on the project, the build can be accomplished in a single step for all environments or it can be built separately for each. In our case, each environment uses different configs, hence separate builds are required. But first, we must get the engine version numbers in the job named setup. Set a trigger to run the workflow on push to the main branch and ignore tags ā€“ because we will use an actions-bot to push tags directly onto the main.

# ci-cd.yml name: CI/*CD* on: Ā  push: Ā  Ā  branches: Ā  Ā  Ā  - main Ā  Ā  tags-ignore: Ā  Ā  Ā  - '*' Ā  Ā  paths-ignore: Ā  Ā  Ā  - 'package.json' Ā  Ā  Ā  - 'pnpm-lock.yaml' Ā  Ā  Ā  - 'CHANGELOG.md' jobs: Ā  setup: Ā  Ā  if: | Ā  Ā  echo "Your condition here" Ā  Ā  name: Get engines Ā  Ā  outputs: Ā  Ā  Ā  NODE_VERSION: ${{ steps.env_vars.outputs.NODE_VERSION }} Ā  Ā  Ā  PNPM_VERSION: ${{ steps.env_vars.outputs.PNPM_VERSION }} Ā  Ā  Ā  RELEASABLE: ${{ your condition here }} Ā  Ā  runs-on: ubuntu-latest Ā  Ā  steps: Ā  Ā  Ā  Ā  uses: actions/checkout@v3 Ā  Ā  Ā  - name: Set ENV variables Ā  Ā  Ā  Ā  id: env_vars Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  echo "NODE_VERSION=$(jq -r .engines.node ./package.json)" >> $GITHUB_OUTPUT Ā  Ā  Ā  Ā  Ā  echo "PNPM_VERSION=$(jq -r .engines.pnpm ./package.json)" >> $GITHUB_OUTPUT

# ...

Versioning is accomplished using the npm package `standard-version` that follows the conventional commits specification and bumps version based on the commit message. Because I had it installed in the project, I chose not to use their open-source action, but the package itself by running a set of commands. Hereā€™s how I made it work. # ci-cd.yml # ... jobs: Ā  setup:

Ā Ā Ā Ā outputs:

Ā Ā Ā Ā Ā Ā NODE_VERSION: 18.12.1 Ā  Ā  Ā  PNPM_VERSION: 7.25.0 Ā  Ā  Ā  RELEASABLE: true # ... Ā  version: Ā  Ā  name: Bump Version and Git Tag Ā  Ā  env: Ā  Ā  Ā  NODE_VERSION: ${{ needs.setup.outputs.NODE_VERSION }} Ā  Ā  outputs: Ā  Ā  Ā  APP_VERSION: ${{ steps.app_version.outputs.APP_VERSION }} Ā  Ā  needs: setup Ā  Ā  if: | Ā  Ā  Ā  needs.setup.result == 'success' && Ā  Ā  Ā  needs.setup.outputs.RELEASABLE == 'true' Ā  Ā  runs-on: ubuntu-latest Ā  Ā  steps: Ā  Ā  Ā  Ā  uses: actions/checkout@v3 Ā  Ā  Ā  - name: Use Node.js (${{ env.NODE_VERSION }}) Ā  Ā  Ā  Ā  uses: actions/setup-node@v3 Ā  Ā  Ā  Ā  with: Ā  Ā  Ā  Ā  Ā  node-version: ${{ env.NODE_VERSION }} Ā  Ā  Ā  - name: Checkout main Ā  Ā  Ā  Ā  uses: actions/checkout@main Ā  Ā  Ā  Ā  with: Ā  Ā  Ā  Ā  Ā  fetch-depth: 0 Ā  Ā  Ā  Ā  Ā  token: ${{ secrets.ACTIONS_BOT_ACCESS_TOKEN }} Ā  Ā  Ā  - name: Configure committer Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  git config user.name "actions-bot" Ā  Ā  Ā  Ā  Ā  git config user.email "project.account@symphony.is" Ā  Ā  Ā  - name: Run standard-version Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  npx standard-version Ā  Ā  Ā  Ā  Ā  git push --follow-tags origin main Ā  Ā  Ā  - name: Output App Version Ā  Ā  Ā  Ā  id: app_version Ā  Ā  Ā  Ā  run: | Ā  Ā  Ā  Ā  Ā  echo "APP_VERSION=$(jq -r .version ./package.json)" >> $GITHUB_OUTPUT

# ...

Weā€™re outputting the app version in the last step of the version job and that output will be passed to the next job build-and-deploy-dev that uses the reusable workflow `build-and-deploy.yml`.Ā 

In this case, Firebase is being used as a hosting platform, thus the service account secret is saved in the GitHub actions repository secrets. Also, this workflow can be triggered manually on `workflow_dispatch` or in this case on `workflow_call` as a part of the pipeline.

# ci-cd.yml # ... jobs: Ā  setup:

Ā Ā Ā Ā outputs: Ā  Ā  Ā  NODE_VERSION: 18.12.1 Ā  Ā  Ā  PNPM_VERSION: 7.25.0 Ā  Ā  Ā  RELEASABLE: true # ... Ā  version:

Ā Ā Ā Ā outputs: Ā  Ā  Ā  APP_VERSION: 1.2.0 # ... Ā  build-and-deploy-dev: Ā  Ā  name: DEVELOPMENT Ā  Ā  uses: ./.github/workflows/build-and-deploy.yml Ā  Ā  needs: Ā  Ā  Ā  - setup Ā  Ā  Ā  - version Ā  Ā  with: Ā  Ā  Ā  version: ${{ needs.version.outputs.APP_VERSION }} Ā  Ā  Ā  environment: dev Ā  Ā  Ā  nodeVersion: ${{ needs.setup.outputs.NODE_VERSION }} Ā  Ā  Ā  pnpmVersion: ${{ needs.setup.outputs.PNPM_VERSION }} Ā  Ā  secrets: Ā  Ā  Ā  FIREBASE_SERVICE_ACCOUNT_DEV: '${{ secrets.FIREBASE_SERVICE_ACCOUNT_DEV }}' Deploy-to-production job looks exactly the same except `environment` (prod) and service account secrets differ from the original. That is, in case you decide to conduct continuous deployment. However, donā€™t forget to feature-flag new changes because you donā€™t want end users to do quality assurance before you.

My point of view

If it fits in one blog post, it probably isnā€™t that hard to implement tech-wise. Other than that, proper conduction of TBD on a project requires a well-formed agile team. In other words, your team must have strong communication and a high level of transparency in order for this to work.

My experience tells me that one brings out the other. Once the team gets used to TBD, it comes naturally for people to communicate better and more transparently in agile meetings.

About the author

Lazar Kulasevic is a Software Engineer working in the engineering hub in NiÅ”. Lazar specializes in Frontend development and possesses extensive knowledge of the versatile ecosystem surrounding JavaScript. Throughout his career, he has actively contributed to various agile teams, adeptly employing methodologies such as Scrum, Kanban, and Scrumban, while adhering to the principles of Continuous Integration and Continuous Deployment (CI/CD).

Contact us if you have any questions about our company or products.

We will try to provide an answer within a few days.