GitHub Actions

What you'll learn

How to run Cypress tests with GitHub Actions as part of CI/CD pipeline
How to parallelize Cypress test runs within GitHub Actions
How to cache build artifacts between installation jobs and worker jobs

GitHub Actions + Cypress Screencasts

GitHub offers developers Actions that provide a way to automate, customize, and execute your software development workflows within your GitHub repository. Detailed documentation is available in the GitHub Action Documentation.

Cypress GitHub Action

Workflows can be packaged and shared as GitHub Actions. GitHub maintains many, such as the checkout and Upload/Download Artifact Actions actions used below.

The Cypress team maintains the official Cypress GitHub Action for running Cypress tests. This action provides npm installation, custom caching, additional configuration options and simplifies setup of advanced workflows with Cypress in the GitHub Actions platform.

Explicit Version Number

GitHub Action Version Number

We recommend using the explicit version number of the Cypress GitHub Action to prevent accidentally running tests with a new version of the action that may have breaking changes.

Read the GitHub Action documentation for more information

For Example:

jobs:
  cypress-run:
    steps:
      uses: cypress-io/github-action@v4.2.0

Basic Setup

The example below is basic CI setup and job using the Cypress GitHub Action to run Cypress tests within the Electron browser. This GitHub Action configuration is placed within .github/workflows/main.yml.

name: Cypress Tests

on: [push]

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      # Install NPM dependencies, cache them correctly
      # and run all Cypress tests
      - name: Cypress run
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          build: npm run build
          start: npm start

Try it out

To try out the example above yourself, fork the Cypress Kitchen Sink example project and place the above GitHub Action configuration in .github/workflows/main.yml.

How this action works:

On push to this repository, this job will provision and start GitHub-hosted Ubuntu Linux instance for running the outlined steps for the declared cypress-run job within the jobs section of the configuration.
The GitHub checkout Action is used to checkout our code from our GitHub repository.
Finally, our Cypress GitHub Action will:
- Install npm dependencies
- Build the project (npm run build)
- Start the project web server (npm start)
- Run the Cypress tests within our GitHub repository within Electron.

Testing in Chrome and Firefox with Cypress Docker Images

GitHub Actions provides the option to specify a container image for the job. Cypress offers various Docker Images for running Cypress locally and in CI.

Below we add the container attribute using a Cypress Docker Image built with Google Chrome and Firefox. For example, this allows us to run the tests in Firefox by passing the browser: firefox attribute to the Cypress GitHub Action.

name: Cypress Tests using Cypress Docker Image

on: [push]

jobs:
  cypress-run:
    runs-on: ubuntu-latest
    container: cypress/browsers:node12.18.3-chrome87-ff82
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      # Install NPM dependencies, cache them correctly
      # and run all Cypress tests
      - name: Cypress run
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          # Specify Browser since container image is compile with Firefox
          browser: firefox

Caching Dependencies and Build Artifacts

The Cypress team maintains the official Cypress GitHub Action for running Cypress tests. This action provides npm installation, custom caching, additional configuration options and simplifies setting up advanced workflows with Cypress in the GitHub Actions platform.

Caching of dependencies and build artifacts between installation and worker jobs can be accomplished with the Upload/Download Artifact Actions.

The install job below uses the upload-artifact action and will save the state of the build directory for the worker jobs.

name: Cypress Tests with installation job

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    container: cypress/browsers:node12.18.3-chrome87-ff82
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Save build folder
        uses: actions/upload-artifact@v2
        with:
          name: build
          if-no-files-found: error
          path: build

      - name: Cypress install
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          # Disable running of tests within install job
          runTests: false
          build: yarn build

The download-artifact action will retrieve the build directory saved in the install job, as seen below in a worker job.

name: Cypress Tests with Dependency and Artifact Caching

on: [push]

jobs:
  # install:
  # ....
  cypress-run:
    runs-on: ubuntu-latest
    container: cypress/browsers:node12.18.3-chrome87-ff82
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Download the build folders
        uses: actions/download-artifact@v2
        with:
          name: build
          path: build

      # Install NPM dependencies, cache them correctly
      # and run all Cypress tests
      - name: Cypress run
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          # Specify Browser since container image is compile with Firefox
          browser: firefox
          build: yarn build

Parallelization

Cypress Cloud offers the ability to parallelize and group test runs along with additional insights and analytics for Cypress tests.

GitHub Actions offers a matrix strategy for declaring different job configurations for a single job definition. Jobs declared within a matrix strategy can run in parallel which enables us run multiples instances of Cypress at same time as we will see later in this section.

Before diving into an example of a parallelization setup, it is important to understand the two different types of GitHub Action jobs that we will declare:

Install Job: A job that installs and caches dependencies that will be used by subsequent jobs later in the GitHub Action workflow.
Worker Job: A job that handles execution of Cypress tests and depends on the install job.

Install Job

The separation of installation from test running is necessary when running parallel jobs. It allows for reuse of various build steps aided by caching.

First, we'll define the install step that will be used by the worker jobs defined in the matrix strategy.

For the steps, notice that we pass runTests: false to the Cypress GitHub Action to instruct it to only install and cache Cypress and npm dependencies without running the tests.

The upload-artifact action will save the state of the build directory for the worker jobs.

name: Cypress Tests with installation job

on: [push]

jobs:
  install:
    runs-on: ubuntu-latest
    container: cypress/browsers:node12.18.3-chrome87-ff82
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress install
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          # Disable running of tests within install job
          runTests: false
          build: yarn build

      - name: Save build folder
        uses: actions/upload-artifact@v2
        with:
          name: build
          if-no-files-found: error
          path: build

Worker Jobs

Next, we define the worker job named ui-chrome-tests that will run Cypress tests with Chrome as part of a parallelized matrix strategy.

Note

Using our Cypress GitHub Action we specify install: false since our dependencies and build were cached in our install job.

The download-artifact action will retrieve the build directory saved in the install job.

name: Cypress Tests with Install Job and UI Chrome Job x 5

on: [push]

jobs:
  install:
  # ...

  ui-chrome-tests:
    runs-on: ubuntu-latest
    container: cypress/browsers:node12.18.3-chrome87-ff82
    needs: install
    strategy:
      fail-fast: false
      matrix:
        # run copies of the current job in parallel
        containers: [1, 2, 3, 4, 5]
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Download the build folders
        uses: actions/download-artifact@v2
        with:
          name: build
          path: build

      - name: 'UI Tests - Chrome'
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          # we have already installed all dependencies above
          install: false
          start: yarn start:ci
          wait-on: 'http://localhost:3000'
          wait-on-timeout: 120
          browser: chrome
          record: true
          parallel: true
          group: 'UI - Chrome'
          spec: cypress/tests/ui/*
        env:
          CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
          # Recommended: pass the GitHub token lets this action correctly
          # determine the unique run id necessary to re-run the checks
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The above configuration using the parallel and record options of the Cypress GitHub Action requires setting up recording to Cypress Cloud.

Setting up Parallelization

To setup multiple containers to run in parallel, the matrix option of the strategy configuration can be set to containers: [1, 2, 3, 4, 5], which will start 5 instances of the defined container image.

Note

The containers array is filled with filler (or dummy) items to provision the desired number of CI machine instances within GitHub Actions.

Ensure Correct Container

The container attribute must be specified using the Cypress Docker Image in the configuration that was used in the install job.

#...
ui-chrome-tests:
  runs-on: ubuntu-latest
  container: cypress/browsers:node12.18.3-chrome87-ff82
  needs: install
  strategy:
    fail-fast: false
    matrix:
      # run copies of the current job in parallel
      containers: [1, 2, 3, 4, 5]

Using Cypress Cloud with GitHub Actions

In the GitHub Actions configuration we have defined in the previous section, we are leveraging three useful features of Cypress Cloud:

Recording test results with the record: true option to Cypress Cloud:
- In-depth and shareable test reports.
- Visibility into test failures via quick access to error messages, stack traces, screenshots, videos, and contextual details.
- Integrating testing with the pull-request (PR) process via commit status check guards and convenient test report comments.
- Detecting flaky tests and surfacing them via Slack alerts or GitHub PR status checks.
Parallelizing test runs and optimizing their execution via intelligent load-balancing of test specs across CI machines with the parallel: true option.
Organizing and consolidating multiple cypress run calls by labeled groups into a single report within Cypress Cloud. In the example above we use the group: "UI - Chrome" option to organize all UI tests for the Chrome browser into a group labeled "UI - Chrome" in the Cypress Cloud report.

Cypress Real World Example with GitHub Actions

A complete CI workflow against multiple browsers, viewports and operating systems is available in the Real World App (RWA).

Clone the Real World App (RWA) and refer to the .github/workflows/main.yml file.

Cypress Real World App GitHub Actions Matrix

Common Problems and Solutions

Re-run jobs passing with empty tests

We recommend passing the GITHUB_TOKEN secret (created by the GH Action automatically) as an environment variable. This will allow the accurate identification of each build to avoid confusion when re-running a build.

name: Cypress tests
on: [push]
jobs:
  cypress-run:
    name: Cypress run
    runs-on: ubuntu-20.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress run
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          record: true
        env:
          # pass GitHub token to detect new build vs re-run build
          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}

Pull requests commit message is `merge SHA into SHA`

You can overwrite the commit message sent to Cypress Cloud by setting an environment variable. See Issue #124 for more details.

name: Cypress tests
on: [push]
jobs:
  cypress-run:
    name: Cypress run
    runs-on: ubuntu-20.04
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Cypress run
        uses: cypress-io/github-action@v4.x.x # use the explicit version number
        with:
          record: true
        env:
          # overwrite commit message sent to Cypress Cloud
          COMMIT_INFO_MESSAGE: ${{github.event.pull_request.title}}
          # re-enable PR comment bot
          COMMIT_INFO_SHA: ${{github.event.pull_request.head.sha}}

We also recommend adding COMMIT_INFO_SHA to re-enable Cypress bot PR comments. See this comment for more details.