GitLab CI
What you'll learn
- How to run Cypress tests with GitLab as part of CI/CD pipeline
- How to parallelize Cypress test runs within GitLab CI/CD
With its hosted CI/CD Service, GitLab offers developers "a tool built into GitLab for software development through the continuous methodologies".
Detailed documentation is available in the GitLab CI/CD Documentation.
Basic Setup
The example below is basic CI setup and job using
GitLab CI/CD
to run Cypress tests within the Electron browser. This GitLab CI configuration
is placed within .gitlab-ci.yml
.
stages:
- test
test:
image: node:latest
stage: test
script:
# install dependencies
- npm ci
# start the server in the background
- npm run start:ci &
# run Cypress tests
- npm run e2e
Try it out
To try out the example above yourself, fork the
Cypress Kitchen Sink
example project and place the above GitHub Action configuration in
.gitlab-ci.yml
.
How this configuration works:
- On push to this repository, this job will provision and start GitLab-hosted
Linux instance for running the outlined
stages
declared inscript
with in thetest
job section of the configuration. - The code is checked out from our GitHub/GitLab repository.
- Finally, our scripts will:
- Install npm dependencies
- Start the project web server (
npm start
) - Run the Cypress tests within our GitHub repository within Electron.
Testing in Chrome and Firefox with Cypress Docker Images
The Cypress team maintains the official
Docker Images for running
Cypress tests locally and in CI, which are built with Google Chrome and Firefox.
For example, this allows us to run the tests in Firefox by passing the
--browser firefox
attribute to cypress run
.
stages:
- test
test:
image: cypress/browsers:node12.14.1-chrome85-ff81
stage: test
script:
# install dependencies
- npm ci
# start the server in the background
- npm run start:ci &
# run Cypress tests
- npx cypress run --browser firefox
Caching Dependencies and Build Artifacts
Caching of dependencies and build artifacts can be accomplished with the cache
configuration. The
caching documentation contains all
options for caching dependencies and build artifacts across many different
workflows. Artifacts from a job can be defined by providing paths and an
optional expiry time.
stages:
- test
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
- .npm/
test:
image: cypress/browsers:node12.14.1-chrome85-ff81
stage: test
script:
# install dependencies
- npm ci
# start the server in the background
- npm run start:ci &
# run Cypress tests
- npx cypress run --browser firefox
artifacts:
when: always
paths:
- cypress/videos/**/*.mp4
- cypress/screenshots/**/*.png
expire_in: 1 day
Parallelization
Cypress Cloud offers the ability to parallelize and group test runs along with additional insights and analytics for Cypress tests.
The addition of the
parallel
attribute to the
configuration of a job will allow us to run multiples instances of Cypress at
same time as we will see later in this section.
Before diving into an example of a parallelization setup, it is important to understand the two different types of jobs that we will declare:
- Install Job: A job that installs and caches dependencies that will be used by subsequent jobs later in the GitLab CI workflow.
- Worker Job: A job that handles execution of Cypress tests and depends on the install job.
Install Job
The separation of installation from test running is necessary when running parallel jobs. It allows for reuse of various build steps aided by caching.
First, we will define the build
stage along with cache
and variables related
to the cache.
Then we define the install
step that will be used by the worker jobs and
assign it to the build
stage.
stages:
- build
## Set environment variables for folders in "cache" job settings for npm modules and Cypress binary
variables:
npm_config_cache: '$CI_PROJECT_DIR/.npm'
CYPRESS_CACHE_FOLDER: '$CI_PROJECT_DIR/cache/Cypress'
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- cache/Cypress
- node_modules
- build
## Install NPM dependencies and Cypress
install:
image: cypress/browsers:node12.14.1-chrome85-ff81
stage: build
script:
- npm ci
Worker Jobs
Next, we add a test
stage and define the worker job named ui-chrome-tests
that will run Cypress tests with Chrome in parallel during the test
stage.
The addition of the
parallel
attribute to the
configuration of a job will allow us to run multiples instances of Cypress at
same time.
stages:
- build
- test
## Set environment variables for folders in "cache" job settings for npm modules and Cypress binary
variables:
npm_config_cache: '$CI_PROJECT_DIR/.npm'
CYPRESS_CACHE_FOLDER: '$CI_PROJECT_DIR/cache/Cypress'
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- .cache/*
- cache/Cypress
- node_modules
- build
## Install NPM dependencies and Cypress
install:
image: cypress/browsers:node12.14.1-chrome85-ff81
stage: build
script:
- npm ci
ui-chrome-tests:
image: cypress/browsers:node12.14.1-chrome85-ff81
stage: test
parallel: 5
script:
# install dependencies
- npm ci
# start the server in the background
- npm run start:ci &
# run Cypress tests in parallel
- npx cypress run --record --parallel --browser chrome --group "UI - Chrome"
The above configuration using the --parallel
and --record
flags to
cypress run requires setting up
recording test results to Cypress Cloud.
Using Cypress Cloud with GitLab CI/CD
In the GitLab CI configuration we have defined in the previous section, we are leveraging three useful features of Cypress Cloud:
-
Recording test results with the
--record
flag to Cypress Cloud:- In-depth and shareable test reports.
- Visibility into test failures via quick access to error messages, stack traces, screenshots, videos, and contextual details.
- Integrating testing with the merge-request process via commit status guards and convenient test report comments.
- Detecting flaky tests and surfacing them via Slack alerts or GitLab PR status checks.
-
Parallelizing test runs and optimizing their execution via intelligent load-balancing of test specs across CI machines with the
--parallel
flag. -
Organizing and consolidating multiple
cypress run
calls by labeled groups into a single report within Cypress Cloud. In the example above we use the--group "UI - Chrome"
flag to organize all UI tests for the Chrome browser into a group labeled "UI - Chrome" in the Cypress Cloud report.
Cypress Real World Example with GitLab CI/CD
A complete CI workflow against multiple browsers, viewports and operating systems is available in the Real World App (RWA).
Clone the Real World App (RWA) and refer to the .gitlab-ci.yml file.