CI/CD for QA: GitHub Actions, Jenkins, and GitLab Compared

Tests that only run on your laptop don't block anything. A developer can merge code that breaks your test suite, and you won't know until you run tests manually. This article covers how to connect Playwright tests to GitHub Actions, GitLab CI, and Jenkins so they run on every pull request automatically, with a complete copy-paste GitHub Actions workflow and the configuration differences between all three platforms.

What CI/CD actually means for QA

CI stands for Continuous Integration. The idea: every time a developer pushes code, an automated process runs, compiles the code, runs tests, reports results. Problems get caught within minutes of being introduced, not days later during manual QA.

CD stands for Continuous Delivery (or Deployment). After CI passes, the code gets automatically deployed to a staging environment, or even production.

For a QA engineer, the relevant part is CI. Your tests run in the pipeline. Every pull request triggers them. If tests fail, the PR can't merge. That's the contract.

Tests that only run on your laptop don't enforce anything. Tests in CI enforce the quality bar on every change, automatically.

GitHub Actions: start here

GitHub Actions is the easiest CI tool to learn and the most widely used for new projects. It's free for public repositories and has a generous free tier for private ones.

The concept: you define a workflow in a YAML file inside your repository. GitHub runs that workflow whenever a trigger event happens: a push, a pull request, a schedule.

Create .github/workflows/playwright.yml in your project root:

mkdir -p .github/workflows

# .github/workflows/playwright.yml
name: Playwright Tests

on:
  push:
    branches: [main, master]
  pull_request:
    branches: [main, master]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright browsers
        run: npx playwright install --with-deps

      - name: Run Playwright tests
        run: npx playwright test

      - name: Upload test report
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 14

Commit this file, push to GitHub, and open the Actions tab. You'll see the workflow running. It installs Node, installs Playwright browsers, runs your tests, and uploads the HTML report as a downloadable artifact.

The if: always() on the upload step means the report gets uploaded even when tests fail, which is exactly when you need it most.

Understanding the workflow file

Environment variables in GitHub Actions

Never hardcode credentials in the workflow file. Store them as GitHub Secrets and reference them as environment variables.

Go to your repository → Settings → Secrets and variables → Actions → New repository secret. Add TEST_USER and TEST_PASS.

Reference them in the workflow:

      - name: Run Playwright tests
        run: npx playwright test
        env:
          TEST_USER: ${{ secrets.TEST_USER }}
          TEST_PASS: ${{ secrets.TEST_PASS }}
          BASE_URL: https://lab.becomeqa.com

In your tests, access them as process.env.TEST_USER. Secrets are masked in logs. GitHub replaces their values with * if they appear in output.

Run tests faster with sharding

By default Playwright uses multiple workers on a single machine. For large suites, you can split across multiple machines using sharding.

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'npm'
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --shard=${{ matrix.shard }}/4

This runs 4 jobs in parallel, each handling 25% of the tests. A suite that takes 20 minutes on one machine takes 5 minutes split across four.

GitLab CI

GitLab CI uses the same concepts as GitHub Actions but different syntax. The config file is .gitlab-ci.yml in the project root.

# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.44.0-jammy

stages:
  - test

playwright:
  stage: test
  script:
    - npm ci
    - npx playwright test
  artifacts:
    when: always
    paths:
      - playwright-report/
    expire_in: 1 week
  variables:
    TEST_USER: $TEST_USER
    TEST_PASS: $TEST_PASS

Key differences from GitHub Actions:

The image: field specifies a Docker image for the job. Using the official Playwright image (mcr.microsoft.com/playwright) means browsers are already installed, so no npx playwright install is needed.

Variables are set in GitLab → Project → Settings → CI/CD → Variables. Reference them with $VARIABLE_NAME in the config.

If you know GitHub Actions, you can pick up GitLab CI in a day. The mental model is identical: triggers, jobs, steps, artifacts. Just different YAML.

Jenkins

Jenkins is older, more complex, and still dominant in large enterprise environments. You'll encounter it at companies with legacy infrastructure.

Jenkins uses a Jenkinsfile in the project root:

pipeline {
  agent {
    docker {
      image 'mcr.microsoft.com/playwright:v1.44.0-jammy'
    }
  }

  environment {
    TEST_USER = credentials('test-user')
    TEST_PASS = credentials('test-pass')
  }

  stages {
    stage('Install') {
      steps {
        sh 'npm ci'
      }
    }

    stage('Test') {
      steps {
        sh 'npx playwright test'
      }
      post {
        always {
          publishHTML(target: [
            reportDir: 'playwright-report',
            reportFiles: 'index.html',
            reportName: 'Playwright Report'
          ])
        }
      }
    }
  }
}

Jenkins requires a running Jenkins server. It's not a cloud service. Someone needs to install, configure, and maintain it. That's why GitHub Actions has largely replaced it for new projects.

For QA specifically: if you join a company using Jenkins, you don't need to understand Jenkins administration. You need to understand the Jenkinsfile syntax well enough to add or modify test stages. The pipeline DSL (the syntax above) is what QA engineers work with day-to-day.

Which tool to learn first

The decision is straightforward:

What a real CI pipeline looks like

A mature Playwright pipeline in a real project typically has more than just "run tests":

# Realistic production pipeline
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npx tsc --noEmit        # TypeScript type check
      - run: npx eslint tests/       # Lint test files

  test:
    needs: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: 'npm' }
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test --project=chromium  # fast feedback on one browser
        env:
          BASE_URL: ${{ vars.BASE_URL }}
          TEST_USER: ${{ secrets.TEST_USER }}
          TEST_PASS: ${{ secrets.TEST_PASS }}
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: playwright-report
          path: playwright-report/

  test-cross-browser:
    needs: test
    if: github.ref == 'refs/heads/main'  # only on main branch
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: 'npm' }
      - run: npm ci
      - run: npx playwright install --with-deps
      - run: npx playwright test  # all browsers

The pattern: fast feedback first (lint + Chromium), then full cross-browser on the main branch. Pull requests get quick results; merges get thorough coverage.

FAQ

Check three things: environment variables (secrets not set in Actions), base URL (hardcoded localhost that doesn't exist in CI), and timing (CI runners are slower, so add retries: 1 in playwright.config.ts for CI).

In GitHub Actions: go to the failed run → Summary → Artifacts → download playwright-report. Extract the zip and open index.html in a browser.

Not natively in Playwright. Some teams use path filtering: only run tests in tests/auth/ when files in src/auth/ changed. It's complex to set up reliably. Start by running the full suite and optimize later if it gets too slow.

Add a schedule trigger:

on:
  schedule:
    - cron: '0 2 * * *'   # 2am UTC every night
  push:
    branches: [main]

Nightly runs are useful for catching intermittent failures that don't show up on every PR run.