CI/CD Quality Gates with Certyn

A quality gate is simple: run tests before deploying. If they fail, stop the deploy. Certyn makes this work with AI-powered testing — no test scripts to maintain, no flaky assertions to debug.

GitHub Actions (Recommended)

The fastest path is the official GitHub Action:

name: Quality Gate

on:
  pull_request:

jobs:
  certyn:
    runs-on: ubuntu-latest
    steps:
      - uses: certyn/action@v1
        with:
          api_key: ${{ secrets.CERTYN_API_KEY }}
          project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
          environment_key: staging
          process_slug: smoke-suite

That's it. The action triggers a smoke test run, polls for completion, and fails the workflow if any test fails.

What Happens During a Gate

1. Trigger — The action sends a request to start a process run
2. Execution — Certyn spins up agent containers that test your app in real browsers
3. Polling — The action checks progress every 10-30 seconds
4. Verdict — When all tests complete, the action passes or fails based on results

The action automatically includes GitHub metadata (commit SHA, branch, PR URL) so you can trace any test result back to the exact code change.

Configuration Options

Timeouts

Default is 30 minutes. Adjust for your suite:

- uses: certyn/action@v1
  with:
    api_key: ${{ secrets.CERTYN_API_KEY }}
    project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
    process_slug: regression-suite
    timeout_seconds: 3600  # 1 hour for regression

Gate rules

Control what counts as a failure:

- uses: certyn/action@v1
  with:
    api_key: ${{ secrets.CERTYN_API_KEY }}
    project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
    process_slug: smoke-suite
    fail_on_failed: true    # Fail if any test fails
    fail_on_blocked: true   # Fail if any test is blocked
    fail_on_cancelled: true # Fail if run is cancelled

Fire-and-forget

Trigger a run without waiting:

- uses: certyn/action@v1
  id: certyn
  with:
    api_key: ${{ secrets.CERTYN_API_KEY }}
    project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
    process_slug: regression-suite
    wait_for_completion: false

- run: echo "Check results at ${{ steps.certyn.outputs.status_url }}"

Tag-based runs

Run tests by tag instead of a named process:

- uses: certyn/action@v1
  with:
    api_key: ${{ secrets.CERTYN_API_KEY }}
    project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
    tags: smoke,critical

Using the CLI Directly

For GitLab CI, Jenkins, CircleCI, or any other CI system, use the CLI:

# GitLab CI example
quality_gate:
  script:
    - curl -fsSL https://certyn.io/install | bash
    - export PATH="$HOME/.local/bin:$PATH"
    - certyn run smoke --project my-app --environment staging --timeout 30m

The CLI returns exit code 1 when the gate fails, which stops the pipeline.

Recommended Strategy

Two suites, two triggers

Trigger	Suite	Purpose
Every PR	Smoke	Fast critical-path check (2-5 min)
Nightly	Regression	Full coverage sweep (15-30 min)

The smoke suite covers login, core workflows, and critical features. It runs on every PR to catch regressions before they merge.

The regression suite runs everything — edge cases, secondary flows, integrations. Run it nightly to catch what smoke missed.

Deployment gates

Add a gate before production deploys:

deploy:
  needs: [build]
  steps:
    - uses: certyn/action@v1
      with:
        api_key: ${{ secrets.CERTYN_API_KEY }}
        project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
        environment_key: staging
        process_slug: smoke-suite

    - name: Deploy to production
      if: success()
      run: ./deploy.sh production

Outputs

The action provides these outputs for downstream steps:

Output	Description
run_id	Certyn run identifier
status_url	URL to view results
state	Final state (completed, cancelled)
conclusion	pass or fail
total	Total test count
passed	Passed count
failed	Failed count

API Key Scopes

Create a CI-specific API key with minimal scopes:

Scope	Purpose
ci:trigger	Start runs
ci:status	Check run status
ci:cancel	Cancel runs on timeout

Tips

• The action uses idempotency keys — retrying a workflow won't create duplicate runs
• Set cancel_on_timeout: true (default) to clean up stuck runs
• Use --json with the CLI for machine-readable output in custom CI scripts
• Start with smoke gates on PRs, then expand to regression gates nightly