The Best Customer CI Setup with Certyn

Most teams overcomplicate CI for customer environments. They either run too much in the deploy path and slow releases down, or they run too little and miss regressions after the deploy is already live.

With Certyn, the strongest setup is simpler:

1. Deploy the customer environment
2. Update the Certyn environment version
3. Run a blocking smoke suite
4. If this is the default environment, trigger App Explorer asynchronously
5. Run regression on a schedule, not in the deploy path

This gives you a fast gate, clear release traceability, and continuous learning after each real deployment.

Why This Model Works

Smoke and exploration solve different problems.

• Smoke suite answers: did the critical paths still work after this deploy?
• App Explorer answers: what changed, what looks suspicious, and what new coverage should we add?

That distinction matters. Smoke should block deploy promotion. Explorer should not.

If you make exploration part of the blocking path, your pipeline gets slower and less predictable. If you skip exploration entirely, your coverage stays static and you miss the opportunity to discover new regressions or outdated test assumptions after the product changes.

The right split is:

• blocking: smoke
• non-blocking: explorer
• scheduled: regression

Step 1: Deploy First

For customer environments, Certyn should validate the version that is actually running, not a guess about what is about to run.

That is why the best place for the main smoke gate is after deploy.

This is especially important when:

• infrastructure or config changes are part of the release
• the environment has customer-specific data or integrations
• different regions or tenants can behave differently

Running smoke after deploy gives you the real answer: did this environment version actually come up cleanly?

Step 2: Update the Environment Version

After deployment, write the immutable release identifier into the Certyn environment:

• git SHA
• image tag
• release number

Example:

certyn environments update \
  --project my-app \
  production \
  --version "$GITHUB_SHA" \
  --changelog "Release $GITHUB_SHA"

This does two jobs at once:

• it makes every Certyn run traceable to an exact release
• it enables version-change automation for processes that should react to a deploy

If you can, attach short changelog text too. That makes incident triage much easier later.

Step 3: Run Smoke as the Deploy Gate

Your smoke suite should be the hard gate after deploy:

- name: Run post-deploy smoke
  uses: certyn/action@v1
  with:
    api_key: ${{ secrets.CERTYN_API_KEY }}
    project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
    environment_key: production
    process_slug: smoke-suite
    timeout_seconds: 1800
    wait_for_completion: true

Keep this suite small and opinionated:

• login
• navigation
• checkout or primary conversion flow
• the most critical customer workflow

If smoke fails, the deployment is not healthy enough to trust.

Step 4: Trigger App Explorer Only for the Default Environment

This is the part many teams miss.

Not every environment needs exploration after every deploy. In practice, you usually want App Explorer on the canonical environment where change matters most for ongoing quality learning.

For many teams that is production. For others it is a stable staging environment that mirrors production closely enough to explore safely.

Trigger Explorer only there:

- name: Start App Explorer
  if: ${{ vars.CERTYN_ENVIRONMENT_KEY == 'production' }}
  uses: certyn/action@v1
  with:
    api_key: ${{ secrets.CERTYN_API_KEY }}
    project_slug: ${{ secrets.CERTYN_PROJECT_SLUG }}
    environment_key: ${{ vars.CERTYN_ENVIRONMENT_KEY }}
    process_slug: explore
    timeout_seconds: 900
    wait_for_completion: false

Replace production with your canonical environment key if your primary environment is named differently.

The important part is wait_for_completion: false.

Explorer should not hold up the pipeline. Its job is to:

• discover behavior changes
• record observations
• propose bugs
• create or update candidate test coverage

That is follow-up intelligence, not a release gate.

One Important Rule: Avoid Duplicate Smoke Triggers

Do not configure two separate smoke starts for the same deploy unless you want two distinct verification events.

Pick one primary deploy trigger for smoke:

• Option A: explicit CI post-deploy smoke step
• Option B: automatic trigger on environment version change

For most customer setups, Option A is better for smoke because the pipeline needs a blocking verdict.

Then use the environment version update as the trigger source for follow-up automation such as App Explorer.

If you choose environment-version automation for Explorer, scope that trigger to exactly one environment, usually production. Do not enable Explorer's version-change trigger on every environment unless you intentionally want exploratory runs everywhere.

API Key Scopes

If your pipeline both updates environments and runs gates, the API key needs more than basic CI scopes.

Recommended minimum:

• ci:trigger
• ci:status
• ci:cancel
• platform:manage

The CI scopes handle smoke and explorer runs. platform:manage is needed for environment updates.

Let Explorer Improve Coverage, Not Rewrite Your Gate Mid-Deploy

Explorer can create or update test cases, which is powerful. But there is an important operational boundary:

do not let a deploy mutate your blocking smoke gate in the same release path without review.

A safer model is:

1. Smoke validates the release
2. Explorer discovers changes and proposes updates
3. The team reviews those updates
4. Approved changes become part of future smoke or regression coverage

That keeps your gate stable while still letting Certyn learn from the live product.

Recommended Operating Model

Here is the full pattern:

Event	What runs	Blocking?
Pull request	smoke-suite	Yes
Customer deploy	update environment version + smoke-suite	Yes
Default environment deploy	app-explorer	No
Nightly	regression-suite	Usually yes, but outside deploy path

This gives you:

• fast deploy confidence
• exact release traceability
• continuous discovery on the environment that matters most
• a growing test suite without putting exploration inside the critical path

Final Recommendation

If you are designing the best Certyn setup for customer environments, this is the model to start with:

• run smoke after deploy
• always update the environment version
• run App Explorer only on the default environment
• keep Explorer asynchronous
• keep regression scheduled

That is the cleanest balance between speed, safety, and learning.