Bug Detection & Deduplication

When an agent encounters a failure during testing, Certyn can automatically create an issue with evidence. The system includes safeguards to prevent duplicate bugs and to link related failures together.

How bugs are created

1. Agent detects a failure during a test run
2. System searches for similar existing issues (80% text similarity threshold)
3. If a matching issue exists, the execution is linked to it instead of creating a duplicate
4. If no match, a new issue is created with Needs Review flagged

Every auto-created issue includes:

• screenshots and step-by-step logs from the session
• the environment and version where it was found
• a link to the execution and test case that triggered it

Duplicate detection

Before creating a new issue, the system checks for duplicates using three strategies:

Strategy	How it works
Text similarity	Compares title and description against open issues (80% threshold)
Same test case	Checks if an open bug already exists for the same test case
Same execution	Checks if a bug was already created during the current execution

If a match is found, the system links the new failure to the existing issue instead of creating a duplicate. This keeps your issue backlog clean and shows how frequently a bug reproduces.

One canonical bug per test case

When a test fails, the system ensures there is exactly one open bug issue for that test case. Subsequent failures of the same test link to the existing bug rather than creating new ones. This prevents noise in the backlog while still tracking every occurrence.

Why bugs are not auto-closed

When a test that previously failed starts passing again, the system flags the issue as potentially resolved and sets Needs Review. It does not automatically close the issue because:

• the pass could be a false positive (flaky test or timing)
• the bug might exist in one environment but not another
• a specific scenario passed while the underlying issue remains
• auto-closing would hide the discovery history

Retest flow

To verify a fix:

1. Ensure the issue has a linked test case (required for retest)
2. Mark the issue as Ready to Test or trigger a retest
3. A new execution runs the linked test case
4. The result updates the issue status:
   • test passes → Verified Passed
   • test fails → Verified Failed
   • test blocked → Blocked