Skip to content

feat: classify the status of an execution that did not pass - #8312

Merged
dejanzele merged 1 commit into
tkc-6813/termination-codesfrom
tkc-6813/classifier
Sep 21, 2026
Merged

dejanzele merged 1 commit into
tkc-6813/termination-codesfrom
tkc-6813/classifier

Conversation

@dejanzele

@dejanzele dejanzele commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Every execution that does not pass now carries the status details object when the runner or the standalone agent ends it. One function in the model package builds the object from the final result and from what the caller knows about the stop. It reads a cancel by a person first, then a cause that a step recorded, then the reason of the stop, then a step of the test that failed, then a job that a caller removed without naming itself. A cause that a step recorded wins over the reason of the stop, because the cause is what the user fixes and the stop is only the mechanism that ended the execution. The runner classifies at the end of a watch, after a lost watch, and after the check for stuck executions. The standalone agent classifies when it declines an execution, and it now stores the message and the code that it dropped before.

A program cannot act on a sentence. The layer and the code give the dashboard a label, the list a filter, and a webhook a value to route on, and they stay the same when the wording changes. A workflow reads the layer, the code, and the step of another execution through the execution() function, so a suite asserts the cause of a failure and not the words of a message. The message and the person who canceled stay out of that record, because a child workflow must not read them.

@dejanzele
dejanzele requested review from a team as code owners September 17, 2026 08:07
@dejanzele
dejanzele requested review from tkonieczny and vsukhin and removed request for a team September 17, 2026 08:07
@dejanzele
dejanzele added this pull request to stack #8311 September 17, 2026 08:07
@testkubebot

testkubebot Bot commented Sep 17, 2026

Copy link
Copy Markdown

✅ Testkube GitHub Integration

Review based on commit c4d303c.

All tests and quality gates passed.


Phase Status
Test Workflow Execution ✅ Passed
Quality Gate ✅ Passed

7 workflows executed

lint-go passed
in 4m49s (🚀 21. Sep. 2026 - 08:56:02 UTC / 🏁 21. Sep. 2026 - 09:00:52 UTC)

lint-proto passed
in 21s (🚀 21. Sep. 2026 - 08:56:03 UTC / 🏁 21. Sep. 2026 - 08:56:24 UTC)

integration-tests passed
in 7m42s (🚀 21. Sep. 2026 - 08:56:02 UTC / 🏁 21. Sep. 2026 - 09:03:45 UTC)

unit-tests passed
in 7m44s (🚀 21. Sep. 2026 - 08:56:03 UTC / 🏁 21. Sep. 2026 - 09:03:47 UTC)

verify-crds passed
in 2m42s (🚀 21. Sep. 2026 - 08:56:02 UTC / 🏁 21. Sep. 2026 - 08:58:45 UTC)

verify-protobuf passed
in 20s (🚀 21. Sep. 2026 - 08:56:03 UTC / 🏁 21. Sep. 2026 - 08:56:23 UTC)

lint-pr passed
in 15s (🚀 21. Sep. 2026 - 08:56:02 UTC / 🏁 21. Sep. 2026 - 08:56:18 UTC)


Manage this Integration

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The PR should not merge until negative-group failures retain their test-failure classification.

Findings

  1. P1 Negative-group failures become unknown

Summary

Adds shared classification of non-passing executions across runner finalization, recovery, and standalone declines, and exposes classification codes to workflow expressions.

  • Earlier functional findings are addressed, including child aliases, fatal recovery, actor attribution, and nested-stop details.
  • One new regression remains: failed negative groups are excluded from test-failure classification when their leaves pass.

Reviews (5) · Last reviewed commit: "feat: classify the status of an executio..."

Comment thread pkg/executiondata/types.go Outdated
Comment thread pkg/runner/recovery.go
Comment thread pkg/api/v1/testkube/status_classify.go Outdated
Comment thread pkg/api/v1/testkube/status_classify.go
@dejanzele

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread pkg/api/v1/testkube/status_classify.go Outdated
@dejanzele

Copy link
Copy Markdown
Contributor Author

@greptileai review

@dejanzele

Copy link
Copy Markdown
Contributor Author

@greptileai review

@dejanzele
dejanzele force-pushed the tkc-6813/classifier branch 3 times, most recently from 859f538 to 8466631 Compare September 17, 2026 15:53
@dejanzele

Copy link
Copy Markdown
Contributor Author

@greptileai review

Comment thread pkg/api/v1/testkube/status_classify.go Outdated
Comment on lines +161 to +164
// A group takes its status from its children, so only a leaf names a failure of the test.
if len(sig.Children) > 0 {
continue
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Negative-group failures become unknown

Critical (must fix before merge): A required negative: true group containing two successful shell steps fails because its result is !step1 || !step2, while both leaves pass. Skipping every group here discards the only failed step, so normal completion is classified as unknown with no step instead of step-failure / exit-code naming the group.

This also makes execution().statusType, statusReason, and statusStep lose the actual test-failure classification. Prefer failed leaves when available, but retain a failed required group as a fallback.

@dejanzele
dejanzele force-pushed the tkc-6813/classifier branch 2 times, most recently from 0533a92 to e54fb7c Compare September 19, 2026 13:53
@dejanzele
dejanzele merged commit 0488e8a into main Sep 21, 2026
10 checks passed
@dejanzele
dejanzele deleted the tkc-6813/classifier branch September 21, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants