Add self-report metrics step to CI workflow - #3199
Conversation
|
Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset. In case there are security findings, they will be communicated to you as a comment inside the PR. Hope you’ll enjoy using Jit. Questions? Comments? Want to learn more? Get in touch with us. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| junit-xml-folder: 'junit-results' | ||
| service-name: 'node-redis' | ||
| service-namespace: 'redis' | ||
| service-version: 'master' |
There was a problem hiding this comment.
Hardcoded service-version ignores non-master branches
Medium Severity
service-version is hardcoded to 'master', but this workflow triggers on pushes and PRs to master, v4.0, and v5 branches. Metrics reported from v4.0 or v5 branches will be incorrectly attributed to master, making it impossible to distinguish metrics by branch. This likely needs a dynamic value like ${{ github.ref_name }}.
| '{"Authorization": "Basic ${{ | ||
| secrets.SELF_CHECK_OTEL_AUTHORIZATION_TOKEN}}"}' | ||
| env: | ||
| OTEL_EXPORTER_OTLP_PROTOCOL: 'http/protobuf' No newline at end of file |
There was a problem hiding this comment.
Metrics step skipped on test failures
Medium Severity
The "Self Report Metrics" step lacks an if: always() condition, so it only runs when all previous steps (including "Run Tests") succeed. Since the purpose of CI visibility is to track test outcomes — especially failures — this step will silently be skipped in the most valuable reporting scenario.
| otlp-headers: "Authorization=Basic ${{secrets.SELF_CHECK_OTEL_AUTHORIZATION_TOKEN}}" | ||
| env: | ||
| OTEL_EXPORTER_OTLP_PROTOCOL: "http/protobuf" | ||
| ACTIONS_STEP_DEBUG: "true" |
There was a problem hiding this comment.
Missing continue-on-error on non-critical metrics step
Medium Severity
The new Self Report Metrics step lacks continue-on-error: true. Since this is a non-critical observability step, any failure (OTLP endpoint outage, expired secret, network issue) will cause the entire CI job to be marked as failed — even when all tests actually passed. This could block PRs and pushes across all 12 matrix combinations whenever the telemetry backend is unavailable.
nkaradzhov
left a comment
nkaradzhov
left a comment
There was a problem hiding this comment.
Cursor's comment seems legit, otherwise, looks good
|
@nkaradzhov The comment is legit, but if we do that we will never know that the step is failing. I kinda prefer it to err so we fix it on time. |
2 participants
Note
Medium Risk
Adds a new CI step that exports test metrics to an external OTLP endpoint using a repository secret; main risk is misconfigured secrets/egress or unintended data exposure from CI.
Overview
Adds a "Self Report Metrics" step to the
TestsGitHub Actions workflow to publish JUnit results as OpenTelemetry metrics viaredis-developer/cae-otel-ci-visibility@v1.0.2.The step is gated to run only on pushes and on same-repo PRs, and sends metrics to a Grafana OTLP endpoint using the
SELF_CHECK_OTEL_AUTHORIZATION_TOKENsecret (withOTEL_EXPORTER_OTLP_PROTOCOLset tohttp/protobuf).Written by Cursor Bugbot for commit 8fc4071. This will update automatically on new commits. Configure here.