Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 49 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,29 @@ inputs:
IoU (intersection-over-union) threshold used by incremental mode to decide
whether a new multi-line comment overlaps an existing one. Two single-line
comments match when on the same line; single- vs multi-line never match.
Value in (0, 1]; ignored unless incremental is true.
Value in (0, 1]. Read only when incremental is true: resolve_outdated
decides overlap by any shared line, with no threshold, because a missed
overlap there resolves a thread whose finding is still live.
required: false
default: '0.6'
resolve_outdated:
description: >-
Resolve the action's own outdated inline review threads (threads GitHub
has already marked outdated because their lines no longer exist in the
diff). Threads a human replied to, threads already resolved, and threads
whose lines a finding from the current run still covers are never touched.
One of: 'false' (default, does nothing and makes no API calls), 'report'
(log what would be resolved, change nothing — run this first), 'true'
(actually resolve). 'true' requires the calling workflow to grant
`contents: write` in its own `permissions:` block: GitHub gates the
resolve mutation on repository write access, not on the pull-request
scope, so `pull-requests: write` alone returns FORBIDDEN. Without it the
action warns and continues. 'report' issues no mutation, so it cannot
tell you in advance whether 'true' would be permitted.
A thread is covered when a current finding shares ANY line with it; the
incremental_overlap_threshold IoU test is not used here.
required: false
default: 'false'
review_comment_batch_size:
description: >-
Maximum number of inline comments packed into a single createReview call.
Expand Down Expand Up @@ -229,6 +249,17 @@ outputs:
comments_failed:
description: Number of inline comments that failed to post.
value: ${{ steps.post.outputs.comments_failed }}
comments_resolved:
description: >-
Number of outdated bot review threads resolved this run. Always 0 unless
resolve_outdated is 'true'.
value: ${{ steps.post.outputs.comments_resolved }}
comments_resolved_preview:
description: >-
Number of candidate threads resolve_outdated 'report' mode found. This is
the uncapped count, so it can exceed the 50 threads a single 'true' run
resolves. Always 0 in 'false' and 'true' modes.
value: ${{ steps.post.outputs.comments_resolved_preview }}
summary_comment_url:
description: URL of the posted/updated summary comment, if any.
value: ${{ steps.post.outputs.summary_comment_url }}
Expand Down Expand Up @@ -397,6 +428,7 @@ runs:
MAX_TOKENS_BUDGET_INPUT: ${{ inputs.max_tokens_budget }}
LLM_REASONING_EFFORT_INPUT: ${{ inputs.llm_reasoning_effort }}
STREAM_PROGRESS_INPUT: ${{ inputs.stream_progress }}
RESOLVE_OUTDATED_INPUT: ${{ inputs.resolve_outdated }}
shell: bash
run: |
if [[ ! "$REVIEW_TASK_TIMEOUT" =~ ^[0-9]+$ ]]; then
Expand Down Expand Up @@ -477,6 +509,20 @@ runs:
esac
echo "STREAM_PROGRESS=$NORMALIZED_STREAM_PROGRESS" >> "$GITHUB_ENV"

# Thread cleanup mode: 'false' touches nothing and makes no API calls,
# 'report' logs what it would resolve, 'true' resolves. Empty maps to
# 'false' so a caller that passes an unset variable stays opt-in.
NORMALIZED_RESOLVE_OUTDATED="$(printf '%s' "$RESOLVE_OUTDATED_INPUT" | tr '[:upper:]' '[:lower:]')"
case "$NORMALIZED_RESOLVE_OUTDATED" in
"") NORMALIZED_RESOLVE_OUTDATED=false ;;
false|report|true) ;;
*)
echo "::error::resolve_outdated must be one of: false, report, true (got '$RESOLVE_OUTDATED_INPUT')"
exit 1
;;
esac
echo "RESOLVE_OUTDATED=$NORMALIZED_RESOLVE_OUTDATED" >> "$GITHUB_ENV"

- name: Install OpenCodeReview
shell: bash
env:
Expand Down Expand Up @@ -954,6 +1000,7 @@ runs:
# Resolved by "Resolve PR refs", which fails the job when no number is
# available; context.issue.number resolves nothing on a workflow_run.
OCR_PR_NUMBER: ${{ env.PR_NUMBER }}
OCR_RESOLVE_OUTDATED: ${{ env.RESOLVE_OUTDATED }}
with:
github-token: ${{ inputs.github_token }}
script: |
Expand Down Expand Up @@ -996,4 +1043,5 @@ runs:
rangeMode: process.env.OCR_RANGE_MODE || '',
rangeFrom: process.env.OCR_RANGE_FROM || '',
rangeTo: process.env.OCR_RANGE_TO || '',
resolveOutdated: process.env.OCR_RESOLVE_OUTDATED,
});
63 changes: 61 additions & 2 deletions examples/github_actions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,8 @@ The action posts a summary issue comment plus inline review comments. Two inputs
|-------|---------|-------------|
| `sticky_summary` | `'true'` | Update an existing summary comment in place instead of posting a new one each run. |
| `incremental` | `'false'` | Only append inline comments whose `(path, line range)` does not overlap an existing bot review comment. History is never deleted (non-destructive). |
| `incremental_overlap_threshold` | `'0.6'` | IoU threshold `incremental` uses to decide whether a multi-line comment overlaps an existing one. Two single-line comments match on the same line; single- vs multi-line never match. Ignored unless `incremental` is `'true'`. |
| `incremental_overlap_threshold` | `'0.6'` | IoU threshold `incremental` uses to decide whether a multi-line comment overlaps an existing one. Two single-line comments match on the same line; single- vs multi-line never match. Read only when `incremental` is enabled — `resolve_outdated` treats any shared line as an overlap, with no threshold, because a missed overlap there closes a thread whose finding is still live. |
| `resolve_outdated` | `'false'` | Resolve the action's own outdated inline threads. `'false'` does nothing; `'report'` logs what it would resolve; `'true'` resolves them. See below. |

```yaml
- uses: alibaba/open-code-review@main
Expand Down Expand Up @@ -346,6 +347,64 @@ Three properties are worth knowing before you enable it:
>
> Because a sticky summary keeps its original author, switching a repository from a custom App token to the default one leaves the old comment attributed to the old app, and every run reports `author_unverified` until that comment is deleted. That is the fail-closed direction (a full review, never a skipped range), and the step log names the app it expected.

### Resolve outdated review threads

Over a long PR, the bot's inline comments pile up on code that no longer exists. `resolve_outdated` closes those threads, using GitHub's own `isOutdated` flag — no LLM is involved in the decision, and the action only ever touches threads GitHub has already marked outdated.

| Value | Behavior |
|-------|----------|
| `'false'` (default) | Does nothing. No GraphQL calls are made at all. |
| `'report'` | Lists what would be resolved, and why each other thread was skipped. Changes nothing. |
| `'true'` | Resolves them. |

Start with `'report'` for a few PRs and read the `[resolve-outdated]` log line before switching to `'true'`.

```yaml
- uses: alibaba/open-code-review@main
with:
resolve_outdated: 'report'
```

`'true'` needs `contents: write` (see below). Rather than adding that to the job that
reviews the diff, keep it in a job of its own that never checks out the pull request:

```yaml
jobs:
review:
permissions:
contents: read
pull-requests: write
steps:
- uses: alibaba/open-code-review@main
with:
resolve_outdated: 'false'

resolve:
needs: review
# No actions/checkout in this job. The token that can write to the
# repository is never in a process that has the PR's code in it.
permissions:
contents: write
pull-requests: read
steps:
- uses: alibaba/open-code-review@main
with:
resolve_outdated: 'true'
```

A thread is left alone unless **the action created it**. Ownership is decided by the marker OCR writes into every inline comment it posts, not by the comment's author — under the default `GITHUB_TOKEN` every workflow in your repository posts as `github-actions[bot]`, so a sibling workflow's review threads would otherwise be indistinguishable from OCR's own. Beyond that, a thread is left alone whenever **a human has replied** to it, when it is already resolved, when a finding from the current run still covers its lines, or when it has more comments than one API page returns (so a reply the action cannot see is never resolved over). Resolution also runs only after a run that actually produced findings: a run that failed to parse OCR's output, or that reported nothing, resolves nothing — "the model said nothing this time" is not evidence the old findings are gone. At most 50 threads are resolved per run; the rest carry over to the next one. Resolution is sequential and paced by `OCR_SUCCESS_DELAY` (2000 ms by default) to stay clear of GitHub's secondary rate limiter, so a full batch of 50 adds roughly 100 s to the job — lower `OCR_SUCCESS_DELAY` if that matters more to you than the pacing.

Outputs: `comments_resolved` (threads resolved, `'true'` mode) and `comments_resolved_preview` (candidate threads found, `'report'` mode). The preview is the uncapped count, so a 60-thread backlog previews as 60 even though a run resolves at most 50. Both are always present.

Things worth knowing before you turn this on:

- **`'true'` requires `contents: write`, and `pull-requests: write` will not do.** GitHub gates `resolveReviewThread` on repository write access rather than on the pull-request scope, and for an Actions token that access comes from Contents. Measured on a scratch PR with three jobs differing only in their `permissions:` block: `pull-requests: write` with `contents: read` returns `FORBIDDEN`, while `contents: write` with only `pull-requests: read` resolves the thread. Without the permission the action logs a warning naming it and continues, so the review itself still posts.
- **`'report'` cannot tell you whether `'true'` would work.** It issues no mutation, so it never learns whether the token has the permission, and a clean-looking preview can still hit `FORBIDDEN` the moment you switch over. GraphQL's `viewerCanResolve` looks like the answer and is not: in the measurement above it read `false` in all three jobs, including the two whose mutations then succeeded.
- **A force-push can mark a live finding's thread outdated.** GitHub sets `isOutdated` when the thread's lines are no longer in the diff, and it stays set even after a force-push that lands identical content, so a rebase can outdate a thread whose finding is still real. The current-run overlap check catches this when the model re-reports the finding; if it doesn't, the thread closes while the problem remains. This is the main reason to run `'report'` first.
- **It only does anything on re-runs.** A thread can only become outdated after a later push, so the feature is exercised only by workflows that review on update (`types: [opened, synchronize, reopened]`, as in the sample workflow above). This repository's own review workflow triggers on `opened` only and never reaches the resolution path — do not read its runs as evidence the feature works for you.
- **The run's findings and GitHub's `isOutdated` can be computed against different commits.** Findings are posted against the head the review actually ran on (`resolved_head` in the result manifest), while GitHub recomputes `isOutdated` against the pull request's current head. A push that lands mid-run leaves a window where a thread is outdated relative to code this run never saw. Same class as the force-push case above, and the same mitigation.
- **Resolving is not deleting.** Resolved threads collapse but stay readable, and anyone can unresolve one.

### Adjust retry and delay settings

When posting review comments individually (fallback mode), the action honors GitHub rate-limit headers (`retry-after`, `x-ratelimit-*`) with exponential backoff. The retry strategy follows GitHub's documented guidance for REST API rate limits — see [Rate limits for the REST API](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2026-03-10) for details on primary/secondary rate limits and recommended retry behavior:
Expand Down Expand Up @@ -547,7 +606,7 @@ The action does not use an `OCR_DEBUG` flag. To diagnose a run:

- **Artifacts**: with `upload_artifacts: 'true'` (the default), the raw `ocr-result.json` and `ocr-stderr.log` are uploaded as workflow artifacts named `ocr-review-result-<run_id>-<run_attempt>`. Download them from the run's **Artifacts** section.
- **Step log**: the "Run OpenCodeReview" step prints both the JSON result and stderr to the workflow log.
- **Action outputs**: the step exposes `comments_total`, `comments_inline`, `comments_skipped`, `comments_failed`, and `summary_comment_url` outputs — inspect them in the job's step outputs.
- **Action outputs**: the step exposes `comments_total`, `comments_inline`, `comments_skipped`, `comments_routed`, `comments_failed`, `comments_resolved`, `comments_resolved_preview`, and `summary_comment_url` outputs — inspect them in the job's step outputs.
- **GitHub step debug**: for verbose Actions runner diagnostics, enable the repository secret `ACTIONS_STEP_DEBUG=true` (standard GitHub Actions mechanism).

To stop uploading the raw artifacts, set `upload_artifacts: 'false'`.
45 changes: 45 additions & 0 deletions scripts/github-actions/action-contract.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -1010,6 +1010,50 @@ function testValidateInputsValidatesStreamProgress() {
}
}

function testValidateInputsValidatesResolveOutdated() {
const validation = validationStep();
assert.ok(validation, "action.yml must retain input validation");
for (const value of ["yes", "1", "on", "resolve"]) {
const fixture = makeFixture();
try {
const result = runStep(validation, inputValues({ resolve_outdated: value }), fixture);
assert.notStrictEqual(
result.status,
0,
`resolve_outdated=${JSON.stringify(value)} should be rejected; ${resultDescription(result)}`
);
assert.match(
`${result.stdout}\n${result.stderr}`,
/resolve_outdated must be one of/,
`rejection for resolve_outdated=${JSON.stringify(value)} must name the input; ${resultDescription(result)}`
);
} finally {
removeFixture(fixture);
}
}
const acceptance = [
["TRUE", "true"],
["Report", "report"],
["", "false"],
["false", "false"],
];
for (const [value, expected] of acceptance) {
const fixture = makeFixture();
try {
const result = runStep(validation, inputValues({ resolve_outdated: value }), fixture);
assert.strictEqual(result.status, 0, `resolve_outdated=${JSON.stringify(value)} should be accepted; ${resultDescription(result)}`);
const exported = readEnvAssignments(path.join(fixture.dir, "github-env"));
assert.strictEqual(
exported.RESOLVE_OUTDATED,
expected,
`validation must export ${JSON.stringify(expected)} for resolve_outdated=${JSON.stringify(value)}`
);
} finally {
removeFixture(fixture);
}
}
}

function testRunKeepsAgentAudienceAndLogFileByDefault() {
const run = stepNamed("Run OpenCodeReview");
assert.ok(run, "action.yml must retain the Run OpenCodeReview step");
Expand Down Expand Up @@ -1824,6 +1868,7 @@ const TESTS = [
["llm_reasoning_effort rejects values outside the OpenAI/GLM vocabulary", testValidateInputsRejectsInvalidReasoningEffort],
["stream_progress defaults to false and describes [ocr] progress", testStreamProgressInputDefaultsToFalse],
["stream_progress validation accepts true/false case-insensitively", testValidateInputsValidatesStreamProgress],
["resolve_outdated validation fails fast on an unknown value", testValidateInputsValidatesResolveOutdated],
["Run OpenCodeReview keeps --audience agent and the log file by default", testRunKeepsAgentAudienceAndLogFileByDefault],
["Run OpenCodeReview streams live progress when opted in", testRunStreamsProgressWhenOptedIn],
["llm_extra_body defaults to disabling thinking", testLlmExtraBodyDefaultDisablesThinking],
Expand Down
Loading