Repository navigation
Add forecast operation to generated agentic maintenance workflow - #31422
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Implemented in Forecast issue publishing now renders a markdown table (workflow / sampled runs / P50 ET) instead of embedding raw JSON. I also validated the zero-projection behavior: in this maintenance job, ET values come from local |
|
@copilot add call to logs to warm cache. Fail command if cache is missing |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Addressed in f86f945. The forecast job now warms cache with |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Root cause confirmed from the run logs: |
|
@copilot increase forecasts time out to 60min Fix warning |
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Addressed in 208b61c and 66488b0. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
| with: | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
| script: | | ||
| const fs = require('node:fs'); |
There was a problem hiding this comment.
Addressed in 0087c0a.
- Moved the inline forecast issue-generation script into
actions/setup/js/create_forecast_issue.cjs. - Updated generated maintenance workflow to call that script via
setup_globals.cjs+main(). - Added tests in
actions/setup/js/create_forecast_issue.test.cjsand updated workflow generator assertions accordingly.
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Addressed in
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot downloads the index cache memory if available to accelerate logs gathering before actually calling logs |
|
@copilot downloads the index cache memory if available to accelerate logs gathering before actually calling |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Implemented in d7a4912. The forecast job now restores |
There was a problem hiding this comment.
Pull request overview
Adds a new forecast maintenance operation to the generated agentic maintenance workflow and extracts forecast issue rendering into a dedicated script for clearer output and better CI reliability.
Changes:
- Add
forecast_reportjob (60m timeout) and wire it into the generated and checked-in maintenance workflow. - Warm forecast inputs via
gh aw logswith cache restore/save around.github/aw/logs, then rungh aw forecast --jsonwith filtered experimental-warning stderr. - Extract forecast issue creation into
actions/setup/js/create_forecast_issue.cjsand add targeted Vitest coverage.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/maintenance_workflow_yaml.go | Generates the new forecast_report maintenance job and adds forecast as a selectable operation. |
| pkg/workflow/maintenance_workflow_test.go | Extends generator tests to assert forecast_report wiring, conditions, caching steps, and setup-go pin counts. |
| .github/workflows/agentics-maintenance.yml | Updates the checked-in workflow to include the new forecast operation/job and caching/warm-up steps. |
| actions/setup/js/create_forecast_issue.cjs | New script that reads forecast JSON and creates a formatted issue body (table + diagnostics + source run link). |
| actions/setup/js/create_forecast_issue.test.cjs | New Vitest coverage for formatting/diagnostics and missing-report behavior. |
| docs/src/content/docs/guides/ephemerals.md | Documents the new forecast operation in the ephemerals maintenance operations list/details. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
docs/src/content/docs/guides/ephemerals.md:120
- The
forecastoperation description says the issue will include the JSON payload in a fenced block, but the implementation now renders a markdown table with formatted ET values, diagnostics, and a source-run footnote (no embedded JSON block). Please update this doc line to match the new issue format.
- **`validate`**: Runs `gh aw compile --validate --no-emit --zizmor --actionlint --poutine --verbose`. If errors or warnings are found, creates or updates a GitHub issue titled `[aw] workflow validation findings` with the full output.
- **`activity_report`**: Runs `gh aw logs --format markdown` for the last 24 hours, 7 days, and 30 days (up to 1000 runs each), then creates an issue titled `[aw] agentic status report` with all three time-range sections as collapsible `<details>` blocks. Downloaded logs are cached under `./.cache/gh-aw/activity-report-logs`. The job has a 2-hour timeout and skips the 30-day query when the GitHub API is rate-limited.
- **`forecast`**: Runs `gh aw forecast --repo <owner/repo> --json`, writes the output to `./.cache/gh-aw/forecast/report.json`, then creates an issue titled `[aw] workflow forecast report` with the JSON payload embedded in a fenced block.
- Files reviewed: 6/6 changed files
- Comments generated: 2
| const p50 = workflow?.monte_carlo?.p50_projected_effective_tokens ?? workflow?.projected_effective_tokens ?? 0; | ||
| return [escapeCell(workflow.workflow_id), workflow.sampled_runs ?? 0, Number(p50)]; |
| | `clean_cache_memories` | Clean up outdated cache-memory entries (same as the automated scheduled cleanup) | | ||
| | `validate` | Run full workflow validation with all linters and file an issue if findings are detected | | ||
| | `activity_report` | Generate a repository activity report for the last 24 hours, week, and month, and create an issue with the results | | ||
| | `forecast` | Run a workflow token-usage forecast and create an issue with the JSON results | | ||
|
|
✨ Enhancement
What does this improve?
This PR adds and hardens the generated
forecast_reportmaintenance operation so it produces clearer, more actionable forecast issues and is more reliable in CI.Why is this valuable?
Forecast issues are now easier to read, link back to their source run, provide better diagnostics when projected ET values are zero, and are easier to maintain/test by moving issue-generation logic into a dedicated script file.
Implementation approach:
forecast_reportto generated maintenance workflow output for forecast operations.forecast_reporttimeout to 60 minutes.gh aw logs --repo "${{ github.repository }}" --start-date -30d --count 1500.github/aw/logscache can be reused when available before callinggh aw logs, then persisted for subsequent runs.actions/setup/js/create_forecast_issue.cjsand updated generated workflow/script wiring to call the file.Testing
create_forecast_issue.cjs.