Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 63 additions & 12 deletions .github/workflows/daily-agentrx-trace-optimizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,14 +66,11 @@ Focus on:

### 1) Build AgentRx input trajectory

Create `/tmp/agentrx/trajectory.json` from MCP-downloaded run data and logs by mapping `runs[]` session records to ordered workflow steps. Include:
- step index
- workflow/run identifiers (`github.workflow_ref`, `github.run_id`)
- status/error signal
- duration/token/cost fields when available (`duration`, `effective_tokens`, `estimated_cost`, `turns`)
- behavior and diagnostics fields when available (`agentic_assessments`, `behavior_fingerprint`, `missing_tool_count`)

Use the last 24h of data and prioritize runs with failures or high latency.
Invoke `trajectory-builder` by passing this exact input block:
```text
run_data_path: /tmp/agentrx/mcp-runs.json
```
Comment on lines +69 to +72
It must produce `/tmp/agentrx/trajectory.json`.

### 2) Run AgentRx pipeline

Expand All @@ -98,6 +95,13 @@ If a later stage fails (for example due to endpoint/auth constraints), continue

### 3) Derive one optimization recommendation

First, invoke `failure-pattern-classifier` by passing this exact input block:
```text
check_path: /tmp/agentrx/runs/gh-aw-daily/check.json
judge_path: /tmp/agentrx/runs/gh-aw-daily/judge.json
```
Capture its markdown table output as the labeled violations list for this section. Then read that labeled table and pick the single highest-impact fix.
Comment on lines +98 to +103

Use AgentRx outputs to identify:
- the most frequent or most expensive failure pattern
- the critical workflow step causing it
Expand Down Expand Up @@ -130,10 +134,11 @@ Body structure:
<details>
<summary>AgentRx Artifacts</summary>

- IR summary
- Invariant/checker highlights
- Judge classification output (if available)
- Known limitations (missing session fields, missing attributes, auth-limited stages)
Invoke `artifacts-summarizer` by passing this exact input block:
```text
run_dir: /tmp/agentrx/runs/gh-aw-daily
```
Paste its markdown output as the body of this details block.

</details>

Expand All @@ -157,3 +162,49 @@ Body structure:
- Otherwise, always call `create_issue` once.

{{#runtime-import shared/noop-reminder.md}}

## agent: `trajectory-builder`
---
description: Builds AgentRx trajectory input from MCP run and log data
model: small
---
You are a structured-data extraction agent.
Expected input format:
`run_data_path: <absolute-path-to-mcp-run-data-json>`
Read the file at `run_data_path` and create `/tmp/agentrx/trajectory.json`.
Use the last 24h of data and prioritize failed or high-latency runs.
Map `runs[]` session records to ordered workflow steps.
Include when present: step index, `github.workflow_ref`, `github.run_id`, status/error signal, `duration`, `effective_tokens`, `estimated_cost`, `turns`, `agentic_assessments`, `behavior_fingerprint`, `missing_tool_count`.
Output valid JSON only and write it to `/tmp/agentrx/trajectory.json`.

## agent: `artifacts-summarizer`
---
description: Summarizes AgentRx stage artifacts for issue details output
model: small
---
You are an artifact summarization agent.
Expected input format:
`run_dir: <absolute-path-to-agentrx-run-dir>`
Read AgentRx stage outputs from `run_dir` (`ir`, `static`, `dynamic`, `check`, `judge`, `report`).
Produce concise markdown bullets for the AgentRx Artifacts details block.
Cover: IR summary, invariant/checker highlights, judge classification output when available, and known limitations such as missing fields or auth-limited stages.
Do not invent values.

## agent: `failure-pattern-classifier`
---
description: Classifies AgentRx violations into predefined optimization fix types
model: small
---
You are a violation classification agent.
Expected input format:
`check_path: <absolute-path-to-check-artifact-json>`
`judge_path: <absolute-path-to-judge-artifact-json>`
Read `check_path` (required) and `judge_path` (if present).
Label every AgentRx violation with exactly one fix type from this taxonomy:
- prompt tightening to reduce invalid tool invocations
- adding precondition checks before expensive tools
- improving retry/backoff strategy
- reducing token-heavy context payloads
- adding missing telemetry attributes for better triage
Return a markdown table with columns: violation, evidence, fix_type, rationale.
Use only provided AgentRx artifacts.