Skip to content

Analyze copilot workflow runs to - update the javascript log parsing string that writes to the step summary - make sure the error pattern in the copilot engine are accurate - update the "logs" command to extract information about the runs. Use the fo... - #1037

Closed
pelikhan with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-ef706cb5-d8c8-4186-a1b1-e6d9a6e1a23f

Conversation

Copilot AI commented Sep 26, 2025 •

Copy link
Copy Markdown
Contributor

Enhanced Copilot Workflow Analysis - Updated with Real Log Format

Major update based on actual Copilot CLI log analysis:

  • Create sample copilot log data for testing and understanding the actual format (used provided sample)
  • Analyze current JavaScript log parsing in parse_copilot_log.cjs - completely rewritten
  • Update JavaScript log parsing to extract copilot CLI configuration, tools, and debug information
  • Review and update error patterns in copilot_engine.go based on copilot CLI error formats
  • Enhance logs command extraction capabilities for copilot runs
  • Create comprehensive tests for the copilot log parser
  • Test the changes with actual copilot workflow runs (pending real workflow execution)
  • Validate that step summaries are properly formatted and informative

Key Discovery

The actual Copilot CLI logs have a very different structure than expected:

  • Structured JSON configuration blocks with START-GROUP/END-GROUP markers
  • Timestamped debug messages with [DEBUG], [INFO], [ERROR] levels
  • Tool definitions as JSON function schemas
  • Configuration details including model, integration ID, repository info
  • Request/response tracking with unique request IDs

Updated JavaScript Parser

New approach for real Copilot CLI format:

  • ✅ Parses structured JSON configuration blocks
  • ✅ Extracts model information and integration details
  • ✅ Lists available tools from function definitions
  • ✅ Tracks API request/response interactions
  • ✅ Provides debug information summary
  • ✅ Handles partial or interrupted logs gracefully

Example output:

## 🤖 GitHub Copilot CLI Execution

### 🔧 Configuration
**Model:** sweagent-capi:claude-sonnet-4
**Integration:** copilot-developer-cli
**Repository:** githubnext/temp-repo
**Task:** Summarize and use print the message using the `print` tool...

### 🛠️ Available Tools
* bash
* write_bash
* read_bash  
* stop_bash
* str_replace_editor

### 🔍 Debug Information
* Model catalog loaded successfully
* Model capabilities retrieved

Go Error Patterns Enhanced

  • Added specific patterns for shell command execution errors
  • Added MCP server connection error patterns
  • Enhanced timestamped error/warning detection with proper log levels
  • All patterns tested and validated

Next Steps

  • Update Go ParseLogMetrics to handle the real log structure
  • Enhance tool call detection for the actual format
  • Test with complete copilot workflow runs when available
  • Validate metrics extraction accuracy
Original prompt

Analyze copilot workflow runs to

  • update the javascript log parsing string that writes to the step summary
  • make sure the error pattern in the copilot engine are accurate
  • update the "logs" command to extract information about the runs.

Use the following command to download logs from copilot runs:

./gh-aw logs --engine copilot Dev -c 3

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 3 commits September 26, 2025 11:50
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants