Repository navigation
Fix #66: Recover from malformed tool calls and stop after repeated format errors - #75
Conversation
…rs (#66) Classify malformed tool calls (invalid/non-object JSON arguments, unknown tool, missing required field, wrong JSON type) as format errors: skip the tool, return a short uniform correction, and end the run with RepeatedFormatError after N consecutive all-malformed steps (default 3, config max_format_errors, flag --max-format-errors). Record counts and markers in the trajectory, --json events and summary; persist the counter across --resume.
b09f8dc to
051ebe1
Compare
|
Rebased onto main by the hourly merge job (head now 051ebe1). Two notes on the rebase:
|
Summary
Executor.FormatErrorcheck before dispatch: a call is a format error when its arguments are invalid JSON / not an object, the tool name isn't in the run's catalog, a schemarequiredfield is missing (or null), or a field has the wrong JSON type. Applies to MCP tool schemas too.format error: <reason>. Call one of: <tools> with arguments matching its schema.(capped at 500 chars). Well-formed calls in the same response run as before.RepeatedFormatError, empty submission, non-zero exit; limits are still checked first.max_format_errorsand flag--max-format-errors(flag overrides config). Must be an integer >= 1.tool.resultgets"format_error": true;resultevent getsformat_errors; summary line showsN format errors(+ last reason on RepeatedFormatError); trajectory getsformat_errors,consecutive_format_errors,format_error_messages(message indexes),last_format_error. Original arguments are kept in the trajectory.RepeatedFormatErrorcheckpoints are resumable like limit statuses.Decisions on open questions
Completed(unchanged).0is rejected (exit 2 for the flag); no "no cap" mode.finish_reason=length) detection not added.schema_versionbump; fields are additive/omitempty and ToolSignature is unchanged, so existing checkpoints still resume.RepeatedFormatError,max_format_errors/--max-format-errors,format_error.Other choices:
--read-only, calling a non-read-only built-in keeps the existing read-only denial (not a format error); a truly unknown tool is a format error (one test expectation updated accordingly).0,-1,abc,1.5) fail before any provider request with a message namingmax_format_errors, but exit 1 like every other config error (AC8 asked for 2; flags do exit 2).1.5is explicitly rejected (yaml would otherwise truncate it to 1).(default 3)); usage golden updated for the wider flag column.Tests
internal/agent/format_test.gocovers AC1–AC7, AC9, AC11 (trajectory markers, resume with consecutive=2 → stops on next malformed step, unmarked malformed call rejected by validation), plus wrong-type and mixed-step reset.cmd/tinyagent/format_test.gocovers AC5/AC7 end-to-end via a fake OpenAI server (default 3, flag 1, config 5, flag overrides config), AC8 (flag/config validation, zero provider requests), AC10 (--jsonstays pure JSONL,format_errorandresult.status/format_errors). Existing suite unchanged otherwise (AC12). gofmt/vet/build/test pass locally.Closes #66