Skip to content

Fix agent action failures from live game testing #5

Closed

Description

@jkbennitt

Context

First live game test (5 ticks, Nemotron 3 Nano 4B, Crashlanded scenario) achieved 87% action execution rate. The 13% failures are all agent prompt/schema issues — the RIMAPI endpoints work correctly when given valid parameters.

Failures to Fix

1. place_blueprint — "Blueprint is null"

Root cause: Agent sends raw params dict but RIMAPI's BuilderService expects a specific PasteAreaRequestDto with map ID, area bounds, and building defs. The agent doesn't know the blueprint format.
Fix: Either teach agents the correct blueprint schema in their system prompt, or add a simpler place_building(def_name, x, z) wrapper that constructs the DTO.

2. haul_resource — "Pawn could not accept job HaulToCell"

Root cause: HaulToCell is not the correct RimWorld JobDef name. The actual haul jobs are HaulToContainer, HaulToCell (needs valid target cell), or just Haul with a target thing.
Fix: Map haul_resource to Haul JobDef with the target thing ID from params, or use work priority approach (set hauling priority to 1) instead of direct job assignment.

3. toggle_power — "Building not found: 0"

Root cause: Agent sends building_id=0 because it doesn't have building IDs in its filtered state. The state adapter doesn't expose individual building data.
Fix: Enrich the state adapter to include building IDs from /api/v1/map/buildings, so agents can reference real buildings.

4. Missing target_colonist_id on pawn-targeting actions

Root cause: Agents sometimes output actions (bed rest, medicine, draft, recreation) without specifying which colonist. The executor now skips these gracefully, but they should be valid.
Fix: Reinforce in system prompts that pawn-targeting actions MUST include target_colonist_id. Possibly add JSON schema validation that rejects plans with missing IDs for pawn actions.

Impact

Fixing these would push execution rate from 87% → ~95%+. The guards in the executor prevent crashes, but the actions should succeed, not be skipped.

Files Likely Modified

  • src/rle/agents/base_role.py — system prompt improvements
  • src/rle/agents/resource_manager.py — haul action format
  • src/rle/agents/construction_planner.py — blueprint format
  • src/rle/orchestration/action_executor.py — haul job mapping
  • src/rle/rimapi/client.py — state adapter enrichment (building IDs)

@CalebisGross — these are all prompt/schema tuning issues from the first live test. The RIMAPI integration is solid. Want to split these up or tackle as a batch?

🤖 Generated with Claude Code

Activity

  1. jkbennitt commented on Mar 22, 2026

    @jkbennitt
    MemberAuthor

    All four fixes shipped and verified with live game test

    What was fixed

    # Failure Fix Commits
    1 place_blueprint — "Blueprint is null" Executor uses designate_area with simple x,z coords instead of raw PasteAreaRequestDto 0e44a5b
    2 haul_resource — "HaulToCell" rejected Set hauling work priority instead of direct job assignment 0e44a5b
    3 toggle_power — "Building not found: 0" get_map() now fetches real buildings from /api/v1/map/buildings 0e44a5b
    4 Missing target_colonist_id System prompt explicitly requires colonist IDs for pawn actions, coords for blueprints, building IDs for power 0e44a5b
    Bonus run_scenario.py missing --provider/--model/--base-url flags Added to match run_benchmark.py CLI interface 25e0215

    Live test results (5 ticks, Crashlanded, Nemotron Nano 4B)

    Metric Before (issue opened) After (fixes applied)
    Execution rate 87% 89%
    Parse rate 100% 100%
    Composite score 0.705 0.707
    Survival 1.000 1.000
    Team confidence ~72% 93%

    The remaining ~11% failures are the 4B model occasionally not following the target_colonist_id instruction. Larger models (120B on OpenRouter) would hit higher compliance. All four root causes identified in this issue are resolved.

    262 tests pass, lint clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions