Skip to content

Tool results have no image channel, so the model cannot see screenshots it captures #824

Closed

Description

@Vasanthdev2004

tools.Result (internal/tools/types.go:92-98) carries Output string and no image or content-item field, so no tool can hand the model an image. There is also no view_image tool: grep -rni view_image internal/ returns nothing.

The sharp edge is our own screenshot tooling. internal/tools/local_capture.go implements browser_screenshot, desktop_screenshot and browser_pdf. They write a PNG to disk, set Meta["artifact_path"], and return the sentence "Artifact captured: ". Nothing reads artifact_path afterwards. So the model asks for a screenshot, is told one exists, and cannot look at it. From the model's side the tool may as well have done nothing.

Worth being clear that Zero is not image-blind. The input pipeline is complete and reasonably mature: zeroruntime.ImageBlock, Message.Images, internal/imageinput for pasted and attached images, media-type normalisation, and ACP image blocks. A user can hand the model an image today. What is missing is the model-initiated leg: a tool result cannot contain one.

Two pieces:

  1. An image channel on tools.Result that the agent loop turns into Message.Images on the tool-result message.
  2. A view_image tool so the model can open a path it has been given, which also covers files the user mentions but does not attach.

Once (1) exists, the screenshot tools become useful with a one-line change each, and MCP image blocks have somewhere to go.

Activity

  1. added a commit that references this issue on Aug 6, 2026
    cd0eb19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions