Skip to content

Make the directory listing limit configurable in the system prompt #2541

Closed

Description

@Cluster444

Right now the files pushed into the system prompt as output from Ripgrep.tree are pretty much useless on a large enough project. The breadth-first nature of it makes it prioritize shallow files, and most useful files are more deeply nested in a lot of projects. For example here's some stats from my rails app

git ls-files - 1168
git ls-files app - 667
git ls-files app/models app/controllers - 228 (100% of this gets ignored currently)

And on top of that, the output is kind of nonsensical, i'm not sure the model is going to make heads of tails of this. For example this is what my /app directory ends up looking like in the prompt

app/
assets/
builds/
[1 truncated]
[2 truncated]
channels/
application_cable/
[2 truncated]
[7 truncated]
constants/
date_formats.rb
controllers/
baskets/
[3 truncated]
[41 truncated]
errors/
user_facing_error.rb
helpers/
content_filters/
[1 truncated]
[37 truncated]
javascript/
actions/
[1 truncated]
[6 truncated]
jobs/
chat/
[1 truncated]
[4 truncated]
lib/
assertion.rb
mailers/
application_mailer.rb
[2 truncated]
models/
[64 truncated]
views/
[32 truncated]

There's at least two config options that would help improve this:

  1. Make the limit configurable
  2. Allow for directory scopes to be applied

And some kind of better formatting that gives a relationship between directories and files as this is all disjointed. Maybe something like

# app/controllers
- this_controller.rb
- that_controller.rb
# app/models
- this.rb
- that.rb
# app/models/this
- taggable.rb
- joinable.rb

this would keep the paths from being repeated, but also give them some element of emphasis that would actually match things like error output.

Activity

  1. github-actions commented on Sep 11, 2025

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Feel free to ignore if none of these address your specific case.

  2. rekram1-node commented on Sep 11, 2025

    @rekram1-node
    Collaborator

    just going to assign it to you since I saw the exchange in discord

  3. taqtiqa-mark commented on Oct 14, 2025

    @taqtiqa-mark

    @Cluster444 IIUC, this will be affected by allowing users more control of the context via /add and /drop, see #1990. Is that correct ?
    There is also the question of whether the raw files are added to the context or a tree-sitter extract?

  4. Cluster444 commented on Oct 15, 2025

    @Cluster444
    ContributorAuthor

    No I dont think that would affect what I was getting at here. This is a builtin that is dumping a directory tree directly into system instructions, and on even small projects it renders it quite useless because it saturates the 200 file limit with all the 1st and 2nd tier files, where most source code is 3-5 dirs deep in many projects, so they never show up.

  5. taqtiqa-mark commented on Oct 15, 2025

    @taqtiqa-mark

    This is a builtin that is dumping a directory tree directly into system instructions...

    Yes, that destination is just plain, flat-out wrong.

    ... where most source code is 3-5 dirs deep in many projects, so they never show up.

    Correct so a agent or command /add folder/needed/one/*.ts and /add folder/two/*.html, where the tree-sitter data from those folders is added to the Context-Text, and not the User-Prompt-text, and definitely not the System-Prompt-Text.

    That would be a sane approach.

  6. taqtiqa-mark commented on Oct 15, 2025

    @taqtiqa-mark

    Substantiating this claim:

    Yes, that destination is just plain, flat-out wrong.

    See: #2108 (comment)

  7. thdxr commented on Mar 25, 2026

    @thdxr
    Member

    To stay organized issues are automatically closed after 90 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions