Skip to content

Release empty cache backing storage during eviction - #23

Open
Xyhlon wants to merge 1 commit into
typst:mainfrom
Xyhlon:fix/release-empty-cache-storage
Open

Xyhlon wants to merge 1 commit into
typst:mainfrom
Xyhlon:fix/release-empty-cache-storage

Conversation

@Xyhlon

@Xyhlon Xyhlon commented Sep 29, 2026

Copy link
Copy Markdown

Eviction removes cached entries but retains their backing allocations. In a long-running Tinymist process, empty accelerator maps and their pool retained approximately 11.62 GiB after cache clearing.

This change releases accelerator storage on evict(0), releases unused accelerator map buffers during ordinary eviction, and resets empty call trees. Accelerator lookup rechecks ID validity and pool length across lock transitions because full eviction can now shrink the pool.

Recently used buffers remain reusable, and eviction frequency is unchanged. Nonempty trees are not compacted. This keeps the change to 38 added and 20 removed production lines, plus three regression tests for storage release, reinsertion, and concurrent access/eviction.

Validation:

  • Expanded suite: 5 unit tests, 17 integration tests, and 4 doctests passed with QUICKCHECK_TESTS=10000 cargo test --all-features -- --test-threads=1.
  • The same 26 tests passed again on the final upstream-main-based branch with the default QuickCheck case count.
  • All 5 no-default-features unit tests passed; formatting and Clippy checks passed with existing warnings.
  • Tinymist v0.15.8 built with the patched dependency and completed a 60-edit replay with identical semantic-token output. After explicit cache clearing, allocator in-use memory fell from 1,164.5 MiB to 143.0 MiB; RSS fell from 2,621.0 MiB to 2,080.9 MiB.

The replay is a single workload measurement using a private document; the regression tests exercise storage retention independently. glibc still retained free arena pages, so releasing these containers does not eliminate all RSS retention. Tests used Rust 1.98.1; the expanded suite ran serially because existing global-eviction tests can interfere with concurrent property tests.

Related history: Tinymist #161 increased eviction frequency and was reverted in #173 because of cache thrashing. This change keeps the eviction schedule intact.

@laurmaedje

Copy link
Copy Markdown
Member

Thanks for the PR. I think the general idea of being more careful with freeing up memory is sensible. That said, we generally don't merge AI-generated PRs and this PR shows various signs of being generated. We don't currently have dedicated details about our AI policy in typst/comemo (only in typst/typst), but we'll look into adjusting that.

@Xyhlon

Xyhlon commented Sep 29, 2026

Copy link
Copy Markdown
Author

No worries, i was using tinymist and over a day of usage the ram allocation grew to 30GiB (kinda making it unusable on most machines) and i had no time to look into it much, hence i let AI do the triage. It found something and i thought either, i leave it be or let you know what it found. Sorry for the inconvenience. I don't want to cause any extra work, but i hoped somebody would look into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants