[fyhon]

Summary

Replace class-level lru_cache with per-instance cache in compile_compatible_method_lru_cache to prevent memory leaks when model instances are deleted.

Fixes #45412

Changes

• Modified src/transformers/pytorch_utils.py: For methods (detected via first parameter named self), the decorator now stores a per-instance lru_cache in the instance's __dict__ using a weakref destructor for cleanup. For standalone functions, the original shared cache behavior is preserved.

Root Cause

compile_compatible_method_lru_cache wraps methods with a class-level functools.lru_cache. Since self is the first positional argument, lru_cache holds a strong reference to the model instance as a cache key, preventing garbage collection even after del model + gc.collect(). This affects RT-DETR, RT-DETR-v2, MaskFormer, Mask2Former, OneFormer, SAM2, and 20+ other models that use this decorator.

Duplicate Check

• gh pr list --state open --search "45412" → no duplicate PRs
• gh pr list --state open --search "memory leak lru_cache" → no duplicate PRs
• Prior PRs fix(pytorch_utils): per-instance lru_cache to fix RT-DETR memory leak #45708 and fix: per-instance cache in compile_compatible_method_lru_cache to prevent memory leak #45780 were closed without merge

Testing

import gc, weakref, functools, inspect

# Simulated test of the fix logic
class Model:
    @compile_compatible_method_lru_cache
    def compute(self):
        return [1, 2, 3]

m = Model()
ref = weakref.ref(m)
m.compute()  # populate cache
del m
gc.collect()
assert ref() is None  # instance is properly garbage collected

Test 1 PASSED: cache returns same object
Test 2 PASSED: instance is garbage collected after del
Test 3 PASSED: standalone function cache works
Test 4 PASSED: different instances have independent caches
Test 5 PASSED: deleting one instance does not affect another

ALL TESTS PASSED

AI Disclosure

This PR was created with AI assistance (Claude). The human maintainer has reviewed and approved the changes before submission.

[ArthurZucker]

@yonigozlan could you take a look when you get a chance? This is waiting on a maintainer: yonigozlan said he'd fix it himself after the ViT refactor (#41693, merged 2026-05-08).