[FEATURE] Consider a reusable mechanism for shared open-file lifetime management

[arnavsharma990]

Is your feature request related to a problem? Please describe.

This issue was identified while fixing #20037, where the FAT filesystem could release a file's cluster chain when the file was unlinked while still open.

The FAT fix in #20101 introduced filesystem-local shared state to track the lifetime of an open file independently of individual file descriptors.

While investigating the same lifetime semantics across other filesystems, I found that NuttX currently has no common mechanism for filesystems that require shared open-file state:

• tmpfs already maintains shared per-file state with reference counting and an unlinked state, and correctly defers reclamation until the last reference is released.
• littlefs handles removal of open files through its existing copy-on-write and orphan handling mechanisms and does not appear to require such an abstraction.
• NFS appears to have a related lifetime-management gap: open state is maintained per open() operation, while removal is performed immediately without a shared open-file lifetime mechanism or silly-rename handling.

This suggests that open-file lifetime management is currently implemented independently by each filesystem, with no reusable mechanism for filesystems that need shared state across open, dup(), unlink, and close operations.

Describe the solution you'd like

I would like to explore an opt-in mechanism for shared open-file lifetime management that can be used by filesystems that require it, without making it a mandatory part of the VFS.

The common pattern could provide a shared per-filesystem object that is looked up or created when a file is opened, reference-counted across open() and dup(), marked for deferred deletion when the file is unlinked, and reclaimed when the final reference is released.

The exact API, ownership model, and placement are intentionally left open for discussion. The fat_shared_s implementation introduced in #20101 provides a concrete example of this lifetime-management pattern, while NFS appears to be a potential second use case.

The mechanism should remain optional so that filesystems that already manage object lifetime correctly do not need to adopt it, and filesystems that do not use it should not incur additional struct file, RAM, or runtime overhead.

If this direction is considered useful, I would be happy to prototype a minimal implementation and evaluate it with an additional filesystem.

Describe alternatives you've considered

• Continue implementing lifetime management independently in each filesystem. This avoids introducing a common abstraction, but requires each filesystem to independently solve the same open, dup(), unlink, and final-close lifetime problems.

• Introduce a mandatory generic inode/object cache in the VFS. This would provide a common mechanism, but appears broader than necessary. tmpfs and littlefs demonstrate that filesystems can already provide correct lifetime semantics using their own internal models. Making such infrastructure mandatory could also impose unnecessary memory or runtime costs on small targets.

• Keep the current FAT-local implementation and address other filesystems independently. This is viable, but the experience from [HELP] FAT: How should unlink defer cluster-chain deletion for open files? #20037/fs/fat: defer cluster-chain free for files unlinked while open #20101 suggests there may be value in identifying a small reusable abstraction for filesystems that encounter the same class of lifetime-management problem.

Verification

• I have verified before submitting the report.

[alisha-1000]

This looks like a useful abstraction, especially since FAT and potentially NFS appear to have similar open-file lifetime requirements.

I think an opt-in mechanism makes sense to avoid adding overhead to filesystems that already handle object lifetime correctly. A shared reference-counted object that tracks open()/dup(), deferred deletion after unlink(), and final reclamation on the last close() seems like a good minimal model.

I’d be interested in exploring a small prototype based on the existing fat_shared_s implementation and evaluating it with NFS as a second use case. This could help determine whether the abstraction is general enough without making the VFS lifecycle machinery unnecessarily complex.

[arnavsharma990]

Thanks for the interest @alisha-1000 ! Since this is still a proposed abstraction, I think it'd actually be best to wait for maintainers to validate this direction before starting to implement.

I've actually been looking into this as a follow-up to #20037/#20101, so if you're interested I'd be happy to pair program with you on the prototype and work through the design together.

Alternatively, if you'd prefer to work on a separate proposal independently I'd be totally fine with that as well, but I'd recommend doing a thorough investigation of the existing FAT, tmpfs, littlefs, and NFS lifetime semantics first since there are some important differences.

[arnavsharma990]

I did a deeper review of the current VFS and filesystem implementations, including FAT, tmpfs, SPIFFS, littlefs, and NFS, to determine whether the lifetime management pattern from #20101 warrants a common abstraction.

My current conclusion is that a generic mechanism does not appear justified at this point.

The important distinction is that although several filesystems use some combination of reference counting and deferred deletion, their ownership and identity models are different:

• FAT

  • Uses a canonical fat_shared_s object keyed by directory sector/index.
  • An open file holds a reference to the shared object.
  • Unlink marks the object as pending while allowing the directory entry to be reused.
  • The FAT cluster chain is reclaimed when the final reference is released.
  • The implementation also requires FAT-specific handling for rename/re-keying, cursor invalidation, truncation, and cluster management.

• tmpfs

  • Uses a heap object whose identity is already represented by the object pointer.
  • Open instances can share the same object and maintain references.
  • Unlink marks the object as UNLINKED and removes its directory reference.
  • Existing final-release logic already defers reclamation until the last reference.
  • It does not require FAT-style keying, re-keying, or cursor invalidation.

• SPIFFS

  • Uses per-open file objects, with dup() sharing the existing object.
  • Supports an UNLINKED state and deferred cleanup.
  • However, independent open() calls do not canonicalize to the same shared object.
  • Its ownership model therefore differs from FAT.

• littlefs

  • Uses a per-open lfs_file with reference handling for duplicated descriptors.
  • Unlink is delegated to the littlefs core.
  • Its copy-on-write semantics handle the underlying lifetime requirements.
  • It does not require the same shared-object mechanism as FAT.

• NFS

  • Maintains per-open client/server handle state.
  • Reference counting primarily handles duplicated descriptors.
  • Removal is performed through an immediate REMOVE RPC.
  • The lifetime issue here is fundamentally different because it involves distributed/server-side coherence rather than local filesystem object reclamation.

The existing VFS struct file / f_refs handling already provides the required descriptor-level sharing for dup(), dup2(), and fcntl() duplication. The remaining lifetime semantics are therefore filesystem-specific.

Based on this investigation, I don't currently see a second filesystem with sufficiently similar ownership and identity semantics to justify a reusable abstraction.

A generic helper would either become tightly coupled to FAT's keyed/re-keyable model, or be reduced to a generic refcount + pending helper whose savings appear too small to justify the additional API, locking, configuration, and maintenance overhead.

I don't think an implementation PR for #20120 would provide enough benefit at this stage.

If a future filesystem encounters the same canonical shared object + unlink-pending + final-reference reclamation requirement, that would provide a much stronger basis for revisiting a common mechanism.

Thanks!