Repository navigation
Improve cache not found state in actors - #1913
Merged
Merged
Conversation
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
… the first time Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1913 +/- ##
==========================================
+ Coverage 64.13% 64.19% +0.05%
==========================================
Files 358 358
Lines 10496 10508 +12
Branches 1250 1258 +8
==========================================
+ Hits 6732 6746 +14
+ Misses 3429 3426 -3
- Partials 335 336 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…-found-state Signed-off-by: Whit Waldo <whit.waldo@innovian.net>
WhitWaldo
pushed a commit
that referenced
this pull request
Sep 24, 2026
…1916) SyncDefaultTracker (#1912) only refreshes default-tracker entries whose ChangeKind is None. #1913 introduced a second clean entry kind, NotFound, which it skips. If a reminder/timer caches "not found" for a key and a reentrancy-scoped call then creates that key, the default tracker keeps reporting the key as absent until the actor deactivates - the same staleness #1912 fixed, now on the absent -> present transition. Treat NotFound like None: a reentrant write refreshes the entry in place with the persisted value, a reentrant remove evicts it. Fixes #1915 Signed-off-by: Oliver Tomlinson <oliverjamestomlinson@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2 of 3 tasks
This was referenced Sep 27, 2026
This was referenced Oct 4, 2026
2 of 3 tasks
This was referenced Oct 8, 2026
dapr-bot
pushed a commit
to magic-peach/python-sdk
that referenced
this pull request
Oct 12, 2026
* Actors: invalidate the default state tracker after a reentrant save With reentrancy enabled, each dispatched method call gets its own state change tracker, but activation, reminders and timers run on the default tracker because no reentrancy id reaches them. A key read during activation stays cached there with change kind none forever, while method calls write the same key through their own trackers. A reminder callback that later reads that key is served the stale activation value. An app that skips its write because the value looks unchanged loses that write silently: nothing is logged anywhere, because no write is ever issued. Drop the default tracker's clean copies of keys written through a reentrancy-scoped tracker, so the next read reloads them from the runtime. Reported in dapr/dapr#10532, where a reminder callback's read-modify-write of an actor state key never persisted while the identical write from an ordinary method call did, and only with reentrancy enabled. Should be backported. Signed-off-by: joshvanl <me@joshvanl.dev> * fix(actor): refresh the default state tracker in place after a reentrant save Instead of dropping the default tracker's clean copy of a key that a reentrant call saved, replace it with the saved value and ttl so the next read from activation, a reminder or a timer is served from cache rather than costing an extra state store read. Removed keys are still dropped and entries with pending changes are left alone. The cached value is passed through the state serializer first, so it has the same shape a fresh read would return (for example a tuple comes back as a list). This mirrors dapr/dotnet-sdk#1912. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): drop the default entry when a reentrant refresh cannot match a fresh read Two cases fall back to dapr#1227's eviction instead of an in-place refresh: a saved None value, which the state provider leaves out of the write, and a state serializer that fails to decode the value after the save has already committed. The save no longer raises after a successful write. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(actor): cache state that is not found in the default state tracker Reading a key that does not exist went back to the state store every time, because a miss was not cached. try_get_state now records a miss as a clean "not found" entry in the default tracker, so later reads, contains_state, try_remove_state and remove_state answer without I/O, and try_add_state, set_state, set_state_ttl, get_or_add_state and add_or_update_state turn it into an add without asking the store again. get_state_names skips it and save_state never sends it. Misses are not cached in reentrant trackers. Nothing refreshes an outer reentrant call's tracker when a nested reentrant call (A -> B -> A) saves, so a cached miss there would hide the nested write and then overwrite it through get_or_add_state or try_add_state. The entry is a private StateMetadata subclass that reports the existing "none" change kind, so the public StateChangeKind enum, ActorStateChange and subclasses of ActorStateManager are unchanged. A reentrant save refreshes a "not found" default entry the same way as a clean one: a write replaces it with the saved value and a remove evicts it. Without that, a reminder that cached a miss would keep seeing the key as absent after a reentrant call created it. Ports dapr/dotnet-sdk#1913 and dapr/dotnet-sdk#1916. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(actor): run the nested reentrant test on Python 3.10 create_task(context=...) only exists from Python 3.11. A task already runs in a copy of the caller's context, so the argument isn't needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> * test(actor): read the empty payload through a fresh state manager The cached miss served the second read from the tracker, so the empty payload never reached the state provider. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> * fix(actor): log when a reentrant save cannot refresh cached state Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-authored-by: joshvanl <me@joshvanl.dev> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
github-merge-queue Bot
pushed a commit
to dapr/python-sdk
that referenced
this pull request
Oct 13, 2026
* Actors: invalidate the default state tracker after a reentrant save With reentrancy enabled, each dispatched method call gets its own state change tracker, but activation, reminders and timers run on the default tracker because no reentrancy id reaches them. A key read during activation stays cached there with change kind none forever, while method calls write the same key through their own trackers. A reminder callback that later reads that key is served the stale activation value. An app that skips its write because the value looks unchanged loses that write silently: nothing is logged anywhere, because no write is ever issued. Drop the default tracker's clean copies of keys written through a reentrancy-scoped tracker, so the next read reloads them from the runtime. Reported in dapr/dapr#10532, where a reminder callback's read-modify-write of an actor state key never persisted while the identical write from an ordinary method call did, and only with reentrancy enabled. Should be backported. Signed-off-by: joshvanl <me@joshvanl.dev> * fix(actor): refresh the default state tracker in place after a reentrant save Instead of dropping the default tracker's clean copy of a key that a reentrant call saved, replace it with the saved value and ttl so the next read from activation, a reminder or a timer is served from cache rather than costing an extra state store read. Removed keys are still dropped and entries with pending changes are left alone. The cached value is passed through the state serializer first, so it has the same shape a fresh read would return (for example a tuple comes back as a list). This mirrors dapr/dotnet-sdk#1912. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): drop the default entry when a reentrant refresh cannot match a fresh read Two cases fall back to #1227's eviction instead of an in-place refresh: a saved None value, which the state provider leaves out of the write, and a state serializer that fails to decode the value after the save has already committed. The save no longer raises after a successful write. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(actor): cache state that is not found in the default state tracker Reading a key that does not exist went back to the state store every time, because a miss was not cached. try_get_state now records a miss as a clean "not found" entry in the default tracker, so later reads, contains_state, try_remove_state and remove_state answer without I/O, and try_add_state, set_state, set_state_ttl, get_or_add_state and add_or_update_state turn it into an add without asking the store again. get_state_names skips it and save_state never sends it. Misses are not cached in reentrant trackers. Nothing refreshes an outer reentrant call's tracker when a nested reentrant call (A -> B -> A) saves, so a cached miss there would hide the nested write and then overwrite it through get_or_add_state or try_add_state. The entry is a private StateMetadata subclass that reports the existing "none" change kind, so the public StateChangeKind enum, ActorStateChange and subclasses of ActorStateManager are unchanged. A reentrant save refreshes a "not found" default entry the same way as a clean one: a write replaces it with the saved value and a remove evicts it. Without that, a reminder that cached a miss would keep seeing the key as absent after a reentrant call created it. Ports dapr/dotnet-sdk#1913 and dapr/dotnet-sdk#1916. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(actor): run the nested reentrant test on Python 3.10 create_task(context=...) only exists from Python 3.11. A task already runs in a copy of the caller's context, so the argument isn't needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> * perf(actor): skip the state store read when setting an untracked key set_state and set_state_ttl read the state store for a key that was not in the tracker only to choose between add and update, but the state provider saves both as the same upsert. The key is now recorded as an update without that read. Update rather than add is deliberate: the key may already exist in the store, so a remove later in the same turn has to send a delete instead of dropping the pending entry. Deleting a key that turns out not to exist is harmless; skipping the delete of one that does is not. Keys the manager knows are absent keep the add kind, so add-then-remove still sends nothing: try_add_state still reads the store and fails when the key exists, and a cached "not found" entry still turns into an add. try_remove_state still reads the store to report whether anything was removed. A reentrant save refreshes the default tracker the same way whatever kind the saved change had. Ports dapr/dotnet-sdk#1914. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): log the cause when a reentrant refresh evicts a state entry Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-authored-by: joshvanl <me@joshvanl.dev> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
dapr-bot
added a commit
to dapr/python-sdk
that referenced
this pull request
Oct 16, 2026
… (#1303) * Refresh the default actor state tracker in place after a reentrant save (#1237) * Actors: invalidate the default state tracker after a reentrant save With reentrancy enabled, each dispatched method call gets its own state change tracker, but activation, reminders and timers run on the default tracker because no reentrancy id reaches them. A key read during activation stays cached there with change kind none forever, while method calls write the same key through their own trackers. A reminder callback that later reads that key is served the stale activation value. An app that skips its write because the value looks unchanged loses that write silently: nothing is logged anywhere, because no write is ever issued. Drop the default tracker's clean copies of keys written through a reentrancy-scoped tracker, so the next read reloads them from the runtime. Reported in dapr/dapr#10532, where a reminder callback's read-modify-write of an actor state key never persisted while the identical write from an ordinary method call did, and only with reentrancy enabled. Should be backported. Signed-off-by: joshvanl <me@joshvanl.dev> * fix(actor): refresh the default state tracker in place after a reentrant save Instead of dropping the default tracker's clean copy of a key that a reentrant call saved, replace it with the saved value and ttl so the next read from activation, a reminder or a timer is served from cache rather than costing an extra state store read. Removed keys are still dropped and entries with pending changes are left alone. The cached value is passed through the state serializer first, so it has the same shape a fresh read would return (for example a tuple comes back as a list). This mirrors dapr/dotnet-sdk#1912. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): drop the default entry when a reentrant refresh cannot match a fresh read Two cases fall back to #1227's eviction instead of an in-place refresh: a saved None value, which the state provider leaves out of the write, and a state serializer that fails to decode the value after the save has already committed. The save no longer raises after a successful write. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): evict TTL'd keys from the default tracker and log refresh failures Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-authored-by: joshvanl <me@joshvanl.dev> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit fb2cd88) Signed-off-by: Samantha Coyle <sam@diagrid.io> * Remember missing actor state keys instead of re-reading them (#1238) * Actors: invalidate the default state tracker after a reentrant save With reentrancy enabled, each dispatched method call gets its own state change tracker, but activation, reminders and timers run on the default tracker because no reentrancy id reaches them. A key read during activation stays cached there with change kind none forever, while method calls write the same key through their own trackers. A reminder callback that later reads that key is served the stale activation value. An app that skips its write because the value looks unchanged loses that write silently: nothing is logged anywhere, because no write is ever issued. Drop the default tracker's clean copies of keys written through a reentrancy-scoped tracker, so the next read reloads them from the runtime. Reported in dapr/dapr#10532, where a reminder callback's read-modify-write of an actor state key never persisted while the identical write from an ordinary method call did, and only with reentrancy enabled. Should be backported. Signed-off-by: joshvanl <me@joshvanl.dev> * fix(actor): refresh the default state tracker in place after a reentrant save Instead of dropping the default tracker's clean copy of a key that a reentrant call saved, replace it with the saved value and ttl so the next read from activation, a reminder or a timer is served from cache rather than costing an extra state store read. Removed keys are still dropped and entries with pending changes are left alone. The cached value is passed through the state serializer first, so it has the same shape a fresh read would return (for example a tuple comes back as a list). This mirrors dapr/dotnet-sdk#1912. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): drop the default entry when a reentrant refresh cannot match a fresh read Two cases fall back to #1227's eviction instead of an in-place refresh: a saved None value, which the state provider leaves out of the write, and a state serializer that fails to decode the value after the save has already committed. The save no longer raises after a successful write. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(actor): cache state that is not found in the default state tracker Reading a key that does not exist went back to the state store every time, because a miss was not cached. try_get_state now records a miss as a clean "not found" entry in the default tracker, so later reads, contains_state, try_remove_state and remove_state answer without I/O, and try_add_state, set_state, set_state_ttl, get_or_add_state and add_or_update_state turn it into an add without asking the store again. get_state_names skips it and save_state never sends it. Misses are not cached in reentrant trackers. Nothing refreshes an outer reentrant call's tracker when a nested reentrant call (A -> B -> A) saves, so a cached miss there would hide the nested write and then overwrite it through get_or_add_state or try_add_state. The entry is a private StateMetadata subclass that reports the existing "none" change kind, so the public StateChangeKind enum, ActorStateChange and subclasses of ActorStateManager are unchanged. A reentrant save refreshes a "not found" default entry the same way as a clean one: a write replaces it with the saved value and a remove evicts it. Without that, a reminder that cached a miss would keep seeing the key as absent after a reentrant call created it. Ports dapr/dotnet-sdk#1913 and dapr/dotnet-sdk#1916. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(actor): run the nested reentrant test on Python 3.10 create_task(context=...) only exists from Python 3.11. A task already runs in a copy of the caller's context, so the argument isn't needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> * test(actor): read the empty payload through a fresh state manager The cached miss served the second read from the tracker, so the empty payload never reached the state provider. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> * fix(actor): log when a reentrant save cannot refresh cached state Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-authored-by: joshvanl <me@joshvanl.dev> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit dbf9867) Signed-off-by: Samantha Coyle <sam@diagrid.io> * Skip the state store existence check when setting actor state (#1239) * Actors: invalidate the default state tracker after a reentrant save With reentrancy enabled, each dispatched method call gets its own state change tracker, but activation, reminders and timers run on the default tracker because no reentrancy id reaches them. A key read during activation stays cached there with change kind none forever, while method calls write the same key through their own trackers. A reminder callback that later reads that key is served the stale activation value. An app that skips its write because the value looks unchanged loses that write silently: nothing is logged anywhere, because no write is ever issued. Drop the default tracker's clean copies of keys written through a reentrancy-scoped tracker, so the next read reloads them from the runtime. Reported in dapr/dapr#10532, where a reminder callback's read-modify-write of an actor state key never persisted while the identical write from an ordinary method call did, and only with reentrancy enabled. Should be backported. Signed-off-by: joshvanl <me@joshvanl.dev> * fix(actor): refresh the default state tracker in place after a reentrant save Instead of dropping the default tracker's clean copy of a key that a reentrant call saved, replace it with the saved value and ttl so the next read from activation, a reminder or a timer is served from cache rather than costing an extra state store read. Removed keys are still dropped and entries with pending changes are left alone. The cached value is passed through the state serializer first, so it has the same shape a fresh read would return (for example a tuple comes back as a list). This mirrors dapr/dotnet-sdk#1912. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): drop the default entry when a reentrant refresh cannot match a fresh read Two cases fall back to #1227's eviction instead of an in-place refresh: a saved None value, which the state provider leaves out of the write, and a state serializer that fails to decode the value after the save has already committed. The save no longer raises after a successful write. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(actor): cache state that is not found in the default state tracker Reading a key that does not exist went back to the state store every time, because a miss was not cached. try_get_state now records a miss as a clean "not found" entry in the default tracker, so later reads, contains_state, try_remove_state and remove_state answer without I/O, and try_add_state, set_state, set_state_ttl, get_or_add_state and add_or_update_state turn it into an add without asking the store again. get_state_names skips it and save_state never sends it. Misses are not cached in reentrant trackers. Nothing refreshes an outer reentrant call's tracker when a nested reentrant call (A -> B -> A) saves, so a cached miss there would hide the nested write and then overwrite it through get_or_add_state or try_add_state. The entry is a private StateMetadata subclass that reports the existing "none" change kind, so the public StateChangeKind enum, ActorStateChange and subclasses of ActorStateManager are unchanged. A reentrant save refreshes a "not found" default entry the same way as a clean one: a write replaces it with the saved value and a remove evicts it. Without that, a reminder that cached a miss would keep seeing the key as absent after a reentrant call created it. Ports dapr/dotnet-sdk#1913 and dapr/dotnet-sdk#1916. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(actor): run the nested reentrant test on Python 3.10 create_task(context=...) only exists from Python 3.11. A task already runs in a copy of the caller's context, so the argument isn't needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> * perf(actor): skip the state store read when setting an untracked key set_state and set_state_ttl read the state store for a key that was not in the tracker only to choose between add and update, but the state provider saves both as the same upsert. The key is now recorded as an update without that read. Update rather than add is deliberate: the key may already exist in the store, so a remove later in the same turn has to send a delete instead of dropping the pending entry. Deleting a key that turns out not to exist is harmless; skipping the delete of one that does is not. Keys the manager knows are absent keep the add kind, so add-then-remove still sends nothing: try_add_state still reads the store and fails when the key exists, and a cached "not found" entry still turns into an add. try_remove_state still reads the store to report whether anything was removed. A reentrant save refreshes the default tracker the same way whatever kind the saved change had. Ports dapr/dotnet-sdk#1914. Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(actor): log the cause when a reentrant refresh evicts a state entry Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Casper Nielsen <casper@diagrid.io> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Casper Nielsen <casper@diagrid.io> Co-authored-by: joshvanl <me@joshvanl.dev> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit fceda09) Signed-off-by: Samantha Coyle <sam@diagrid.io> --------- Signed-off-by: joshvanl <me@joshvanl.dev> Signed-off-by: Casper Nielsen <casper@diagrid.io> Signed-off-by: Samantha Coyle <sam@diagrid.io> Co-authored-by: Casper Nielsen <casper@diagrid.io> Co-authored-by: joshvanl <me@joshvanl.dev> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Dapr Bot <56698301+dapr-bot@users.noreply.github.com>
This was referenced Oct 18, 2026
This was referenced Oct 25, 2026
This was referenced Nov 1, 2026
This was referenced Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Addresses issue @olitomlinson raised in #1909. Now, when the call is initially made for a key in the actor's state and that request is made to the runtime, this now captures the
HttpStatusCodeshared in the response. Based on that, if the status code reflects a200 OK, the value is stored as usual without any add/remove change values. But if there's any other status code provided, it means the retrieval was not successful and it will instead store a value with aNotFoundstate change value. Should another request come in for the same key, it will immediately return a null value when this is in the cache, but should it be added or updated from the SDK, such changes will override this state and work as expected.I added a wall of unit tests to validate that I didn't break anything.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #1909
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: