[core][move semantics][4/n] Consumer raylet records which pulls are moves by karticam · Pull Request #67041 · ray-project/ray

karticam · 2026-10-15T01:48:44Z

This is PR 4 of a series of PRs to implement move semantics for plasma objects in core.

The complete design doc is here: https://docs.google.com/document/d/1ZJhgRmwVyKfNCR21CWf6xMcc0B6HJwSPQsfebTRvnnE/edit?tab=t.0#heading=h.7wcod1f9afpg

After #66803, an actor task's spec says which of its by-ref args it consumes (TaskArg.is_move). This PR carries that flag to the raylet of the node where the actor runs, and makes that raylet remember, for each such arg it has to pull, that the pull is a move. Nothing reads the record yet: the next PR uses it on the receive path to pin the pulled copy as the object's new primary copy and report that to the owner.

Changes

TaskSpecification::GetDependencyIsMove(). One bool per entry of GetDependencies(), in the same order.
The flag travels with the arg wait. BeginArgsFetch and the AsyncWaitForArgs callback take the vector; RayletIpcClient::WaitForActorCallArgs sends it in a new is_move: [bool] field of WaitForActorCallArgsRequest, parallel to object_ids.
The raylet records it when it starts a pull. ProcessWaitForActorCallArgs → AsyncWait → LeaseDependencyManager::StartOrUpdateWaitRequest(worker, refs, is_move). In the branch that handles a non-local arg, right before the pull, a moved arg is recorded with ObjectManager::MarkMovePull. If the object is not local but a pull for it already exists (a ray.get in flight, or another worker's ray.wait), the arg is still recorded and no second pull is started. An arg that is already local is neither pulled nor recorded; a later PR handles that case. StartOrUpdateWaitRequest is also used for an ordinary ray.wait; that caller passes an empty is_move vector, so a plain ray.wait never records anything.
ObjectManager keeps the record. It is a set of object IDs. The set is written on the main thread (MarkMovePull adds, CancelPull removes) and will be read on the RPC threads (IsMovePull, from ReceiveObjectChunk, once per chunk, in the next PR), so it is guarded by a mutex; reads take a shared lock.

An object is removed from the set when its last pull ends, not its first (there could be multiple pulls for the same object). PullManager already tracks which pulls still need an object and tells CancelPull which objects nobody needs any more; the record is erased only for those. So if two pulls want the same object (a ray.get in flight plus the actor's wait, or c.remote(r, r)), ending one of them keeps the record.

Tests

task_spec_test: GetDependencyIsMove on the TestArgIsMove fixture.
lease_dependency_manager_test: a moved, a plain and a local moved arg (one record, two pulls); a plain ray.wait records nothing; a moved arg whose pull already exists from a ray.get or from another worker's wait; the same moved arg passed twice starts one pull.
object_manager_test: the record is idempotent, survives cancelling one of two pulls, and is dropped with the last one.
python/ray/tests/test_consume_once.py (32 passed), and a two-node script covering a cross-node consume with a plain arg, c.remote(r, r), a same-node consume, an owner ray.wait(fetch_local=True) before the consume, and a plain ray.wait inside an actor.

gemini-code-assist · 2026-10-15T01:48:49Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

gemini-code-assist · 2026-10-15T04:44:23Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

cursor

Stale Bugbot comment from a previous run.

cursor · 2026-10-15T04:46:46Z

+
+  /// Safe to call from any thread.
+  bool IsMovePull(const ObjectID &object_id) const override
+      ABSL_LOCKS_EXCLUDED(move_pull_mutex_);


Missing Doxygen on new APIs

Low Severity

New and updated C++ declarations such as MarkMovePull, IsMovePull, and GetDependencyIsMove do not use Doxygen /** ... */ comments with @brief and @param/@return. That violates the Doxygen documentation rule for C++ functions and classes.

Additional Locations (2)

src/ray/common/task/task_spec.h#L288-L291

src/ray/object_manager/object_manager_interface.h#L34-L40

^{Triggered by project rule: Bugbot Rules}

^{Reviewed by Cursor Bugbot for commit 69d2a94. Configure here.}

Sparks0219 · 2026-10-30T23:48:36Z

  ASSERT_FALSE(task_spec.ArgIsMove(0));
  ASSERT_TRUE(task_spec.ArgIsMove(1));
  ASSERT_FALSE(task_spec.ArgIsMove(2));
+  // One entry per by-ref arg, in GetDependencies() order.


don't need this comment

Sparks0219 · 2026-10-30T23:53:58Z

  /// thread, so the IPC is not blocked behind in-progress work on the task
  /// execution service).
+  ///
+  /// \param is_move Parallel to `args`: whether the task consumes the arg


can you plz add the other fields to make it look nicer thanks

Sparks0219 · 2026-10-30T23:54:54Z

  // Sent back to the core worker to notify the wait has completed.
  task_id: string;
  attempt_number: int;
+  // Parallel to object_ids: true if the task consumes the object (TaskArg.is_move).


get this out of here

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 10b5bda. Configure here.}

cursor · 2026-11-03T03:12:02Z


+  /// Return, for each dependency in GetDependencies() order, whether this task
+  /// consumes it (TaskArg.is_move).
+  std::vector<bool> GetDependencyIsMove() const;


Missing Doxygen on new C++ APIs

Low Severity

New and modified C++ declarations are missing Doxygen /** ... */ comments with @brief and @param/@return. GetDependencyIsMove has no @return, MarkMovePull/IsMovePull on ObjectManager have no API comment, and BeginArgsFetch documents only is_move.

Additional Locations (2)

src/ray/object_manager/object_manager.h#L203-L209

src/ray/core_worker/task_execution/common.h#L131-L136

^{Triggered by project rule: Bugbot Rules}

^{Reviewed by Cursor Bugbot for commit 10b5bda. Configure here.}

Signed-off-by: kartica.modi <kartica.modi@anyscale.com>

karticam assigned Sparks0219 Oct 15, 2026

karticam added core Issues that should be addressed in Ray Core go add ONLY when ready to merge, run all tests labels Oct 15, 2026

karticam marked this pull request as ready for review October 15, 2026 04:44

karticam requested a review from a team as a code owner October 15, 2026 04:44

cursor Bot reviewed Oct 15, 2026

View reviewed changes

karticam added this pull request to stack #66768 October 15, 2026 04:47

karticam mentioned this pull request Oct 15, 2026

[core][move semantics][5/n] Raylet pins a pulled consume_once object as the primary copy and reports it to the owner #67093

Open

karticam added the unstale A PR that has been marked unstale. It will not get marked stale again if this label is on it. label Oct 16, 2026

karticam mentioned this pull request Oct 18, 2026

[core][move semantics][6/n] Edge cases with ray.get() and consumer pull #67133

Open

karticam force-pushed the karticam/consumer-learns-is-move branch from 69d2a94 to 3303ac6 Compare October 21, 2026 05:14

Sparks0219 reviewed Oct 30, 2026

View reviewed changes

karticam force-pushed the karticam/consumer-learns-is-move branch from 3303ac6 to 85674f6 Compare November 3, 2026 03:00

karticam force-pushed the karticam/consumer-learns-is-move branch from 85674f6 to 10b5bda Compare November 3, 2026 03:07

cursor Bot reviewed Nov 3, 2026

View reviewed changes

Consumer maintains a set of objects that have to be pulled by move

ab45efc

Signed-off-by: kartica.modi <kartica.modi@anyscale.com>

karticam force-pushed the karticam/consumer-learns-is-move branch from 10b5bda to ab45efc Compare November 14, 2026 02:36

Navigation Menu

Repository navigation

[core][move semantics][4/n] Consumer raylet records which pulls are moves - #67041

[core][move semantics][4/n] Consumer raylet records which pulls are moves#67041
karticam wants to merge 1 commit into
karticam/is-move-in-task-argsfrom
karticam/consumer-learns-is-move

karticam commented Oct 15, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented Oct 15, 2026

Uh oh!

gemini-code-assist Bot commented Oct 15, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot Oct 15, 2026

Uh oh!

Sparks0219 Oct 30, 2026

Uh oh!

Sparks0219 Oct 30, 2026

Uh oh!

Sparks0219 Oct 30, 2026

Uh oh!

Sparks0219 Oct 30, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Nov 3, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

karticam commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Uh oh!

gemini-code-assist Bot commented Oct 15, 2026

Uh oh!

gemini-code-assist Bot commented Oct 15, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Oct 15, 2026

Choose a reason for hiding this comment

Missing Doxygen on new APIs

Uh oh!

Sparks0219 Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

Sparks0219 Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

Sparks0219 Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

Sparks0219 Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Nov 3, 2026

Choose a reason for hiding this comment

Missing Doxygen on new C++ APIs

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

karticam commented Oct 15, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading