Add a method to the mpsc::Receiver for producing a non-blocking iterator by mitchmindtree · Pull Request #34724 · rust-lang/rust

mitchmindtree · 2016-07-08T14:22:21Z

Currently, the mpsc::Receiver offers methods for receiving values in both blocking (recv) and non-blocking (try_recv) flavours. However only blocking iteration over values is supported. This PR adds a non-blocking iterator to complement the try_recv method, just as the blocking iterator complements the recv method.

Use-case

I predominantly use rust in my work on real-time systems and in particular real-time audio generation/processing. I use mpsc::channels to communicate between threads in a purely non-blocking manner. I.e. I might send messages from the GUI thread to the audio thread to update the state of the dsp-graph, or from the audio thread to the GUI thread to display the RMS of each node. These are just a couple examples (I'm probably using 30+ channels across my various projects). I almost exclusively use the mpsc::Receiver::try_recv method to avoid blocking any of the real-time threads and causing unwanted glitching/stuttering. Now that I mention it, I can't think of a single time that I personally have used the recv method (though I can of course see why it would be useful, and perhaps the common case for many people).

As a result of this experience, I can't help but feel there is a large hole in the Receiver API.

blocking	non-blocking
`recv`	`try_recv`
`iter`	🙀

For the most part, I've been working around this using while let Ok(v) = r.try_recv() { ... }, however as nice as this is, it is clearly no match for the Iterator API.

As an example, in the majority of my channel use cases I only want to check for n number of messages before breaking from the loop so that I don't miss the audio IO callback or hog the GUI thread for too long when an unexpectedly large number of messages are sent. Currently, I have to write something like this:

let mut take = 100;
while let Ok(msg) = rx.try_recv() {
    // Do stuff with msg
    if take == 0 {
        break;
    }
    take -= 1;
}

or wrap the try_recv call in a Range<usize>/FilterMap iterator combo.

On the other hand, this PR would allow for the following:

for msg in rx.try_iter().take(100) {
    // Do stuff with msg
}

I imagine this might also be useful to game devs, embedded or anyone doing message passing across real-time threads.

Currently, the `mpsc::Receiver` offers methods for receiving values in both blocking (`recv`) and non-blocking (`try_recv`) flavours. However only blocking iteration over values is supported. This commit adds a non-blocking iterator to complement the `try_recv` method, just as the blocking iterator complements the `recv` method.

rust-highfive · 2016-07-08T14:22:36Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

alexcrichton · 2016-07-09T01:35:52Z

Thanks for the PR! I was pretty skeptical at first glance but I think your description has won me over. My only heistation is the fact that we're hiding whether iteration was terminated due to "would block" or "disconnected", but perhaps that's not so bad in the long run? If a channel is disconnected then any attempt to receive on it again will again receive a disconnection

cc @rust-lang/libs

mitchmindtree · 2016-07-09T03:00:13Z

@alexcrichton thanks for taking a look!

My only heistation is the fact that we're hiding whether iteration was terminated due to "would block" or "disconnected, but perhaps that's not so bad in the long run?"

I attempted to make sure that this behaviour is clear in the new iterator's docs, but perhaps I could also add a note that recommends using try_recv, a loop and a match expr (with an example?) in the case that the user wishes to loop while distinguishing between reasons for termination?

alexcrichton · 2016-07-10T06:27:02Z

@mitchmindtree ah yeah the docs are definitely quite clear in this regard, I was just hoping we could surface it at the API level as well (but I'm not sure that's possible here)

brson · 2016-07-11T18:48:21Z

Thanks for the good writeup. Looks fine to me.

aturon · 2016-07-13T03:16:16Z

I'm happy to land this API on an unstable basis.

alexcrichton · 2016-07-19T04:23:32Z

Discussed during libs triage the decision was to merge, thanks again for the PR @mitchmindtree!

@bors: r+

bors · 2016-07-19T04:23:34Z

📌 Commit b02b38e has been approved by alexcrichton

bors · 2016-07-19T19:38:21Z

⌛ Testing commit b02b38e with merge 376e0f9...

bors · 2016-07-19T19:44:12Z

💔 Test failed - auto-linux-64-opt

mitchmindtree · 2016-07-20T02:39:49Z

Ahh, I'm missing the issue field for the unstable decorators. Would you like me to create a tracking issue for this? Or would you prefer it if I use the issue number of this PR?

alexcrichton · 2016-07-20T02:59:34Z

Ah yeah that'd do it, if you want to open an issue I'll be sure to tag it appropriately and you can update this PR to reference it.

…ver_try_iter stability attributes

alexcrichton · 2016-07-20T05:18:44Z

@bors: r+ aed2e5c

Thanks!

bors · 2016-07-20T17:07:07Z

⌛ Testing commit aed2e5c with merge 72dae88...

bors · 2016-07-20T18:20:25Z

💔 Test failed - auto-win-msvc-64-opt-rustbuild

alexcrichton · 2016-07-20T20:21:40Z

Looks like a failure perhaps in the tests?

On Wed, Jul 20, 2016 at 11:20 AM, bors notifications@github.com wrote:

💔 Test failed - auto-win-msvc-64-opt-rustbuild
https://buildbot.rust-lang.org/builders/auto-win-msvc-64-opt-rustbuild/builds/1851

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#34724 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95PktU_SSJEZ7n8HE3ogQWzpYifnRks5qXmbsgaJpZM4JIE8s
.

… instead of break from the loop

mitchmindtree · 2016-07-21T09:35:42Z

Ahh sorry about that @alexcrichton, I just made the fix and successfully ran the tests locally - should be good to go now!

alexcrichton · 2016-07-21T16:14:12Z

@bors: r+

No worries!

bors · 2016-07-21T16:14:14Z

📌 Commit 05af033 has been approved by alexcrichton

bors · 2016-07-21T18:01:17Z

⌛ Testing commit 05af033 with merge dbc9edd...

bors · 2016-07-21T18:47:31Z

💔 Test failed - auto-win-gnu-32-opt

alexcrichton · 2016-07-21T20:28:32Z

@bors: retry

sorry for the number of retries...

On Thu, Jul 21, 2016 at 11:47 AM, bors notifications@github.com wrote:

💔 Test failed - auto-win-gnu-32-opt
https://buildbot.rust-lang.org/builders/auto-win-gnu-32-opt/builds/5014

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#34724 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95KqNmeLAbD4iy8AIOA105suRdxc0ks5qX77GgaJpZM4JIE8s
.

bors · 2016-07-22T05:39:49Z

⌛ Testing commit 05af033 with merge 0d75975...

…ichton Add a method to the mpsc::Receiver for producing a non-blocking iterator Currently, the `mpsc::Receiver` offers methods for receiving values in both blocking (`recv`) and non-blocking (`try_recv`) flavours. However only blocking iteration over values is supported. This PR adds a non-blocking iterator to complement the `try_recv` method, just as the blocking iterator complements the `recv` method. Use-case ------------- I predominantly use rust in my work on real-time systems and in particular real-time audio generation/processing. I use `mpsc::channel`s to communicate between threads in a purely non-blocking manner. I.e. I might send messages from the GUI thread to the audio thread to update the state of the dsp-graph, or from the audio thread to the GUI thread to display the RMS of each node. These are just a couple examples (I'm probably using 30+ channels across my various projects). I almost exclusively use the `mpsc::Receiver::try_recv` method to avoid blocking any of the real-time threads and causing unwanted glitching/stuttering. Now that I mention it, I can't think of a single time that I personally have used the `recv` method (though I can of course see why it would be useful, and perhaps the common case for many people). As a result of this experience, I can't help but feel there is a large hole in the `Receiver` API. | blocking | non-blocking | |------------|--------------------| | `recv` | `try_recv` | | `iter` | 🙀 | For the most part, I've been working around this using `while let Ok(v) = r.try_recv() { ... }`, however as nice as this is, it is clearly no match for the Iterator API. As an example, in the majority of my channel use cases I only want to check for *n* number of messages before breaking from the loop so that I don't miss the audio IO callback or hog the GUI thread for too long when an unexpectedly large number of messages are sent. Currently, I have to write something like this: ```rust let mut take = 100; while let Ok(msg) = rx.try_recv() { // Do stuff with msg if take == 0 { break; } take -= 1; } ``` or wrap the `try_recv` call in a `Range<usize>`/`FilterMap` iterator combo. On the other hand, this PR would allow for the following: ```rust for msg in rx.try_iter().take(100) { // Do stuff with msg } ``` I imagine this might also be useful to game devs, embedded or anyone doing message passing across real-time threads.

bors · 2016-07-22T08:36:20Z

mitchmindtree added 2 commits July 8, 2016 22:12

Add a test for Receiver::try_iter

b354887

rust-highfive assigned alexcrichton Jul 8, 2016

Add the unstable attribute to the new mpsc::Receiver::try_iter API

b02b38e

alexcrichton added the T-libs-api [DEPRECATED; DO NOT USE] label Jul 9, 2016

mitchmindtree mentioned this pull request Jul 20, 2016

Tracking issue for mpsc::Receiver::try_iter #34931

Closed

Add the missing tracking issue field for rust-lang#34931 to the recei…

aed2e5c

…ver_try_iter stability attributes

Fix issue in receiver_try_iter test where response sender would panic…

05af033

… instead of break from the loop

bors merged commit 05af033 into rust-lang:master Jul 22, 2016

Conversation

mitchmindtree commented Jul 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Use-case

Uh oh!

rust-highfive commented Jul 8, 2016

Uh oh!

alexcrichton commented Jul 9, 2016

Uh oh!

mitchmindtree commented Jul 9, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton commented Jul 10, 2016

Uh oh!

brson commented Jul 11, 2016

Uh oh!

aturon commented Jul 13, 2016

Uh oh!

alexcrichton commented Jul 19, 2016

Uh oh!

bors commented Jul 19, 2016

Uh oh!

bors commented Jul 19, 2016

Uh oh!

bors commented Jul 19, 2016

Uh oh!

mitchmindtree commented Jul 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alexcrichton commented Jul 20, 2016

Uh oh!

alexcrichton commented Jul 20, 2016

Uh oh!

bors commented Jul 20, 2016

Uh oh!

bors commented Jul 20, 2016

Uh oh!

alexcrichton commented Jul 20, 2016

Uh oh!

mitchmindtree commented Jul 21, 2016

Uh oh!

alexcrichton commented Jul 21, 2016

Uh oh!

bors commented Jul 21, 2016

Uh oh!

bors commented Jul 21, 2016

Uh oh!

bors commented Jul 21, 2016

Uh oh!

alexcrichton commented Jul 21, 2016

Uh oh!

bors commented Jul 22, 2016

Uh oh!

bors commented Jul 22, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mitchmindtree commented Jul 8, 2016 •

edited

Loading

mitchmindtree commented Jul 9, 2016 •

edited

Loading

mitchmindtree commented Jul 20, 2016 •

edited

Loading