Skip to content

Make client a start/stop service - #272

Merged
brandur merged 1 commit into
masterfrom
brandur-client-start-stop-2
Mar 17, 2024
Merged

Make client a start/stop service#272
brandur merged 1 commit into
masterfrom
brandur-client-start-stop-2

Conversation

@brandur

@brandur brandur commented Mar 16, 2024

Copy link
Copy Markdown
Contributor

Here, continue service refactoring to its ultimate conclusion, and make
the client itself a start/stop service, having a couple advantages:

  • (IMO) considerably simplifies the start and stop code, putting it all
    in one place, and largely even one function. Fewer helpers to follow
    around to understand what's going on.

  • Makes the client behave gracefully on double starts/stops.

  • Allows the client to easily handle start/stop under duress. Any number
    of goroutines can be starting or stopping it simultaneously and it's
    guaranteed free from races.

Because the client's stop functions are different from the signature of
other start/stop services (taking a context and returning an error), I
had to modify the base start/stop service somewhat and allow for an
additional StopInit helper (like StartInit) that lets stop behavior
be customized while still providing race-free operations.

@brandur
brandur force-pushed the brandur-client-start-stop-2 branch from fb3c6c6 to 7696b6e Compare March 16, 2024 23:09
@brandur

brandur commented Mar 16, 2024

Copy link
Copy Markdown
Contributor Author

This one a lot more readable without whitespace changes:

https://github.com/riverqueue/river/pull/272/files?w=1

Comment thread client_test.go
err := client.Stop(ctx)
require.Error(t, err)
require.Equal(t, "client not started", err.Error())
})

@brandur brandur Mar 16, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally left this error in, but to guarantee non-raciness, I would've had to an extra mutex to protect it, and it didn't seem worth it since the error's not particularly valuable. (The error wasn't race protected before either, but now we're actually testing it so the problem was revealed.)

@brandur
brandur requested a review from bgentry March 17, 2024 00:05
Comment thread client.go
Comment on lines +678 to +679
go func() {
defer close(stopped)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be more readable and easily debuggable if some of these closures were split into internal client methods. Thinking in particular of the potential for a trace to have anonymous goroutines rather than named methods.

Not a huge concern but it might be worth trying to shuffle some of this around to improve the readability of Start()? Particularly here and in the large startup closure above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a look at this, and not against it, but I think I'm going to punt for the time being. I kind of like having all the logic Start (and the majority of the stop) logic in one place because you can scan it all at once without having to jump around. Also lets us reduce the number of struct variables and helpers in favor of definitions that can't leak into other contexts, and are defined more visibly, closer to home.

Comment thread client_test.go Outdated
Comment on lines +2577 to +2578
// ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
// t.Cleanup(cancel)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mistakenly commented?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops yep. Actually we should talk about this, but the reason I had it commented is that in case your program is actually stuck, the canceled context supersedes go test's -timeout option, so instead of -timeout stopping the test of a full stack trace of every goroutine that was still running, you end up with a canceled test with no debugging information.

So a long way of saying that tighter -timeout parameters are a better option than context timeouts when it comes to use in tests, but in the interest of not getting into that now, I just uncommented these lines.

}

// StopInit provides a way to build a more customized Stop implementation. It
// should be avoided unless there'a an exception reason not to because Stop

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// should be avoided unless there'a an exception reason not to because Stop
// should be avoided unless there's an exceptional reason not to because Stop

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks.

@brandur
brandur force-pushed the brandur-fast-completer branch 3 times, most recently from e5d4268 to 71abf47 Compare March 17, 2024 20:22
Base automatically changed from brandur-fast-completer to master March 17, 2024 20:25
@brandur
brandur force-pushed the brandur-client-start-stop-2 branch 3 times, most recently from 4091846 to 54fed4c Compare March 17, 2024 20:34
Here, continue service refactoring to its ultimate conclusion, and make
the client itself a start/stop service, having a couple advantages:

* (IMO) considerably simplifies the start and stop code, putting it all
  in one place, and largely even one function. Fewer helpers to follow
  around to understand what's going on.

* Makes the client behave gracefully on double starts/stops.

* Allows the client to easily handle start/stop under duress. Any number
  of goroutines can be starting or stopping it simultaneously and it's
  guaranteed free from races.

Because the client's stop functions are different from the signature of
other start/stop services (taking a context and returning an error), I
had to modify the base start/stop service somewhat and allow for an
additional `StopInit` helper (like `StartInit`) that lets stop behavior
be customized while still providing race-free operations.
@brandur
brandur force-pushed the brandur-client-start-stop-2 branch from 54fed4c to bed599c Compare March 17, 2024 20:37
@brandur

brandur commented Mar 17, 2024

Copy link
Copy Markdown
Contributor Author

ty.

@brandur
brandur merged commit a939d34 into master Mar 17, 2024
@brandur
brandur deleted the brandur-client-start-stop-2 branch March 17, 2024 20:44
bgentry added a commit that referenced this pull request May 30, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request May 30, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request May 30, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request May 31, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 3, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 4, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 9, 2024
This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.
bgentry added a commit that referenced this pull request Jun 9, 2024
* fix StopAndTest after ongoing Stop

This regression was introduced in #272, which I noticed because our demo
app is not currently cancelling active jobs gracefully before exiting.

This commit fixes the behavior by always cancelling the work context
upon `StopAndCancel()`, regardless of whether or not another shutdown is
in progress.

* refactor StopAndCancel tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants