fix(argo-cd): make ttlSecondsAfterFinished configurable for redisSecretInit - #4050
fix(argo-cd): make ttlSecondsAfterFinished configurable for redisSecretInit#4050duncan485 wants to merge 3 commits into
Conversation
|
Bumping from 60 to 600 is a serious change IMO. 1 minute to 10 minutes. Please change back to 60 as the default. People can bump it higher if needed after your change. Unless you can defend the 9 minute bump? Thank you for the contribution. |
|
🤖 Following up on my earlier comment after digging through the history, because the 60/600 question turns out to be more interesting than a simple default change. What I found:
So the key question: which Argo CD version are you running when you hit this? This chart's current appVersion (3.5.x) should contain the finalizer fix. If you're reproducing the race on 3.5.x, that's an upstream regression worth reporting on argoproj/argo-cd#6880 with your details, and a wider TTL default here is a reasonable stopgap. If you're on an older Argo CD, the default should stay Either way, two asks:
The configurability itself is welcome regardless of where the default lands. Thanks for tracing this into #2887; that issue has needed movement for a long time. |
| extraArgs: [] | ||
|
|
||
| # -- Seconds after which the completed Redis secret-init Job is automatically deleted | ||
| ttlSecondsAfterFinished: 600 |
There was a problem hiding this comment.
This bumps the default from 60 to 600 as well as making it configurable, so the render changes for everyone rather than only for people who set it. Shouldn't that be a minor rather than a patch? (versioning)
|
Hey @jmeridth I am actually on argo 3.3.9 (helm version 9.5.11). So I don't think that upsteam worked for me. I will edit my MR so it just makes it configurable, but still defaults to 60. Than at least we can try if this fixes the actual issue. If it does, we can change the default in a later MR? |
|
@duncan485 thanks for confirming the version — argo-cd 3.3.9 matters more than you might expect: the upstream finalizer fix for exactly this race (argoproj/gitops-engine#646) shipped in the 3.0 line, so reproducing the TTL race on 3.3.9 is evidence of a gap in that fix. When you have a moment, please post your reproduction details on argoproj/argo-cd#6880 (version, self-managed setup, the stuck-hook state); the maintainers there have been asking whether post-3.0 reports still exist. Your revised plan is exactly right for this PR: configurable |
f446161 to
0f7fd28
Compare
…ance documentation for hook-completion race condition Signed-off-by: duncan485 <bakkerduncan@gmail.com>
|
Hey @jmeridth Thanks, I updated the MR. As the default is now kept at 60, I think say a patch version should be the right bump. I will definitely post a to the original story on how to reproduce. But I am also verry curious to see if this solves it in my case. |
…etInit Signed-off-by: duncan485 <bakkerduncan@gmail.com>
Description
This PR makes
ttlSecondsAfterFinishedconfigurable viaredisSecretInit.ttlSecondsAfterFinished(default bumped from60to600seconds), giving ArgoCD's controller much more time to observe hook completion before the Job is cleaned up.Note on DCO:
If the DCO action in the integration test fails, one or more of your commits are not signed off. Please click on the Details link next to the DCO action for instructions on how to resolve this.
-->
Checklist: