Fix clean_jobs missing --namespace flag causing BackoffLimitExceeded - #837
Conversation
The clean_jobs Makefile target ran kubectl delete jobs without specifying
the --namespace flag, so it searched the default namespace instead of the
release namespace (e.g. codacy-release). This meant stale hook jobs (DB
migrations etc.) from prior failed deployments were never cleaned up, and
helm's next attempt would find the existing BackoffLimitExceeded job and
immediately fail again.
Add --namespace ${NAMESPACE} to the kubectl delete command so it targets
the correct namespace for every environment (dev, sandbox, release).
AI-Generated: true
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Pull Request Overview
The PR addresses a deployment blocker by adding the missing --namespace flag to the clean_jobs Kubernetes deletion command, ensuring stale jobs are cleared from the correct environment. Codacy analysis indicates that the changes are up to standards with no new quality issues or complexity regressions.
While the logic appears sound, there is a gap in automated verification. A required test scenario to confirm the correct population of the namespace and label selector variables in the kubectl command is currently missing. Implementing this verification is recommended to prevent future regressions in environment-specific cleanup tasks.
Test suggestions
- Verify that clean_jobs executes kubectl delete with the correct namespace and label selector variables
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify that clean_jobs executes kubectl delete with the correct namespace and label selector variables
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
Problem
deploy_to_doks_release(job 30881) failed at the "Install Codacy" step with:The previous fix (#836) added a
clean_jobsMakefile target to delete stale hook jobs before the helm upgrade. However, that target ran:kubectl delete jobs -l app.kubernetes.io/instance=codacy-release --ignore-not-found=true…without a
--namespaceflag, so kubectl searched the default namespace instead ofcodacy-release. The command returned "No resources found" and the stale hook jobs (DB migrations, etc.) incodacy-releasewere never deleted. On the next helm upgrade attempt, helm found the existingBackoffLimitExceededjob and immediately failed again.Fix
Add
--namespace ${NAMESPACE}to thekubectl delete jobscommand in theclean_jobstarget so it targets the correct namespace for every environment (dev, sandbox, release).clean_jobs: -kubectl delete jobs -l app.kubernetes.io/instance=${RELEASE_NAME} --namespace ${NAMESPACE} --ignore-not-found=trueTest
Re-running
deploy_to_doks_releaseshould now successfully delete stale hook jobs in thecodacy-releasenamespace before the helm upgrade proceeds.https://app.circleci.com/agents/gh/codacy/chat/53169a49-798b-45fd-a1cc-9721760e5036