Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wait for the webhook service to be listening before advertising the Jobset replica as ready. #607

Closed
mbobrovskyi opened this issue Jun 20, 2024 · 5 comments · Fixed by #608
Assignees

Comments

@mbobrovskyi
Copy link
Contributor

mbobrovskyi commented Jun 20, 2024

What would you like to be added:
As mentioned on the title, wait for the webhook service to be listening before advertising the Jobset replica as ready. Like on the kueue here.

Why is this needed:
It causes flakes in kueue https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kueue/2408/pull-kueue-test-e2e-main-1-28/1802723422271180800. To fix it we need to wait for jobset operator ready and to be sure that webhooks runs success.

@mbobrovskyi
Copy link
Contributor Author

mbobrovskyi commented Jun 20, 2024

cc @mimowo @alculquicondor

@mimowo
Copy link
Contributor

mimowo commented Jun 20, 2024

+1 in Kueue we use readiness probes to delay marking the deployment as available until the webhook service is ready.

This allows users to wait for the webhooks service by waiting for availability of the deployment.

We use this mechanism in kueue e2e tests here, but it can also be checked conveniently by users by kubectl wait --for=condition=available .

@kannon92
Copy link
Contributor

Sounds very sane. @mbobrovskyi would you want to contribute a patch?

@mbobrovskyi
Copy link
Contributor Author

Yes.

/assign

@danielvegamyhre
Copy link
Contributor

Makes sense, thanks for working on this @mbobrovskyi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants