Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create InferenceModel Controller #409

Open
kfswain opened this issue Feb 26, 2025 · 1 comment
Open

Create InferenceModel Controller #409

kfswain opened this issue Feb 26, 2025 · 1 comment

Comments

@kfswain
Copy link
Collaborator

kfswain commented Feb 26, 2025

What would you like to be added:
An InferenceModel controller is a separate deployable from the EPP

Why is this needed:
Gateway controllers will only run reconciliation on the InferencePool object, and set the status, as the EPP is a part of the InferencePool.

The EPP will, in the long term, be HA, and likely active-active, so should remain read-only. We need a distinct deployable controller that can handle validation and status updates of the InferenceModel

@Kuromesi
Copy link
Contributor

So updates and validation will be proceed in a separate pod? And does InferencePool need that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants