We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
What would you like to be added:
Allow defining a default base model in the lora syncer configuration
Why is this needed:
It is repetitive to list it in each list item in our case since we are mostly assuming a single base model per pool.
so instead of
apiVersion: v1 kind: ConfigMap metadata: name: vllm-llama2-7b-adapters data: configmap.yaml: | vLLMLoRAConfig: name: vllm-llama2-7b port: 8000 ensureExist: models: - base-model: meta-llama/Llama-2-7b-hf id: tweet-summary-1 source: vineetsharma/qlora-adapter-Llama-2-7b-hf-TweetSumm - base-model: meta-llama/Llama-2-7b-hf id: tweet-summary-2 source: mahimairaja/tweet-summarization-llama-2-finetuned
we should be able to define it as
apiVersion: v1 kind: ConfigMap metadata: name: vllm-llama2-7b-adapters data: configmap.yaml: | vLLMLoRAConfig: name: vllm-llama2-7b port: 8000 defaultBaseModel: meta-llama/Llama-2-7b-hf ensureExist: models: - id: tweet-summary-1 source: vineetsharma/qlora-adapter-Llama-2-7b-hf-TweetSumm - id: tweet-summary-2 source: mahimairaja/tweet-summarization-llama-2-finetuned
The text was updated successfully, but these errors were encountered:
No branches or pull requests
What would you like to be added:
Allow defining a default base model in the lora syncer configuration
Why is this needed:
It is repetitive to list it in each list item in our case since we are mostly assuming a single base model per pool.
so instead of
we should be able to define it as
The text was updated successfully, but these errors were encountered: