Tracking issues for release 0.2.0

[ahg-g]

Must haves

• Support ext_proc FULL_DUPLEX_STREAMED mode #388
• The metrics refresh time might be much larger than the refreshMetricsInterval #99
• Refactor the vllm specific code to become model server agnostic #383
• Make ModelName immutable #408
• Remove EndpointPickerNotHealthy condition from InferencePool Status #385
• Remove v1alpha1 API #401
• Creating multiple InferenceModels with the same ModelName causes inconsistencies #394
• Rename TargetPortNumber to PortNumber in ExtensionRef #376
• InferencePool should have nested status #379
• Contract between Envoy Extproc and EPP is incorrect. #298

Nice to have

• Code health and repo maintenance
  • Address all vulnerabilities flagged on the published images #344
  • Create a cmd top level directory and move main.go there #345
  • Add license headers to all files #281
  • Rearrange directory structure of the repo #348
  • Create a separate pkg for the controllers #359
  • Create a separate datastore pkg #358
  • Clean up any more V(numeric) for log levels #307
• Testing
  • Integration tests should create pods instead of injecting the datastore directly #357
  • Verify metrics in integration test #326
  • Add integration tests on error cases #325
  • Add an integration test on traffic splitting behavior #324
  • Controllers unit tests should be done via the Reconcile call #356
• Guides
  • The getting started guide should include cleanup instructions #245
  • Move the metrics guide to the site #347
  • Move the user guide and manifests to the site #305
• Features
  • Emit events to indicate InferencePool and InferenceModel status #382
• Metrics
  • Handle response body parsing for both streaming and non-streaming cases #178
  • Allow defining a default base model in the lora syncer configuration #340
  • [Metrics] Add request error metrics #269
• Optimizations
  • Setup server-side filter of the pods #369

This is an initial list subject to changes after discussing with the community

[ahg-g]

/cc @danehans @kfswain

[Jeffwan]

I will leave some feedback by end of week.