Fixed n>1 causing list index out of range with VLM

[jasonyux]

Motivation

When using VLM inference (e.g., the ones in test/srt/test_vision_openai_server.py), changing n=1 to n>1 will cause the following error:

(other error message omitted)
  File "/home/docker/.local/lib/python3.10/site-packages/sglang/srt/managers/tokenizer_manager.py", line 351, in _handle_batch_request
    await self._get_pixel_values(obj.image_data[index])
IndexError: list index out of range

Modifications

After printing out a few things, I find that this is because obj.image_data is not cloned into batches, unlike obj.sampling_params. So simply changing to obj.image_data worked.

---

Edit 1: the above will cause errors when facing batch requests. So instead of removing indexing, just clone the obj.image_data in a similar way as how other fields are cloned (e.g., obj.sampling_params)

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[merrymercy]

@jasonyux Can you fix the errors in CI? You can reproduce it by

python3 sglang/test/srt/test_openai_server.py

[jasonyux]

sure. I am working on it.