YannQi

/

R-4B

like 135

Hello, and thank you for your fantastic work on this model!

I'm encountering an issue when trying to serve the YannQi/R-4B model using the official vllm/vllm-openai Docker image. I've tried both the :latest and :v0.10.1.1 tags, but unfortunately, both result in an error.

I used the following command:

docker run --rm \
    --gpus all \
    --name R-4B \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    -p 8003:8000 \
    --ipc=host \
    vllm/vllm-openai:v0.10.1.1 \
    --model YannQi/R-4B \
    --gpu-memory-utilization 0.9 \
    --trust-remote-code

The error message:

ValueError: There is no module or parameter named 'model.language_model' in TransformersForMultimodalLM

Could you possibly provide some guidance on how to resolve this? Thanks in advance for your help!