Unable to Pass User Query & Tool Calls in a single request.
#81
by
sudhe-er
- opened
Hi Mistral Community,
Thank you for making this Vision Model open-source, it is one of the best model in terms of Function Calling, Low Latency, Reasoning Capabilities in recent times. So I've downloaded this model and deployed in our Local GPUs using vLLM Library for high-throughput and inference. However, I've observed whenever connecting to this model with user query and tool calls in a same api request, am running into issues stating that, "You can't pass user query and tool calls at a same time when inferencing". I don't think it's an issue with the OpenAI API specification that this vLLM community uses for inference. Can you look into this and shed some light on this. Thank you in advance!