TGI with Qwen 2.5 Coder 7B base
Looking at thelist of supported modelsfor TGI, I do not see Qwen 2.5 listed.Could I host an inference server for a fine-tuned model whose base is Qwen 2.5 Coder 7B in BF16 using the TGI offer?
2024-11-25T03:17:24Z
I think it’s probably just that the manual hasn’t been updated.Qwen 2.5 works with the Serverless Inference API and is the most popular.On the other hand, it is currently almost impossible to run models that you have improved yourself with the Serverless Inference API. I think it’s because the HF server doesn’t have enough resources.Of course, there is no problem with running it locally or using it from Spaces or other virtual environments after placing it on HF.huggingface.coModels - Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.huggingface.coSupported ModelsWe’re on a journey to advance and democratize artificial intelligence through open source and open science.
2024-11-25T09:31:51Z
Yes Qwen 2.5 is supported, since it is identical to Qwen 2.This is similar to the llama models (the docs only list llama, but it supports any version of llama).