Problem with exllamav2
#1
by
tatianapoliakova
- opened
The latest exllamav2 and flash attention were installed. I use RTX 3xxx. I get gibberish, other models like qwen 2.5 work great. It seems that exllamav2 doesn't support the latest models yet.
you need to use the DEV branch of exllamav2 as there is no support on the main branch