run 20B with kernels-community/vllm-flash-attn3 on Ampere architecture, it predicts stupid things. Without flash-attn(remove attn_implementation), it runs normally.
Can someone help on this?
liougehooa
liougehooa
·
AI & ML interests
None yet
Recent Activity
new activity
8 days ago
Qwen/Qwen-Image:It's so unsafe that it can generate inappropriate adult content easily
commented on
an
article
15 days ago
Welcome GPT OSS, the new open-source model family from OpenAI!
new activity
6 months ago
unsloth/DeepSeek-R1-GGUF:failed to load model of UD-Q2_K_XL
Organizations
It's so unsafe that it can generate inappropriate adult content easily
37
#21 opened 16 days ago
by
liougehooa
commented on
Welcome GPT OSS, the new open-source model family from OpenAI!
15 days ago
failed to load model of UD-Q2_K_XL
#41 opened 6 months ago
by
liougehooa
liked
a
model
about 1 year ago
Repetitive Response Issue with Mistral-7B-v0.1 Model on Single Query
2
#119 opened over 1 year ago
by
9mavrick