neuralmagic
/

Meta-Llama-3.1-405B-Instruct-quantized.w4a16

Text Generation

compressed-tensors

Model card Files Files and versions Community

Resources

View closed (0)

how many resources were used for quantizing this model?

#4 opened 6 months ago by

Unable to use fp8 kv cache with neuralmagic quants on ampere

#3 opened 6 months ago by

Storage format differs from other w4a16 models

#2 opened 6 months ago by

weights does not exist when trying to deploy in sagemaker endpoint

#1 opened 6 months ago by

LorenzoCevolaniAXA