Sinensis
/

L3.3-MS-Nevoria-70b-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions

Sinensis commited on Jun 1

Commit

ed26bbf

·

verified ·

1 Parent(s): a0ef08d

Update README.md

Files changed (1) hide show

README.md +8 -4

README.md CHANGED Viewed

@@ -10,7 +10,11 @@ tags:
 ## AWQ quantization of [Steelskull/L3.3-MS-Nevoria-70b](https://huggingface.co/Steelskull/L3.3-MS-Nevoria-70b)
-zero_point: True,
-q_group_size: 128,
-w_bit: 4,
-version: "GEMM"

 ## AWQ quantization of [Steelskull/L3.3-MS-Nevoria-70b](https://huggingface.co/Steelskull/L3.3-MS-Nevoria-70b)
+  "quantization_config": {
+    "bits": 4,
+    "group_size": 128,
+    "modules_to_not_convert": null,
+    "quant_method": "awq",
+    "version": "gemm",
+    "zero_point": true
+  }