Update README.md
Browse files
README.md
CHANGED
|
@@ -3,8 +3,67 @@ license: apache-2.0
|
|
| 3 |
base_model:
|
| 4 |
- tomg-group-umd/DynaGuard-4B
|
| 5 |
- tomg-group-umd/DynaGuard-1.7B
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
---
|
| 7 |
|
| 8 |
# **DynaGuard-AIO-GGUF**
|
| 9 |
|
| 10 |
-
> DynaGuard is a dynamic guardrail model framework that enables user-defined content moderation policies for generative AI, providing real-time, configurable controls on both model inputs and outputs. It employs ultralightweight, efficient models optimized for on-device or cloud deployments, allowing organizations to protect against risks like data leakage, PII exposure, toxicity, jailbreaking, and prompt injection with sub-50ms latency and minimal compute overhead. Users can create custom guardrails in natural language, tailoring moderation behaviors, targeted content categories, and downstream actions, making DynaGuard ideal for enterprise AI security, compliance, and safe application development across diverse hardware platforms.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
base_model:
|
| 4 |
- tomg-group-umd/DynaGuard-4B
|
| 5 |
- tomg-group-umd/DynaGuard-1.7B
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
+
pipeline_tag: text-generation
|
| 9 |
+
library_name: transformers
|
| 10 |
+
tags:
|
| 11 |
+
- text-generation-inference
|
| 12 |
---
|
| 13 |
|
| 14 |
# **DynaGuard-AIO-GGUF**
|
| 15 |
|
| 16 |
+
> DynaGuard is a dynamic guardrail model framework that enables user-defined content moderation policies for generative AI, providing real-time, configurable controls on both model inputs and outputs. It employs ultralightweight, efficient models optimized for on-device or cloud deployments, allowing organizations to protect against risks like data leakage, PII exposure, toxicity, jailbreaking, and prompt injection with sub-50ms latency and minimal compute overhead. Users can create custom guardrails in natural language, tailoring moderation behaviors, targeted content categories, and downstream actions, making DynaGuard ideal for enterprise AI security, compliance, and safe application development across diverse hardware platforms.
|
| 17 |
+
|
| 18 |
+
## Model Files
|
| 19 |
+
|
| 20 |
+
### DynaGuard-1.7B
|
| 21 |
+
|
| 22 |
+
| File Name | Quant Type | File Size |
|
| 23 |
+
| - | - | - |
|
| 24 |
+
| DynaGuard-1.7B.BF16.gguf | BF16 | 3.45 GB |
|
| 25 |
+
| DynaGuard-1.7B.F16.gguf | F16 | 3.45 GB |
|
| 26 |
+
| DynaGuard-1.7B.F32.gguf | F32 | 6.89 GB |
|
| 27 |
+
| DynaGuard-1.7B.Q2_K.gguf | Q2_K | 778 MB |
|
| 28 |
+
| DynaGuard-1.7B.Q3_K_L.gguf | Q3_K_L | 1 GB |
|
| 29 |
+
| DynaGuard-1.7B.Q3_K_M.gguf | Q3_K_M | 940 MB |
|
| 30 |
+
| DynaGuard-1.7B.Q3_K_S.gguf | Q3_K_S | 867 MB |
|
| 31 |
+
| DynaGuard-1.7B.Q4_0.gguf | Q4_0 | 1.05 GB |
|
| 32 |
+
| DynaGuard-1.7B.Q4_1.gguf | Q4_1 | 1.14 GB |
|
| 33 |
+
| DynaGuard-1.7B.Q4_K.gguf | Q4_K | 1.11 GB |
|
| 34 |
+
| DynaGuard-1.7B.Q4_K_M.gguf | Q4_K_M | 1.11 GB |
|
| 35 |
+
| DynaGuard-1.7B.Q4_K_S.gguf | Q4_K_S | 1.06 GB |
|
| 36 |
+
| DynaGuard-1.7B.Q5_0.gguf | Q5_0 | 1.23 GB |
|
| 37 |
+
| DynaGuard-1.7B.Q5_1.gguf | Q5_1 | 1.32 GB |
|
| 38 |
+
| DynaGuard-1.7B.Q5_K.gguf | Q5_K | 1.26 GB |
|
| 39 |
+
| DynaGuard-1.7B.Q5_K_M.gguf | Q5_K_M | 1.26 GB |
|
| 40 |
+
| DynaGuard-1.7B.Q5_K_S.gguf | Q5_K_S | 1.23 GB |
|
| 41 |
+
| DynaGuard-1.7B.Q6_K.gguf | Q6_K | 1.42 GB |
|
| 42 |
+
| DynaGuard-1.7B.Q8_0.gguf | Q8_0 | 1.83 GB |
|
| 43 |
+
|
| 44 |
+
### DynaGuard-4B
|
| 45 |
+
|
| 46 |
+
| File Name | Quant Type | File Size |
|
| 47 |
+
| - | - | - |
|
| 48 |
+
| DynaGuard-4B.BF16.gguf | BF16 | 8.05 GB |
|
| 49 |
+
| DynaGuard-4B.F16.gguf | F16 | 8.05 GB |
|
| 50 |
+
| DynaGuard-4B.F32.gguf | F32 | 16.1 GB |
|
| 51 |
+
| DynaGuard-4B.Q2_K.gguf | Q2_K | 1.67 GB |
|
| 52 |
+
| DynaGuard-4B.Q3_K_L.gguf | Q3_K_L | 2.24 GB |
|
| 53 |
+
| DynaGuard-4B.Q3_K_M.gguf | Q3_K_M | 2.08 GB |
|
| 54 |
+
| DynaGuard-4B.Q3_K_S.gguf | Q3_K_S | 1.89 GB |
|
| 55 |
+
| DynaGuard-4B.Q4_K_M.gguf | Q4_K_M | 2.5 GB |
|
| 56 |
+
| DynaGuard-4B.Q4_K_S.gguf | Q4_K_S | 2.38 GB |
|
| 57 |
+
| DynaGuard-4B.Q5_K_M.gguf | Q5_K_M | 2.89 GB |
|
| 58 |
+
| DynaGuard-4B.Q5_K_S.gguf | Q5_K_S | 2.82 GB |
|
| 59 |
+
| DynaGuard-4B.Q6_K.gguf | Q6_K | 3.31 GB |
|
| 60 |
+
| DynaGuard-4B.Q8_0.gguf | Q8_0 | 4.28 GB |
|
| 61 |
+
|
| 62 |
+
## Quants Usage
|
| 63 |
+
|
| 64 |
+
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
| 65 |
+
|
| 66 |
+
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
| 67 |
+
types (lower is better):
|
| 68 |
+
|
| 69 |
+
|