add files

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,17 @@
+
+---
+base_model:
+- NousResearch/DeepHermes-3-Llama-3-8B-Preview
+---
+
+This is a converted weight from [DeepHermes-3-Llama-3-8B-Preview](https://huggingface.co/NousResearch/DeepHermes-3-Llama-3-8B-Preview) model in [unsloth 4-bit dynamic quant](https://archive.is/EFz7P) using this [collab notebook](https://colab.research.google.com/drive/1P23C66j3ga49kBRnDNlmRce7R_l_-L5l?usp=sharing).
+
+## About this Conversion
+
+This conversion uses **Unsloth** to load the model in **4-bit** format and force-save it in the same **4-bit** format.
+
+### How 4-bit Quantization Works
+- The actual **4-bit quantization** is handled by **BitsAndBytes (bnb)**, which works under **Torch** via **AutoGPTQ** or **BitsAndBytes**.
+- **Unsloth** acts as a wrapper, simplifying and optimizing the process for better efficiency.
+
+This allows for reduced memory usage and faster inference while keeping the model compact.

config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7728e75b08bb5a717e6e185d78d5af63d56a6ffb648daab5cfc80fee447eb20d
+size 1377

generation_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:394c06e45d928bf2efc564213881fdd32b063ceffc1a043d3c9bdbc0c3b89f8d
+size 264

model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ada40470dfbd4300f8228280fdc2aab18b4e7e06bd38b478be444ee193a4c113
+size 4652072838

model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8dacd2ce63e3cb6911d00c41a17e0203fe3c4630e830ebde7e0c73227cdc586
+size 1050673280

model.safetensors.index.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61d34b8f4de8b9a5d7cfe406a070063869de2c6ed1d36ce3fed068fea5c5afdf
+size 132271

special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a6d7fa83a01e8192333cd7b848541159709c4b206739071980432612f807807
+size 444

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b96a16793869fa44589efebc7a70184a93f8b4f2b44f4be957ea7f520e323d3e
+size 56569