Create README.md

README.md

@@ -0,0 +1,32 @@
+---
+tags:
+- deepsparse
+---
+## Usage
+
+```python
+from deepsparse import TextGeneration
+model = TextGeneration(model="deployment")
+print(model("How to make banana bread?", max_new_tokens=200).generations[0].text)
+"""
+1. Start by preparing the ingredients. You will need 3 cups of all-purpose flour, 1 cup of sugar, 1 egg, 1/2 teaspoful of vanilla extract, 1/2 teaspoful of baking powder, 1/2 teaspoful of salt, and 1 cup of milk.
+2. Mix the flour, sugar, and egg in a bowl.
+3. In a separate bowl, mix the vanilla extract and baking powder.
+4. Add the salt and milk to the flour mixture.
+5. Mix everything together until you have a smooth batter.
+6. Pour the batter into a greased 9-inch pan.
+7. Bake the bread at 350°F for 20 minutes.
+8. Let the bread cool and then slice it into individual slices.
+9.
+"""
+```
+## One-shot and Export
+
+```bash
+git clone https://github.com/neuralmagic/sparseml
+pip install -e "sparseml[transformers]"
+python sparseml/src/sparseml/transformers/sparsification/obcq/obcq.py TinyLlama/TinyLlama-1.1B-Chat-v0.4 open_platypus --recipe recipe.yaml --save True
+python sparseml/src/sparseml/transformers/sparsification/obcq/export.py --task text-generation --model_path obcq_deployment 
+cp deployment/model.onnx deployment/model-orig.onnx
+python onnx_kv_inject.py --input-file deployment/model-orig.onnx --output-file deployment/model.onnx
+```