RedHatAI
/

Sparse-Llama-3.1-8B-tldr-2of4

Text Generation

Generated from Trainer

text-generation-inference

compressed-tensors

Model card Files Files and versions

alexmarques commited on Jun 6

Commit

16460ea

·

verified ·

1 Parent(s): eb855d8

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -302,7 +302,7 @@ ds.to_json("tldr_1000.json")
 2. Start a vLLM server using your target model:
 ```bash
-vllm serve RedHatAI/Sparse-Llama-3.1-8B-tldr
 ```
 3. Run the benchmark with GuideLLM:

 2. Start a vLLM server using your target model:
 ```bash
+vllm serve RedHatAI/Sparse-Llama-3.1-8B-tldr-2of4
 ```
 3. Run the benchmark with GuideLLM: