Update README.md
Browse files
README.md
CHANGED
@@ -52,13 +52,12 @@ vllm serve Menlo/Lucy-128k \
|
|
52 |
--host 0.0.0.0 \
|
53 |
--port 1234 \
|
54 |
--enable-auto-tool-choice \
|
55 |
-
--tool-call-parser hermes
|
56 |
-
--rope-scaling '{"rope_type":"yarn","factor":3.2,"original_max_position_embeddings":40960}' --max-model-len 131072
|
57 |
```
|
58 |
|
59 |
Or `llama-server` from `llama.cpp`:
|
60 |
```bash
|
61 |
-
llama-server ...
|
62 |
```
|
63 |
|
64 |
### Recommended Sampling Parameters
|
|
|
52 |
--host 0.0.0.0 \
|
53 |
--port 1234 \
|
54 |
--enable-auto-tool-choice \
|
55 |
+
--tool-call-parser hermes
|
|
|
56 |
```
|
57 |
|
58 |
Or `llama-server` from `llama.cpp`:
|
59 |
```bash
|
60 |
+
llama-server ...
|
61 |
```
|
62 |
|
63 |
### Recommended Sampling Parameters
|