|
|
|
|
|
--- |
|
|
language: |
|
|
- en |
|
|
license: other |
|
|
library_name: sglang |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- grok-2 |
|
|
- xai |
|
|
- sglang |
|
|
- inference |
|
|
- triton |
|
|
base_model: xai-org/grok-2 |
|
|
model-index: |
|
|
- name: grok-2 |
|
|
results: [] |
|
|
|
|
|
--- |
|
|
|
|
|
# Grok 2 |
|
|
|
|
|
This repository contains the weights of Grok 2, a model trained and used at xAI in 2024. |
|
|
|
|
|
- License: Grok 2 Community License Agreement (./LICENSE) |
|
|
- Ownership: xAI (this document does not change license or weights) |
|
|
|
|
|
## Weights |
|
|
|
|
|
Download from the Hub (≈500 GB total; 42 files): |
|
|
|
|
|
```bash |
|
|
hf download xai-org/grok-2 --local-dir /local/grok-2 |
|
|
|
|
|
python3 -m sglang.launch_server \ |
|
|
--model /local/grok-2 \ |
|
|
--tokenizer-path /local/grok-2/tokenizer.tok.json \ |
|
|
--tp 8 \ |
|
|
--quantization fp8 \ |
|
|
--attention-backend triton |
|
|
|
|
|
python3 -m sglang.test.send_one --prompt \ |
|
|
"Human: What is your name?<|separator|>\n\nAssistant:" |
|
|
|
|
|
|
|
|
https://github.com/sgl-project/sglang/blob/97a38ee85ba62e268bde6388f1bf8edfe2ca9d76/python/sglang/srt/tokenizer/tiktoken_tokenizer.py#L106 |
|
|
|
