language:

- en
license: other
library_name: sglang
pipeline_tag: text-generation
tags:
- grok-2
- xai
- sglang
- inference
- triton
base_model: xai-org/grok-2
model-index:
- name: grok-2
results: []

# Grok 2

This repository contains the weights of Grok 2, a model trained and used at xAI in 2024.

- License: Grok 2 Community License Agreement (./LICENSE)
- Ownership: xAI (no changes to license or weights in this PR)

## Weights

- Download from the Hub (≈500 GB total; 42 files):
  hf download xai-org/grok-2 --local-dir /local/grok-2
If you see transient errors, retry until it completes.

## Hardware and Parallelism

- This checkpoint is configured for TP=8.
- Recommended: 8× GPUs (each > 40 GB memory).

## Serving with SGLang (>= v0.5.1)

Install SGLang from https://github.com/sgl-project/sglang/

Launch an inference server:
python3 -m sglang.launch_server \
  --model /local/grok-2 \
  --tokenizer-path /local/grok-2/tokenizer.tok.json \
  --tp 8 \
  --quantization fp8 \
  --attention-backend triton
Send a test request (chat template aware):
python3 -m sglang.test.send_one --prompt \
"Human: What is your name?<|separator|>\n\nAssistant:"
You should see the model respond with its name: “Grok”.

More ways to send requests:

- https://docs.sglang.ai/basic_usage/send_request.html
- Note: this is a post-trained model; use the correct chat template:
https://github.com/sgl-project/sglang/blob/97a38.../tiktoken_tokenizer.py#L106

## Community Usage (Examples)

- Local-only serving behind VPN/Nginx allowlist
- Log and audit inference (timestamps and SHA-256 manifests)
- Optional cloud fallback to xAI’s API when local capacity is unavailable

These are usage patterns only; they don’t alter license or weights.

## Limitations and Safety

- Large memory footprint (multi-GPU recommended)
- Follow the Grok 2 Community License
- Redact any sensitive data before inference if routing via cloud services

## License

Weights are licensed under the Grok 2 Community License Agreement (./LICENSE).

خياراتك الآن