---
language:
- ar
- en
base_model:
- xai-org/grok-2
tags:
- agent
- finance
- code
---
language:

- en
license: other
library_name: sglang
pipeline_tag: text-generation
tags:
- grok-2
- xai
- sglang
- inference
- triton
base_model: xai-org/grok-2
model-index:
- name: grok-2
results: []

# Grok 2

This repository contains the weights of Grok 2, a model trained and used at xAI in 2024.

- License: Grok 2 Community License Agreement (./LICENSE)
- Ownership: xAI (this document does not change license or weights)

## Weights

Download from the Hub (≈500 GB total; 42 files):
hf download xai-org/grok-2 --local-dir /local/grok-2
If you see transient errors, retry until it completes. On success, the directory should contain 42 files (~500 GB).

## Hardware and Parallelism

- This checkpoint is configured for TP=8.
- Recommended: 8× GPUs (each > 40 GB memory).

## Serving with SGLang (>= v0.5.1)

Install SGLang from https://github.com/sgl-project/sglang/

Launch an inference server:
python3 -m sglang.launch_server \
  --model /local/grok-2 \
  --tokenizer-path /local/grok-2/tokenizer.tok.json \
  --tp 8 \
  --quantization fp8 \
  --attention-backend triton
Send a test request (chat template aware):
python3 -m sglang.test.send_one --prompt \
"Human: What is your name?<|separator|>\n\nAssistant:"
You should see the model respond with its name: “Grok”.

More ways to send requests:

- https://docs.sglang.ai/basic_usage/send_request.html

Note: this is a post-trained model; use the correct chat template:

- https://github.com/sgl-project/sglang/blob/97a38ee85ba62e268bde6388f1bf8edfe2ca9d76/python/sglang/srt/tokenizer/
tiktoken_tokenizer.py#L106

## Community Usage (Examples)

- Local-only serving behind VPN/Nginx allowlist
- Log and audit inference (timestamps and SHA‑256 manifests)
- Optional fallback to xAI’s API when local capacity is unavailable

These examples describe usage patterns only; they do not alter license or weights.

## Limitations and Safety

- Large memory footprint (multi-GPU recommended)
- Follow the Grok 2 Community License
- Redact any sensitive data before inference if routing via cloud services

## License

Weights are licensed under the Grok 2 Community License Agreement (./LICENSE).

تعليق PR جاهز (الصق في وصف الطلب)

- Summary: Fix model card metadata (YAML at top), remove duplicated sections, fence all code blocks, and keep license/ownership
unchanged.
- Scope: README.md only. No weights or license changes.
- Rationale: Makes the card copy‑paste runnable for SGLang and resolves Hub’s YAML metadata warning.