Improve vLLM usage guidance

README.md

@@ -55,7 +55,7 @@ pipeline_tag: sentence-similarity
 # Model Overview
 
 ### Description
-**Nemotron-3-Embed-1B-NVFP4** is the quantized version of the [Nemotron-3-Embed-1B-BF16](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16) model, which is developed for text question-answering retrieval. For more information, please check [here](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16). The **Nemotron-3-Embed-1B-NVFP4** model is quantized with [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer), using nvidia-modelopt v0.45.0. 
+**Nemotron-3-Embed-1B-NVFP4** is the quantized version of the [Nemotron-3-Embed-1B-BF16](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16) model, which is developed for text question-answering retrieval. For more information, please check [here](https://huggingface.co/nvidia/Nemotron-3-Embed-1B-BF16). The **Nemotron-3-Embed-1B-NVFP4** model is quantized with [NVIDIA Model Optimizer](https://github.com/NVIDIA/Model-Optimizer), using nvidia-modelopt v0.45.0.
 This model was evaluated on 34 languages: English, Arabic, Assamese, Bengali, Bulgarian, Chinese, Danish, Dutch, Finnish, French, German, Hindi, Hinglish, Indonesian, Italian, Japanese, Korean, Malay, Marathi, Nepali, Norwegian, Persian, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Tamil, Telugu, Thai, Ukrainian, Urdu, Vietnamese. Read more details in our [Blog Post](https://huggingface.co/blog/nvidia/nemotron-3-embed-wins-rteb).
 
 This model is ready for commercial use.
@@ -145,7 +145,10 @@ pip install --upgrade "vllm==0.25.0" openai requests numpy
 
 ### vLLM Offline Python
 
-Use the offline Python API for local vLLM inference without an HTTP server. `LLM.embed` accepts formatted strings. Add the `query: ` and `passage: ` prefixes manually. The example uses a 4,096-token limit. Review [CUDA graph sizing](#cuda-graph-sizing) before increasing it.
+Use the offline Python API for local vLLM inference without an HTTP server.
+`LLM.embed` accepts formatted strings.
+Add the `query: ` and `passage: ` prefixes manually.
+The example limits input sequences to 4,096 tokens and leaves batch and CUDA graph sizing at the vLLM defaults.
 
 <details>
 <summary>vLLM Offline Python Example</summary>
@@ -156,7 +159,6 @@ from vllm import LLM
 
 MODEL_ID = "nvidia/Nemotron-3-Embed-1B-NVFP4"
 MAX_MODEL_LEN = 4096
-MAX_BATCHED_TOKENS = 4096
 
 QUERIES = [
     "Write a Python function that counts the frequency of each element in a list of lists.",
@@ -176,8 +178,6 @@ def main():
     llm = LLM(
         model=MODEL_ID,
         max_model_len=MAX_MODEL_LEN,
-        max_num_batched_tokens=MAX_BATCHED_TOKENS,
-        max_cudagraph_capture_size=MAX_BATCHED_TOKENS,
     )
     texts = ["query: " + query for query in QUERIES] + [
         "passage: " + doc for doc in DOCUMENTS
@@ -222,58 +222,37 @@ q[3]   -0.0222   0.0265   0.1261   0.7677
 
 ### vLLM Online Serving
 
-Tested vLLM builds read the checkpoint NVFP4 metadata, so no quantization flag is required. Start the server with the following command:
+vLLM reads the checkpoint's NVFP4 and pooling metadata, so you do not need to set quantization or runner flags.
 
-```bash
-MODEL_ID=nvidia/Nemotron-3-Embed-1B-NVFP4
-MAX_MODEL_LEN=4096
-MAX_BATCHED_TOKENS=4096
-
-vllm serve "$MODEL_ID" \
-  --max-model-len "$MAX_MODEL_LEN" \
-  --max-num-batched-tokens "$MAX_BATCHED_TOKENS" \
-  --max-cudagraph-capture-size "$MAX_BATCHED_TOKENS"
-```
-
-#### CUDA Graph Sizing
-
-The checkpoint supports sequences up to 32,768 tokens. The examples use 4,096 as a conservative starting point.
-
-Use the following guidance to tune CUDA graph capture:
-
-- Set `--max-model-len` to the longest request you intend to serve. Tune `--max-num-batched-tokens` for the workload, concurrency, and available GPU memory. When chunked prefill is disabled, the batched-token budget must be at least the model-length limit.
-- For default capture buckets up to 8,192, set `--max-cudagraph-capture-size` equal to `--max-num-batched-tokens`. This setting makes batches up to the scheduler budget eligible for CUDA graph execution. Batches outside the captured range use a slower uncaptured path.
-- Larger capture ranges increase startup time and graph memory. Benchmark representative traffic on the target hardware. Do not capture beyond the batched-token budget.
-
-Use a maximum capture size of 4,096 as the conservative default for services that restart or autoscale regularly. A maximum capture size of 8,192 can be reasonable when a longer cold start is acceptable. For maximum capture sizes above 8,192, pass a smaller, workload-aligned set with `--cudagraph-capture-sizes` to keep startup time under control.
-
-<details>
-<summary>Example Sparse Capture Sizes for Inputs up to 32,768 Tokens</summary>
-
-The following command uses a sparse capture-size list:
+Use the following Bash or Zsh command as a starting point on NVIDIA Blackwell GPUs.
+Set `VLLM_CACHE_ROOT` to a writable directory that persists across restarts so vLLM can reuse compatible compilation and FP4 GEMM autotuning artifacts:
 
 ```bash
-MODEL_ID=nvidia/Nemotron-3-Embed-1B-NVFP4
-MAX_BATCHED_TOKENS=32768
+export VLLM_CACHE_ROOT=/mnt/vllm-cache
+mkdir -p "$VLLM_CACHE_ROOT"
 
-vllm serve "$MODEL_ID" \
-  --max-num-batched-tokens "$MAX_BATCHED_TOKENS" \
+vllm serve nvidia/Nemotron-3-Embed-1B-NVFP4 \
   --cudagraph-capture-sizes \
-    1 2 4 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120 128 \
-    136 144 152 160 168 176 184 192 200 208 216 224 232 240 248 256 \
-    384 512 768 1024 1536 2048 3072 4096 6144 8192 12288 16384 \
-    24576 32768
+    1 2 4 {8..248..8} {256..4096..16} \
+    {4224..8192..128} {8448..16384..256} \
+    {16896..32768..512}
 ```
 
-Choose sizes from representative batch-token measurements. vLLM pads each execution batch to the next captured size, so denser lists reduce padding but require more startup time and graph memory.
+The example uses `/mnt/vllm-cache`, but you can use any writable directory that persists across restarts.
+For container deployments, mount persistent storage at the directory you choose.
+Bash and Zsh expand the capture-size ranges before vLLM starts.
+For other launchers, pass the expanded integer list to `--cudagraph-capture-sizes`.
 
-In illustrative tests on an NVIDIA GB10 system with vLLM 0.25.0, cold startup using automatic buckets took about 74 seconds at a maximum capture size of 4,096 and 121 seconds at 8,192. At 32,768, startup with automatic buckets was projected to take tens of minutes. With the sparse list of 49 capture sizes above, startup completed in about one minute. Results vary by workload and hardware.
+The sparse CUDA graph profile is a practical starting point. Benchmark representative traffic before changing it or increasing `--api-server-count`.
+For other cache or tuning configurations, refer to the vLLM [`VLLM_CACHE_ROOT` reference](https://docs.vllm.ai/en/v0.25.0/usage/security/?h=vllm_cache_root#cache-directory-configuration), [`--cudagraph-capture-sizes` reference](https://docs.vllm.ai/en/v0.25.0/configuration/engine_args/#-cudagraph-capture-sizes), and [Parallel Processing guide](https://docs.vllm.ai/en/v0.25.0/configuration/optimization/#parallel-processing).
 
-</details>
+Expect the first startup to take several minutes while vLLM tunes FP4 GEMMs and captures CUDA graphs.
+vLLM recaptures CUDA graphs on every launch.
+Startup time varies by hardware and deployment configuration.
 
 Add `--host` or `--port` to the serving command if you need non-default network settings.
 
-To serve a local checkpoint, replace `MODEL_ID` with its path. Add `--served-model-name nvidia/Nemotron-3-Embed-1B-NVFP4` if clients should continue using the Hugging Face model ID.
+To serve a local checkpoint, replace the model ID with its path. Add `--served-model-name nvidia/Nemotron-3-Embed-1B-NVFP4` if clients should continue using the Hugging Face model ID.
 
 #### Recommended Retrieval Endpoint