Improve vLLM usage guidance
#4
by oliverholworthy - opened
Summary
- Simplify the offline vLLM example by relying on validated defaults.
- Add a Blackwell serving example with FlashAttention and a 32K CUDA Graph capture profile.
- Explain capture-size, API-server, and startup tradeoffs, with links to the relevant vLLM tuning documentation.
Validation
- Tested the serving profile with vLLM 0.25.0 on NVIDIA GB10: CUDA Graph capture completed in 43 seconds, used 12.40 GiB, and the API became ready in approximately 132 seconds.
- Verified that Bash and Zsh expand the profile to 371 strictly increasing sizes from 1 through 32,768.
- Confirmed that the CLI flags exist in the tested upstream vLLM 0.22.1 and 0.25.0 releases.
- Ran
git diff --check.
oliverholworthy changed pull request status to open
oliverholworthy changed pull request status to merged