vLLM

Overview
vLLM on OneCloudPlanet — a production inference server: when Ollama is not enough, vLLM delivers maximum throughput on the same GPU via PagedAttention and continuous batching.

What is vLLM?
The fastest open LLM inference engine: hundreds of parallel requests, OpenAI-compatible API, Llama/Qwen/DeepSeek/Mistral from Hugging Face.

Why on OneCloudPlanet?
More requests per GPU — lower cost per token.
Recommended configuration: L40S 48 GB for 7–14B; A100/H100 and multi-GPU for larger. See Cloud GPU.
For products: point your app at vLLM like at OpenAI — one base_url change.

How to launch
1) Sign in at console.ocplanet.cloud
2) Open Marketplace and choose vLLM
3) Pick a configuration and location
4) Wait for deploy
5) Change the default credentials

Post-deploy checklist
• Pull a model from Hugging Face
• Load-test with your traffic profile
• API behind Firewall
• Model weights in Object Storage

FAQ
vLLM or Ollama? Ollama for experiments; vLLM for production traffic.
Quantized models? Yes: AWQ, GPTQ, FP8.