Description

• Deploys a vLLM OpenAI-compatible LLM server as an MLRun application runtime, with configurable GPU usage, node selection, tensor parallelism, and runtime flags.

module Version

MLRun Version

• 1.10.0

Kind

• generic

Tags

• genai

Publisher

• Iguazio