LLM Deployment

Templates for serving open and commercial LLMs with common inference stacks

Use these templates to deploy LLM endpoints with preconfigured serving options.

Llama 3.1 8B Instruct

Compact Llama 3.1 deployment template for general-purpose inference

Qwen2.5 72B Instruct

Large Qwen2.5 template for advanced multilingual and reasoning tasks

Mistral Small 24B Instruct

Mid-size Mistral template balancing quality, memory use, and throughput