Skip to main content

SGLang Cookbook

License PRs Welcome

A community-maintained repository of practical guides and recipes for deploying and using SGLang in production environments. Our mission is simple: answer the question "How do I use SGLang (and related models) on hardware Y for task Z?" with clear, actionable solutions.

🎯 What You'll Find Here​

This cookbook aggregates battle-tested SGLang recipes covering:

  • Models: Mainstream LLMs and Vision-Language Models (VLMs)
  • Use Cases: Inference serving, deployment strategies, multimodal applications
  • Hardware: GPU and CPU configurations, optimization for different accelerators
  • Best Practices: Configuration templates, performance tuning, troubleshooting guides

Each recipe provides step-by-step instructions to help you quickly implement SGLang solutions for your specific requirements.

Guides​

DeepSeek​

Ernie​

GLM​

InternVL​

InternLM​

Jina AI​

Llama​

MiniMax​

OpenAI​

Qwen​

Moonshotai​

NVIDIA​

🚀 Quick Start​

  1. Browse the recipe index above to find your model
  2. Follow the step-by-step instructions in each guide
  3. Adapt configurations to your specific hardware and requirements
  4. Join our community to share feedback and improvements

🤝 Contributing​

We believe the best documentation comes from practitioners. Whether you've optimized SGLang for a specific model, solved a tricky deployment challenge, or discovered performance improvements, we encourage you to contribute your recipes!

Ways to contribute:

  • Add a new recipe for a model not yet covered
  • Improve existing recipes with additional tips or configurations
  • Report issues or suggest enhancements
  • Share your production deployment experiences

To contribute:

# Fork the repo and clone locally
git clone https://github.com/YOUR_USERNAME/sglang-cookbook.git
cd sglang-cookbook

# Create a new branch
git checkout -b add-my-recipe

# Add your recipe following the template in DeepSeek-V3.2
# Submit a PR!

🛠️ Local Development​

Prerequisites​

  • Node.js >= 20.0
  • npm or yarn

Setup and Run​

Install dependencies and start the development server:

# Install dependencies
npm install

# Start development server (hot reload enabled)
npm start

The site will automatically open in your browser at http://localhost:3000.

📖 Resources​

📄 License​

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Let's build this resource together! 🚀 Star the repo and contribute your recipes to help the SGLang community grow.