GLM-4.7

1. Model Introduction

GLM-4.7 is the latest and most powerful language model in the GLM series developed by Zhipu AI, featuring state-of-the-art capabilities in reasoning, function calling, and multi-modal understanding.

As the newest iteration in the GLM series, GLM-4.7 achieves significant improvements across all domains:

• Extended Context Window: Expanded context window supporting even longer documents and complex multi-turn conversations
• Enhanced Reasoning: Improved reasoning capabilities with better chain-of-thought processing
• Superior Coding: Significantly improved code generation and understanding, with better real-world application performance
• Advanced Tool Use: More robust tool calling and agent capabilities for complex workflows
• Optimized Performance: Better throughput and latency characteristics across all hardware platforms

For more details, please refer to the official GLM-4.7 documentation.

Key Features:

• State-of-the-Art Reasoning: Enhanced reasoning capabilities for the most complex problem-solving tasks
• Multiple Quantizations: BF16 and FP8 variants for different performance/memory trade-offs
• Hardware Optimization: Specifically tuned for AMD MI300X/MI325X/MI355X GPUs
• High Performance: Optimized for both throughput and latency scenarios

Available Models:

• BF16 (Full precision): zai-org/GLM-4.7 - Recommended for MI300X/MI325X/MI355X
• FP8 (8-bit quantized): zai-org/GLM-4.7-FP8 - Recommended for MI300X/MI325X/MI355X

License:

Please refer to the official GLM-4.7 model card for license details.

2. SGLang Installation

SGLang offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.

Please refer to the official SGLang installation guide for installation instructions.

3. Model Deployment

This section provides deployment configurations optimized for different hardware platforms and use cases.

3.1 Basic Configuration

Interactive Command Generator: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform, quantization method, deployment strategy, and thinking capabilities.

Hardware Platform

MI300XMI325XMI355X

Quantization

BF16FP8

Deployment Strategy

TPTensor ParallelDPData ParallelEPExpert ParallelMTPMulti-token Prediction

Thinking Capabilities

DisabledEnabled

Tool Call Parser

DisabledEnabled

Run this Command:

python -m sglang.launch_server \
  --model zai-org/GLM-4.7 \
  --tp 4 \
  --context-length 8192 \
  --mem-fraction-static 0.9

3.2 Configuration Tips

For more detailed configuration tips, please refer to GLM-4.7 Usage.

4. Model Invocation

4.1 Basic Usage

For basic API usage and request examples, please refer to:

• SGLang Basic Usage Guide

4.2 Advanced Usage

4.2.1 Reasoning Parser

GLM-4.7 supports Thinking mode by default. Enable the reasoning parser during deployment to separate the thinking and the content sections:

python -m sglang.launch_server \
  --model zai-org/GLM-4.7 \
  --reasoning-parser glm47 \
  --tp 8 \
  --host 0.0.0.0 \
  --port 8000

Streaming with Thinking Process:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY"
)

# Enable streaming to see the thinking process in real-time
response = client.chat.completions.create(
    model="zai-org/GLM-4.7",
    messages=[
        {"role": "user", "content": "Solve this problem step by step: What is 15% of 240?"}
    ],
    temperature=0.7,
    max_tokens=2048,
    stream=True
)

# Process the stream
has_thinking = False
has_answer = False
thinking_started = False

for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta

        # Print thinking process
        if hasattr(delta, 'reasoning_content') and delta.reasoning_content:
            if not thinking_started:
                print("=============== Thinking =================", flush=True)
                thinking_started = True
            has_thinking = True
            print(delta.reasoning_content, end="", flush=True)

        # Print answer content
        if delta.content:
            # Close thinking section and add content header
            if has_thinking and not has_answer:
                print("\n=============== Content =================", flush=True)
                has_answer = True
            print(delta.content, end="", flush=True)

print()

Output Example:

=============== Thinking =================
To solve this problem, I need to calculate 15% of 240.
Step 1: Convert 15% to decimal: 15% = 0.15
Step 2: Multiply 240 by 0.15
Step 3: 240 × 0.15 = 36
=============== Content =================

The answer is 36. To find 15% of 240, we multiply 240 by 0.15, which equals 36.

Note: The reasoning parser captures the model's step-by-step thinking process, allowing you to see how the model arrives at its conclusions.

4.2.2 Tool Calling

GLM-4.7 supports tool calling capabilities. Enable the tool call parser:

python -m sglang.launch_server \
  --model zai-org/GLM-4.7 \
  --reasoning-parser glm47 \
  --tool-call-parser glm47 \
  --tp 8 \
  --host 0.0.0.0 \
  --port 8000

Python Example (with Thinking Process):

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="EMPTY"
)

# Define available tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city name"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

# Make request with streaming to see thinking process
response = client.chat.completions.create(
    model="zai-org/GLM-4.7",
    messages=[
        {"role": "user", "content": "What's the weather in Beijing?"}
    ],
    tools=tools,
    temperature=0.7,
    stream=True
)

# Process streaming response
thinking_started = False
has_thinking = False

for chunk in response:
    if chunk.choices and len(chunk.choices) > 0:
        delta = chunk.choices[0].delta

        # Print thinking process
        if hasattr(delta, 'reasoning_content') and delta.reasoning_content:
            if not thinking_started:
                print("=============== Thinking =================", flush=True)
                thinking_started = True
            has_thinking = True
            print(delta.reasoning_content, end="", flush=True)

        # Print tool calls
        if hasattr(delta, 'tool_calls') and delta.tool_calls:
            # Close thinking section if needed
            if has_thinking and thinking_started:
                print("\n=============== Content =================", flush=True)
                thinking_started = False

            for tool_call in delta.tool_calls:
                if tool_call.function:
                    print(f"Tool Call: {tool_call.function.name}")
                    print(f"   Arguments: {tool_call.function.arguments}")

        # Print content
        if delta.content:
            print(delta.content, end="", flush=True)

print()

Output Example:

=============== Thinking =================
The user is asking about the weather in Beijing. I need to use the get_weather function to retrieve this information.
I should call the function with location="Beijing".
=============== Content =================

Tool Call: get_weather
   Arguments: {"location": "Beijing", "unit": "celsius"}

Note:

• The reasoning parser shows how the model decides to use a tool
• Tool calls are clearly marked with the function name and arguments
• You can then execute the function and send the result back to continue the conversation

Handling Tool Call Results:

# After getting the tool call, execute the function
def get_weather(location, unit="celsius"):
    # Your actual weather API call here
    return f"The weather in {location} is 22°{unit[0].upper()} and sunny."

# Send tool result back to the model
messages = [
    {"role": "user", "content": "What's the weather in Beijing?"},
    {
        "role": "assistant",
        "content": None,
        "tool_calls": [{
            "id": "call_123",
            "type": "function",
            "function": {
                "name": "get_weather",
                "arguments": '{"location": "Beijing", "unit": "celsius"}'
            }
        }]
    },
    {
        "role": "tool",
        "tool_call_id": "call_123",
        "content": get_weather("Beijing", "celsius")
    }
]

final_response = client.chat.completions.create(
    model="zai-org/GLM-4.7",
    messages=messages,
    temperature=0.7
)

print(final_response.choices[0].message.content)
# Output: "The weather in Beijing is currently 22°C and sunny."

5. Benchmark

This section uses industry-standard configurations for comparable benchmark results.

5.1 Speed Benchmark

Test Environment:

• Hardware: AMD MI300X (8x), AMD MI325X (8x), AMD MI355X (8x)
• Model: GLM-4.7
• Tensor Parallelism: 8
• SGLang Version: 0.5.6.post1

Benchmark Methodology:

We use industry-standard benchmark configurations to ensure results are comparable across frameworks and hardware platforms.

5.1.1 Standard Test Scenarios

Three core scenarios reflect real-world usage patterns:

Scenario	Input Length	Output Length	Use Case
Chat	1K	1K	Most common conversational AI workload
Reasoning	1K	8K	Long-form generation, complex reasoning tasks
Summarization	8K	1K	Document summarization, RAG retrieval

5.1.2 Concurrency Levels

Test each scenario at three concurrency levels to capture the throughput vs. latency tradeoff (Pareto frontier):

• Low Concurrency: --max-concurrency 1 (Latency-optimized)
• Medium Concurrency: --max-concurrency 16 (Balanced)
• High Concurrency: --max-concurrency 100 (Throughput-optimized)

5.1.3 Number of Prompts

For each concurrency level, configure num_prompts to simulate realistic user loads:

• Quick Test: num_prompts = concurrency × 1 (minimal test)
• Recommended: num_prompts = concurrency × 5 (standard benchmark)
• Stable Measurements: num_prompts = concurrency × 10 (production-grade)

---

5.1.4 Benchmark Commands

Scenario 1: Chat (1K/1K) - Most Important

• Model Deployment

python -m sglang.launch_server \
  --model zai-org/GLM-4.7 \
  --tp 8

• Low Concurrency (Latency-Optimized)

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --num-prompts 10 \
  --max-concurrency 1 \
  --request-rate inf

• Medium Concurrency (Balanced)

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --num-prompts 80 \
  --max-concurrency 16 \
  --request-rate inf

• High Concurrency (Throughput-Optimized)

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --num-prompts 500 \
  --max-concurrency 100 \
  --request-rate inf

Scenario 2: Reasoning (1K/8K)

• Low Concurrency

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 8000 \
  --num-prompts 10 \
  --max-concurrency 1 \
  --request-rate inf

• Medium Concurrency

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 8000 \
  --num-prompts 80 \
  --max-concurrency 16 \
  --request-rate inf

• High Concurrency

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 8000 \
  --num-prompts 320 \
  --max-concurrency 64 \
  --request-rate inf

Scenario 3: Summarization (8K/1K)

• Low Concurrency

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1000 \
  --num-prompts 10 \
  --max-concurrency 1 \
  --request-rate inf

• Medium Concurrency

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1000 \
  --num-prompts 80 \
  --max-concurrency 16 \
  --request-rate inf

• High Concurrency

python -m sglang.bench_serving \
  --backend sglang \
  --model zai-org/GLM-4.7 \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1000 \
  --num-prompts 320 \
  --max-concurrency 64 \
  --request-rate inf

5.1.5 Understanding the Results

Key Metrics:

• Request Throughput (req/s): Number of requests processed per second
• Output Token Throughput (tok/s): Total tokens generated per second
• Mean TTFT (ms): Time to First Token - measures responsiveness
• Mean TPOT (ms): Time Per Output Token - measures generation speed
• Mean ITL (ms): Inter-Token Latency - measures streaming consistency

Why These Configurations Matter:

• 1K/1K (Chat): Represents the most common conversational AI workload. This is the highest priority scenario for most deployments.
• 1K/8K (Reasoning): Tests long-form generation capabilities crucial for complex reasoning, code generation, and detailed explanations.
• 8K/1K (Summarization): Evaluates performance with large context inputs, essential for RAG systems, document Q&A, and summarization tasks.
• Variable Concurrency: Captures the Pareto frontier - the optimal tradeoff between throughput and latency at different load levels. Low concurrency shows best-case latency, high concurrency shows maximum throughput.

Interpreting Results:

• Compare your results against baseline numbers for your hardware
• Higher throughput at same latency = better performance
• Lower TTFT = more responsive user experience
• Lower TPOT = faster generation speed

5.2 Accuracy Benchmark

Document model accuracy on standard benchmarks:

5.2.1 GSM8K Benchmark

• Benchmark Command

python -m sglang.test.few_shot_gsm8k \
  --num-questions 200 \
  --port 30000