Deploy Step-3.7-Flash on GPU Cloud: Self-Host StepFun's 198B MoE Agentic Vision Model with vLLM (2026 Setup Guide)

Step-3.7-Flash is 198B total parameters with 11B active per forward pass. The GPU math here is the same as for any large MoE: all 198B weight bytes must sit in VRAM, but inference throughput behaves like an 11B dense model. That gap between storage cost and compute cost is what makes this model practical to self-host.

The vision encoder adds a fixed 1.8B ViT component on top of the 196B language backbone. It handles native image and document input, which positions Step-3.7-Flash as an agentic model. StepFun designed it for concurrent agent workflows where multiple workers need fast, short generation cycles with periodic vision inputs.

This guide covers VRAM sizing for the BF16, FP8, and NVFP4 checkpoints, vLLM deployment on Spheron, agentic serving configuration, and the full cost breakdown.

What Is Step-3.7-Flash

Step-3.7-Flash is StepFun's mid-2026 vision-language model. It targets agentic use cases where low per-call latency and native visual understanding matter more than single-call benchmark scores.

The hybrid sliding-window (512-token window) plus global attention design is what enables 256K context without the quadratic KV growth of full self-attention. Most tokens attend locally via the sliding window; global attention layers at a 3:1 ratio handle long-range dependencies. MTP-3 (3-way Multi-Token Prediction) is a separate speculative decoding feature that predicts multiple output tokens in parallel and is not the attention mechanism. This is what makes --disable-cascade-attn a required flag for this model in vLLM; the hybrid SWA/GA schedule is not compatible with cascade attention.

The 11B active parameter count means each individual agentic call is cheap in compute terms. That matters for multi-step loops where the model is making rapid decisions rather than generating long outputs. It does not affect VRAM requirements: all 198B weight bytes must still load into GPU memory before any inference happens.

For broader context on how MoE models work at inference time, see the MoE inference optimization guide. For the VLM-specific architecture patterns, including ViT encoder sizing and visual token inflation, see the vision language models deployment guide.

GPU VRAM Requirements

The critical distinction for Step-3.7-Flash is the same as for every MoE VLM: "11B active" means per-forward-pass compute. VRAM footprint is determined by total parameter count (198B), not active count.

The 1.8B ViT encoder adds a fixed weight overhead on top of the language backbone. At FP16 that is roughly 3.6 GB; at FP8 it is about 1.8 GB. This overhead is already included in the totals below, but it is worth stating explicitly because text-only MoE models of similar size do not carry it.

For GPU selection, always add a 15% framework overhead on top of the weight figures before comparing against VRAM capacity. BF16 weights at 396 GB become roughly 455 GB with overhead, beyond a 4x H100 SXM5 node (4 × 80 GB = 320 GB) and requiring 4x H200 SXM5 at minimum.

Here are the recommended configurations on Spheron with current pricing:

What does not work:

• 1x H100 SXM5 (80 GB): Too small even for NVFP4. The NVFP4 weight footprint is approximately 99 GB, which already exceeds 80 GB before any framework overhead or KV cache.
• H100/H200 + NVFP4: The official NVFP4 checkpoint targets Blackwell hardware (B200/B300). Hopper-generation GPUs (H100, H200) are not supported for NVFP4. Use FP8 on Hopper.
• 1x B200 SXM6 + FP8: FP8 weights are roughly 198 GB and the B200 SXM6 has 192 GB VRAM. Weights alone exceed VRAM capacity before any framework overhead or KV cache. Use NVFP4 for single-GPU B200 deployment.
• A100 80G: Ampere lacks the FP8 Tensor Engine required for the official FP8 checkpoint. NVFP4 serving on Ampere has not been validated by StepFun. Skip A100 for Step-3.7-Flash.

For the 2x H200 FP8 config, the H200 SXM5 on Spheron is the recommended entry point if you want 128K context in production. For the 4x H100 FP8 option, H100 SXM5 rental on Spheron provides the same FP8 quality at lower per-GPU cost.

Pricing fluctuates based on GPU availability. The prices above are based on 22 Jun 2026 and may have changed. Check current GPU pricing → for live rates.

Deploy Step-3.7-Flash with vLLM on Spheron

Step 1: Choose GPU and quantization

Pick the configuration that fits your VRAM and budget:

• 2x H200 SXM5 + FP8: Recommended production default. 128K context, ~84 GB for KV cache, NVLink interconnect for efficient tensor parallelism.
• 4x H100 SXM5 + FP8: Same FP8 quality at lower per-GPU cost. More KV headroom than the 2x H200 setup for the same context window. H100 spot pricing is roughly 8% cheaper than on-demand right now.
• 1x B200 SXM6 + NVFP4: Best single-GPU option (Blackwell only). NVFP4 weights (~99 GB) fit comfortably in 192 GB with about 93 GB for KV cache and 128K context. NVFP4 is not supported on Hopper (H100, H200).

Step 2: Provision a Spheron GPU instance

Log in at app.spheron.ai and navigate to GPU Cloud. Select your GPU tier and deploy with the PyTorch 2.5 / CUDA 12.4 base image. Attach at least 250 GB of persistent storage for model weights. Step-3.7-Flash FP8 weights are roughly 198 GB, so 250+ GB storage prevents re-downloading on instance restarts.

See docs.spheron.ai for SSH setup and instance configuration guidance.

Step 3: Install vLLM and download weights

pip install 'vllm>=0.9.0'
pip install huggingface_hub hf_transfer

export HF_TOKEN=your_token_here
export HF_HUB_ENABLE_HF_TRANSFER=1

# Download the pre-quantized FP8 checkpoint (recommended for H100/H200 deployments)
huggingface-cli download stepfun-ai/Step-3.7-Flash-FP8

# Or for single-GPU B200 SXM6 NVFP4 deployment (Blackwell only)
huggingface-cli download stepfun-ai/Step-3.7-Flash-NVFP4

If vLLM does not natively recognize the model class, add --trust-remote-code to the serve command. For models released after the last stable vLLM build, check docs.vllm.ai for supported model status.

Step 4: Launch vLLM with hybrid attention configuration

2x H200 SXM5, FP8 (recommended production):

vllm serve stepfun-ai/Step-3.7-Flash-FP8 \
  --tensor-parallel-size 2 \
  --enable-expert-parallel \
  --max-model-len 131072 \
  --kv-cache-dtype fp8_e5m2 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --disable-cascade-attn \
  --tool-call-parser step3p5 \
  --reasoning-parser step3p5 \
  --enable-auto-tool-choice \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
  --host 0.0.0.0 \
  --port 8000

1x B200 SXM6, NVFP4 (single-GPU option, Blackwell only):

vllm serve stepfun-ai/Step-3.7-Flash-NVFP4 \
  --quantization modelopt \
  --max-model-len 131072 \
  --kv-cache-dtype fp8_e5m2 \
  --gpu-memory-utilization 0.92 \
  --enable-chunked-prefill \
  --disable-cascade-attn \
  --host 0.0.0.0 \
  --port 8000

4x H100 SXM5, FP8 (budget option):

vllm serve stepfun-ai/Step-3.7-Flash-FP8 \
  --tensor-parallel-size 4 \
  --enable-expert-parallel \
  --max-model-len 131072 \
  --kv-cache-dtype fp8_e5m2 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --disable-cascade-attn \
  --host 0.0.0.0 \
  --port 8000

Why --disable-cascade-attn: Step-3.7-Flash's hybrid sliding-window (512-token window) plus global attention at a 3:1 ratio is not compatible with cascade attention. This flag is required for the model to run correctly. --enable-chunked-prefill is also included as a general improvement for long-context prefilling, breaking long input sequences into fixed-size chunks to prevent KV cache spikes when agentic loops pass large documents.

Why the agentic flags (recommended command only): --tool-call-parser step3p5 and --reasoning-parser step3p5 enable correct parsing of tool calls and reasoning outputs. The step3p5 identifier is vLLM's registered parser name for the full Step 3.x model family. Despite the "3.5" in the name, vLLM intentionally reuses this parser for both Step-3.5-Flash and Step-3.7-Flash (confirmed in the official vLLM recipes documentation for Step-3.7-Flash). --enable-auto-tool-choice allows the model to select tools automatically. --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' activates MTP-3 (3-way Multi-Token Prediction) for speculative decoding, which can meaningfully improve throughput on short agentic calls.

Note on NVFP4 hardware: The official StepFun NVFP4 checkpoint targets Blackwell hardware (B200/B300). NVFP4 is not supported on Hopper-generation GPUs (H100, H200). For single-GPU NVFP4 deployment, use B200 SXM6.

Step 5: Test the endpoint

Text completion (baseline check):

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "stepfun-ai/Step-3.7-Flash-FP8",
    "messages": [
      {
        "role": "user",
        "content": "Summarize the key differences between FP8 and NVFP4 quantization for LLM inference."
      }
    ],
    "max_tokens": 256,
    "temperature": 0.2
  }'

Multimodal vision request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "stepfun-ai/Step-3.7-Flash-FP8",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Describe what you see in this image and extract any visible text."
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/240px-PNG_transparency_demonstration_1.png"
            }
          }
        ]
      }
    ],
    "max_tokens": 256
  }'

A correct multimodal response returns a choices[0].message.content string describing the image. This confirms the ViT encoder is loaded and routing correctly through the language backbone.

Step 6: Cloud-init startup script

For reproducible instance bootstrapping, use this startup script in your Spheron deployment configuration:

#!/bin/bash
set -e

echo "--- Setting Up Environment ---"

sudo apt-get update -y
sudo apt-get install -y python3-venv

sudo python3 -m venv /opt/step_flash_venv
source /opt/step_flash_venv/bin/activate

pip install --upgrade pip
pip install 'vllm>=0.9.0' huggingface_hub hf_transfer

echo "--- Downloading Step-3.7-Flash FP8 weights ---"

export HF_TOKEN=your_hf_token_here
export HF_HUB_ENABLE_HF_TRANSFER=1

huggingface-cli download stepfun-ai/Step-3.7-Flash-FP8

echo "--- Launching vLLM Server ---"

nohup vllm serve stepfun-ai/Step-3.7-Flash-FP8 \
    --tensor-parallel-size 2 \
    --enable-expert-parallel \
    --max-model-len 131072 \
    --kv-cache-dtype fp8_e5m2 \
    --gpu-memory-utilization 0.90 \
    --enable-chunked-prefill \
    --disable-cascade-attn \
    --host 0.0.0.0 \
    --port 8000 \
    --api-key "your_api_key_here" > /var/log/vllm.log 2>&1 &

echo "--- Waiting for server to initialize (ETA 15-25 minutes for FP8 weight download) ---"

for i in {1..3600}; do
  if curl -sf "http://localhost:8000/v1/models" > /dev/null; then
    echo "vLLM server is ready!"
    break
  fi
  if [ $((i % 30)) -eq 0 ]; then
    echo "Still waiting... ($i seconds elapsed)"
  fi
  sleep 1
done

curl -sf http://localhost:8000/health > /dev/null || { echo 'ERROR: vLLM failed to start'; exit 1; }

Agentic Workload Patterns

Step-3.7-Flash's design targets three agentic use cases specifically.

Concurrent coding agents

Multiple agent workers each making rapid short calls. The 11B active parameter count means per-call compute is close to an 11B dense model, even though all 198B bytes sit in VRAM. At 400 tok/s with short 50-200 token outputs, you can serve many concurrent agent workers from a single GPU node.

Tune --max-num-seqs based on your target concurrency. Start at 16 and increase while monitoring GPU memory utilization. For coding agents that submit patches in parallel, 32-64 concurrent sequences on a 2x H200 FP8 setup is a reasonable ceiling before KV cache starts to saturate.

Multi-step search and document parsing loops

The 256K context window lets agentic loops ingest large documents without chunking. Each image passed to the ViT encoder produces visual tokens that occupy KV cache space. For loops that pass a new screenshot or document scan on each iteration, KV cache fills faster than in text-only deployments at the same max-model-len.

Reduce --max-num-seqs compared to text-only serving at the same VRAM budget. The visual token inflation formula from the VLM deployment guide referenced above applies directly: each 1024px image adds roughly 1024 visual tokens to the per-request KV budget.

For multi-step workflows that need durable retry logic across agent calls, see the AI agent workflow orchestration guide for how Temporal and Inngest handle failure recovery without re-running expensive inference steps.

Vision-first agentic perception

Screenshots, UI parsing, chart understanding, and document OCR as primary inputs in an agentic loop. Step-3.7-Flash's native 1.8B ViT encoder handles this without a separate vision model. For the agentic RAG guide use case where Step-3.7-Flash processes retrieved document images alongside text chunks, this eliminates the need to run a separate document parsing model alongside the LLM.

Cost per vision request comparison: at $3.70/hr for a single B200 SXM6 NVFP4 serving at 300 tok/s with 200-token outputs, each vision request costs approximately $0.0007. GPT-4o Vision API charges per image token in addition to text tokens; at typical screenshot resolution, API cost per request runs $0.05-0.15 depending on image size. Self-hosting Step-3.7-Flash becomes cost-positive at moderate agentic request volumes.

See the Spheron docs for SSH setup and multi-instance configuration guidance.

Throughput, Cost Per Million Tokens, and Benchmarks

Throughput figures for Step-3.7-Flash are not yet independently benchmarked across all configurations. The estimates below are derived from the reported ~400 tok/s figure and comparable 198B MoE architectures. Validate on your Spheron instance before making capacity decisions.

Cost per million token formula: ($/hr / 3600) / (tok_s / 1_000_000)

The $/M tokens estimates assume the midpoint of the throughput range. Vision requests with image inputs will reduce effective throughput because ViT encoding adds latency before the first text token is generated.

For reference: GPT-4o Vision API pricing is $5-15/M tokens depending on image size and output length. At 2x H200 FP8 on-demand pricing, Step-3.7-Flash runs at roughly $6/M tokens estimated before volume optimizations like spot pricing and batching.

Pricing fluctuates based on GPU availability. The prices above are based on 22 Jun 2026 and may have changed. Check current GPU pricing → for live rates.

Spot vs On-Demand for Agentic Workloads

Step-3.7-Flash's 11B active parameter design creates short generation windows per agentic call, typically 50-300 output tokens per step. When a spot instance is reclaimed mid-generation, only the current call's output is lost, not the full agent session. An agent controller with retry logic (re-submitting the interrupted call) recovers in seconds rather than minutes.

This makes Step-3.7-Flash a good candidate for spot pricing on development workloads and batch agentic jobs. For a 4x H100 SXM5 cluster running FP8:

For H100 SXM5, spot at $2.91/hr is roughly 8% cheaper than on-demand at $3.17/hr, making spot worthwhile for workloads that can tolerate brief interruptions. For H200 SXM5, spot at $3.31/hr vs on-demand at $4.54/hr is a roughly 27% savings, making spot the right choice for development and batch inference jobs where brief interruptions are acceptable.

Production Checklist

Before routing real agent traffic to Step-3.7-Flash:

• Enable streaming. Agent frameworks expect stream: true responses. Without streaming, the agent controller waits for the full completion before acting on output, which breaks the responsiveness of multi-step loops.
• Cap concurrent sequences. Set --max-num-seqs 16 or lower for vision-heavy agentic workloads. Each image in a request inflates KV cache significantly; more concurrent sequences multiplies this effect.
• Mount persistent storage for weights. FP8 weights are roughly 198 GB. Re-downloading on every instance restart adds 20-30 minutes of downtime. Mount a persistent volume at the HuggingFace cache path.
• Monitor KV cache utilization. Enable Prometheus scraping with --enable-metrics and alert on vllm:gpu_cache_usage_perc above 85%. Vision requests fill the KV cache faster than text-only requests at the same sequence length.
• Pin your vLLM version. Step-3.7-Flash was released in late May 2026. If native model class support is not yet in the stable release, pin the version that works and document it in your startup script.
• Use the correct repo slugs. The confirmed HuggingFace slugs are stepfun-ai/Step-3.7-Flash-FP8 and stepfun-ai/Step-3.7-Flash-NVFP4 for the pre-quantized checkpoints, and stepfun-ai/Step-3.7-Flash for the BF16 base. Verify at huggingface.co/stepfun-ai before deploying.
• License review. Confirm the license terms for Step-3.7-Flash at the official HuggingFace model page before commercial production deployment.

Step-3.7-Flash's 11B active-parameter design makes it one of the more cost-efficient ways to run a 256K-context vision-language model for agentic pipelines. FP8 on a 2x H200 or 4x H100 cluster covers most production footprints, and a single B200 SXM6 with NVFP4 is the recommended path for single-GPU development and lower-concurrency workloads.

Get started on Spheron →