HSTU Generative Recommenders: Deploy on GPU Cloud (2026)

Recommendation systems ran on CPUs for a decade because ranking a candidate list was a cheap, memory-bound lookup problem. That assumption broke when Meta published HSTU. Per-candidate compute jumped 6,500x in three years, from 40 MFLOP to 260 GFLOP, according to MLCommons' DLRMv3 benchmark. That's not a tuning problem you solve with a bigger CPU fleet. It's a GPU workload now, and most teams building recommendation systems haven't updated their infrastructure assumptions to match.

This post covers what HSTU generative recommenders actually are, the M-FALCON inference algorithm that makes them servable at scale, and the GPU sizing math for deploying one in production. If you're already comfortable with the GPU memory math for dense LLMs, see the GPU memory requirements guide for the general framework this post builds on. For picking hardware once you know your bottleneck, best GPU for AI inference in 2026 has current benchmarks across H100, H200, and B200.

Why Recommendation Systems Are Becoming a GPU Workload, Not a CPU One

The short answer: the core computation changed from a sparse lookup to a dense attention pass over long sequences, and dense attention only makes sense on GPUs. Here's how that shift happened and who's already running it in production.

From DLRM Ranking Cascades to Generative Recommenders

For most of the 2010s and early 2020s, production recommendation systems looked the same everywhere: retrieve a few thousand candidates cheaply, then rank them with a Deep Learning Recommendation Model (DLRM). DLRM towers combine sparse embedding lookups (one per feature, per item) with a handful of dense layers, and the sparse lookups dominate both memory footprint and compute time. That's a CPU-friendly shape. Big embedding tables, small dense compute.

HSTU (Hierarchical Sequential Transduction Units) throws that cascade out. Instead of scoring a fixed candidate set with feature crosses, HSTU treats a user's action history, clicks, watches, purchases, in order, as a token sequence and autoregressively predicts what comes next, the same recipe LLMs use for next-token prediction. Meta introduced the architecture in "Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations". The reframing matters because it replaces sparse feature crosses with dense multi-head self-attention over the user's full interaction sequence, and attention compute scales with sequence length, not table size.

The Compute Curve: 40 MFLOP to 260 GFLOP Per Candidate in Three Years

MLCommons added DLRMv3, a generative-recommendation benchmark, to MLPerf Inference to capture exactly this shift. The numbers are the clearest evidence that recommendation moved workload classes:

Source: MLCommons DLRMv3 inference benchmark. The embedding table growing 20x is itself a serving challenge, but it's the 6,500x jump in per-candidate compute that actually forces a GPU. A 20x-bigger table is still a memory-capacity problem you can solve with more CPU RAM or sharding. A 6,500x compute jump is a throughput problem, and CPUs lose that fight against parallel matrix multiply hardware every time.

Who's Already Running This in Production (Meta, Shopify, Kuaishou, Meituan)

This isn't a research paper waiting for adoption, it's already running at scale across four companies with public engineering writeups:

• Meta ran its scaling-law study for HSTU up to 1.5 trillion parameters (8,192-length sequences, 1,024 embedding dimension, 24 HSTU layers), and separately reports training jobs on 64 to 256 H100s (arXiv:2402.17152). Once deployed, the architecture won 12.4% of online A/B tests (measured via normalized entropy) across engagement and consumption metrics on a platform with billions of users, and beat baselines by up to 65.8% in offline NDCG (arXiv:2402.17152).
• Shopify built its production recommender on HSTU and reported it beat its previous stack in controlled testing. Shopify's own framing: "A recommender that is powerful but slow does not help buyers." After optimizing CUDA kernels for a 7.3x training-pipeline speedup, an August model version in online A/B testing delivered +0.94% relative Shop orders, +5% high-quality CTR, +0.71% conversion rate, and +4.8% relative lift in final served product recall at 2 (Shopify Engineering).
• Kuaishou launched OneRec, an end-to-end generative recommendation system that replaces pieces of its traditional retrieve-then-rank cascade, in mid-2025. OneRec now serves roughly 25% of Kuaishou's total app QPS (arXiv:2506.13695).
• Meituan built MTGRBoost, a distributed training system purpose-built to scale generative recommendation models, using dynamic hash tables and sequence balancing across GPU fleets (arXiv:2505.12663).

As ML researcher Yuan Meng put it: "the tide seems to have finally turned after Meta's HSTU delivered perhaps the biggest offline/online metric and serving efficiency gains in recent years" (Yuan Meng). Google, Alibaba, ByteDance, Tencent, and several other large platforms are reportedly evaluating or building similar generative recommendation pipelines.

HSTU's M-FALCON Inference Algorithm: What It Actually Requires

A recommender that scores candidates one at a time with full attention doesn't survive contact with production latency budgets. M-FALCON is the piece that makes HSTU servable, and understanding it tells you what to actually provision.

HSTU in Plain Terms: Hierarchical Sequential Transduction Units

HSTU is a transformer variant tuned for recommendation's specific data shape: extremely high cardinality (billions of items, not a fixed vocabulary), non-stationary streaming data (new items and users constantly), and long, noisy interaction sequences. It processes user history as a sequence, the same way a language model processes tokens, and uses self-attention to learn which past actions predict the next one. The published benchmark on 8,192-length sequences showed HSTU running 5.3x to 15.2x faster than FlashAttention2-based transformers at the same length (arXiv:2402.17152), largely because the architecture is designed around recommendation-shaped sparsity rather than adapted from a general-purpose language model.

M-FALCON: Microbatched Attention for Scoring Multiple Candidates at Once

Naive ranking inference with a sequence transducer means re-running the full encoder for every candidate you want to score, which is far too slow for a real-time ranking pass with thousands of candidates. M-FALCON, short for Microbatched-Fast Attention Leveraging Cacheable OperatioNs, batches bm candidates into a single forward pass and modifies the attention mask and relative attention bias so the encoder computation is shared, cutting cross-attention cost from O(bmn2d) to approximately O(n2d) once bm is small relative to sequence length n.

The result, per Meta's published numbers (arXiv:2402.17152): 1.5x throughput improvement at 1,024 candidates and 2.48x at 16,384 candidates, all under the same inference budget, while serving a model 285x more computationally complex than a traditional DLRM ranker. That "cacheable operations" framing is the same underlying idea as KV cache reuse in LLM serving, avoid recomputing attention state that doesn't change across a batch. If you're already running LLM inference, the KV cache optimization guide covers the general version of this trick, and the microbatching-to-amortize-cost approach itself is conceptually identical to the batching strategies in the batch LLM inference guide.

GPU Memory Math: Embedding Tables vs Dense HSTU Weights

Sizing a GPU deployment for HSTU means separating two very different memory problems, and conflating them is the single most common mistake teams make moving off DLRM.

Embedding tables hold one vector per item (and often per user, per feature). MLCommons' DLRMv3 reference benchmark uses a 1TB float16 table (1 billion items at 512 dimensions), fully sharded across 8 GPUs. That's the same "hold everything, compute on a fraction" problem MoE models create with expert weights, worth reading the MoE inference optimization guide if you've already solved this for a mixture-of-experts LLM, since the sharding and lookup patterns transfer directly.

Dense HSTU weights are the actual transformer, the part doing sequence-to-sequence attention. This is comparatively tiny next to a trillion-item embedding table, but it's where the GPU-bound compute lives.

The DLRMv3 benchmark's own query time breakdown at 8 GPUs makes the split concrete: GPU-side dense HSTU compute takes 25% of total query time, GPU-side sparse operations take 5%, and CPU-side embedding lookup against the 1TB table takes 43% (the rest is batching and data movement overhead). Once data reaches the GPU, dense attention compute is 5x more expensive than the GPU's own sparse lookup step, confirming that HSTU inference is attention-bound, not memory-lookup-bound, on the GPU side. The CPU-side embedding lookup remains the single largest line item overall, which is why production HSTU serving is usually a two-stage pipeline rather than a single monolithic pass.

Here's the math worked through for a case smaller than Meta's, since most teams evaluating this aren't at trillion-item scale. Take a marketplace with 50 million active items and a 256-dimensional fp16 embedding table: that's 50,000,000 x 256 x 2 bytes, about 25.6GB, comfortably inside a single H100 80GB alongside the dense HSTU weights. A dense stack at Meta's reported 1,024-dim, 24-layer HSTU configuration runs a few billion parameters, small next to the embedding table and cheap to keep resident in the same GPU's memory. At that scale you don't need the CPU-side lookup path or cross-GPU sharding at all, one H100 SXM5 handles embedding lookup and the dense M-FALCON pass in the same process. The two-stage split only becomes necessary once your item catalog or user base pushes the table past what a single GPU (or your sharding budget) can hold, which is closer to the 1TB territory the DLRMv3 reference benchmark is built around.

Serving a Generative Recommender in Production on GPU Cloud

Provisioning for HSTU means matching hardware to each stage separately, not buying one GPU tier and hoping it covers both the embedding lookup and the dense attention pass equally well.

Two-Stage Serving: Embedding Lookup vs Dense Attention Compute

Split your serving pipeline the way the DLRMv3 benchmark data suggests: an embedding lookup stage against your (likely CPU-resident or GPU-sharded, depending on table size) item table, feeding a dense HSTU/M-FALCON attention stage on GPU. This mirrors the prefill/decode split that's become standard in LLM serving, two stages with very different resource profiles that benefit from being scaled independently rather than colocated on identical hardware. If you've set up disaggregated serving for LLMs before, the prefill-decode disaggregation guide covers the same separation-of-concerns pattern in more depth.

Picking GPUs for HSTU: HBM Headroom for Long User Sequences

As of 10 Jul 2026, on-demand per-GPU pricing on Spheron looks like this:

If your embedding tables are the constraint, whether because you're sharding a large item catalog across GPU memory or your users generate very long interaction histories, H200's 141GB or B200's 192GB give you more room before you're forced into additional sharding complexity. If your dense HSTU pass is the bottleneck (the more common case per the DLRMv3 breakdown above), H100 SXM5's NVLink interconnect matters more than raw VRAM, since M-FALCON's microbatching benefits from fast inter-GPU communication when scoring many candidates per user request. See Spheron's H100 GPU rental for current SXM5 and PCIe availability.

Pricing fluctuates based on GPU availability. The prices above are based on 10 Jul 2026 and may have changed. Check current GPU pricing for live rates.

Scaling Training and Serving Across Multiple GPUs

Training an HSTU generative recommender at any real scale means multi-GPU from day one. Meta reports training jobs on 64 to 256 H100s (arXiv:2402.17152). Meituan's MTGRBoost, purpose-built for scaling generative recommendation training, delivers 1.6x to 2.4x throughput over TorchRec and holds 62.75%-78.5% of ideal linear speedup at 128 GPUs (the range depends on embedding dimension and model complexity), using dynamic hash tables and sequence-length balancing to keep every GPU fed evenly despite wildly variable user sequence lengths (arXiv:2505.12663). Kuaishou's OneRec training cluster runs 90 servers with 8 flagship GPUs each, 400Gbps NVLink intra-node and 400Gbps RDMA inter-node, reaching 23.7% training MFU, a 5.2x improvement over its prior ranking model (arXiv:2506.13695).

On the inference side, the DLRMv3 benchmark measured a 3.3x throughput improvement scaling from 1 to 8 H100 GPUs: 300 QPS at 78ms average (130ms P99) on a single GPU versus 1,000 QPS at 60ms average (82ms P99) once scaled to 8 GPUs, both at batch size 10 (MLCommons). Kuaishou serves OneRec inference on a leaner setup by comparison, 4 NVIDIA L20 GPUs plus 2 CPUs per server over PCIe, inside a 200Gb RDMA datacenter fabric, hitting 28.8% inference MFU (arXiv:2506.13695). The takeaway: your serving GPU choice doesn't need to match your training GPU choice. Meta and Kuaishou both train on flagship hardware and serve on a mix tuned for cost and latency, not raw FLOPs.

If you're running an HSTU recommender as one workload among several ranking or retrieval services sharing GPU capacity, the multi-tenant LLM serving guide covers the isolation and quota patterns you'll want, even though it's written for LLM APIs rather than recommenders specifically, the noisy-neighbor and per-workload metering concerns are the same.

A Basic GPU Cloud Deployment Checklist for HSTU-Style Models

1. Separate your two memory problems before picking hardware. Size your embedding table (items x embedding dim x bytes/param, plus sharding overhead) independently from your dense HSTU weight size. Don't let a huge table push you onto a GPU tier your dense compute doesn't actually need, and vice versa.
2. Benchmark your own sequence length. HSTU's compute scales with user history length, not just parameter count. A platform with 500-item average sequences has a very different GPU profile than one with 8,192-length histories like Meta's production model.
3. Provision multi-GPU with NVLink for microbatched serving. M-FALCON's throughput gains depend on efficient batching across candidates; PCIe-only setups will bottleneck on inter-GPU communication faster than NVLink SXM configurations. Spheron's H200 GPU instances ship with NVLink out of the box.
4. Use spot instances for training, on-demand for serving latency. Training jobs (Meta's 64-256 H100 runs, Meituan's 100+ GPU MTGRBoost jobs) tolerate interruption far better than a live ranking API. Reserve on-demand capacity for the serving tier where P99 latency actually matters.
5. Instrument the CPU/GPU split. Per the DLRMv3 breakdown, CPU-side embedding lookup can be your largest single latency contributor even when your GPUs are doing the more expensive per-FLOP work. Monitor both stages separately rather than treating end-to-end latency as one number.
6. Check the docs before your first deploy. Spheron's documentation covers GPU provisioning, NVLink verification, and networking configuration for multi-GPU setups.

If your current recommendation stack is still a CPU-bound DLRM cascade, the GPU sizing questions here (embedding table memory vs dense compute, NVLink for microbatched inference) apply the moment you move any part of it to HSTU-style sequence modeling. Sanity-check the underlying VRAM math before you provision anything.

Generative recommenders push recommendation ranking onto the same GPU-bound attention compute that LLM serving already runs on. Spheron's H100 SXM5 and H200 instances give you the NVLink bandwidth M-FALCON's microbatched serving needs, with per-minute billing so training and serving capacity scale independently.

Spheron B200 instances →