Reflection’s Beam weights are available under Apache 2.0, giving engineers a text-only mixture-of-experts model with 501.3 billion total parameters, 23.4 billion active parameters per token and a one-million-token context window. The October 16 release is accompanied by day-zero vLLM support and an unusually detailed serving report—but its headline throughput numbers require careful interpretation.
The published model card lists BF16, FP8 and NVFP4 variants and identifies coding, reasoning and agentic tool use as primary applications. The weights release is distinct from Reflection’s October 5 preview: this development gives deployment teams artifacts they can inspect and serve, rather than only vendor benchmark claims.
The engineering story is the interaction between sparse computation, hybrid attention and specialized inference kernels. Activating approximately 23B parameters does not turn a 501B checkpoint into a 23B memory footprint. Deployment planning must account separately for weight storage, attention-cache memory, parallel communication and workload-specific decoding behavior.
Hybrid attention limits the full-context cache
The vLLM engineering report describes a 52-layer decoder with hidden size 5,120. Forty-three layers use a 512-token sliding window with rotary positional embeddings. Nine layers attend over the full context without positional encoding, or NoPE. Grouped-query attention uses 48 query heads and eight key/value heads.
Only those nine global-attention layers retain key/value entries for the entire sequence. The report estimates their cache at 18 KiB per token with FP8 storage, or about 18 GiB for a full million-token context before sharding. That figure is not the model’s complete GPU-memory requirement: weights, local-attention state and runtime allocations still matter.
Beam’s sparse layers route across 256 experts, selecting eight experts per token. Its first three layers instead use dense SwiGLU feed-forward networks. Unlike DeepSeek-V3, the described architecture has no shared expert. These implementation choices mean a superficially similar MoE model is not necessarily a drop-in performance proxy.
Kernel fusion explains the local speedup
vLLM reports that coding agents profiled its initial, largely PyTorch implementation and optimized kernels. On TP8, meaning tensor parallelism across eight GPUs, batch-one decode fell from 8.95 milliseconds to 3.30 milliseconds per step. An eight-sequence, 2,048-token prefill fell from 328 to 176 milliseconds.
The changes include fused normalization and residual operations, merged query/key/value and gate projections, fused cache writes and a specialized FP32 router kernel. The 2.71-times decode improvement compares against that initial implementation—not against every competing serving engine or model.
A separate prefill experiment on four GB300 GPUs reports 168.8K input tokens per second with attention data parallelism plus MegaMoE, versus 98.7K for the compared tensor-parallel setup. Inputs were 32K tokens and outputs one token, intentionally isolating prefill rather than conversational decoding.
Why 708 tokens per second is not a chat guarantee
With the DFlash2 speculative drafter, vLLM reports 708 output tokens per second for a single request on TP8. The test uses SPEED-Bench’s low-entropy category: approximately 1K-token prompts, mostly code completion and text sorting, greedy decoding and outputs up to 4K tokens. The authors explicitly warn that open-ended chat should have lower draft acceptance.
Adaptive verification is almost neutral at concurrency one, but raises aggregate output throughput from 15,012 to 16,632 tokens per second at concurrency 128—a reported 10.8% improvement. These are different operating points, not interchangeable measures of individual-user speed.
Similarly, the AgentX result of approximately 88K total tokens per second per GPU includes input, output and cached input. Its reported cache-hit rates exceed 91%. It must not be presented as 88K newly generated tokens per second.
The architecture discussion above is community commentary; the numerical specifications here come from the model card and serving report.
Evaluate the checkpoint, precision and harness together
Reflection’s model card reports SWE-bench Verified scores of 81.6% for BF16, 80.2% for FP8 and 80.7% for NVFP4. These are developer-reported evaluations, not independent guarantees. Different numerics do not produce an identical ranking on every benchmark, so deployment choices should use representative local tests.
Related video: Reflection Beam 1.0 overview. This is secondary commentary, not the source of the benchmark measurements.
For practitioners, the decision is not simply whether Beam is open. It is whether its memory requirements, serving topology, tool-call behavior and accuracy meet a specific workload. Measure prefill, uncached output throughput, tail latency and task completion separately before treating any headline figure as a production capacity estimate.
