2026-11-04 · Fonte：deep-www.marktechpost.com-p10
Alibaba's Qwen team ships an open-weight multimodal mixture-of-experts model framed as an early preview of the Qwen4 architecture, pairing a 125B backbone with N-gram embeddings and multi-token prediction.
Alibaba's Qwen team has released Qwen3.8-Flash-Next, an open-weight, multimodal mixture-of-experts model presented as an early preview of the architecture that will underpin the Qwen4 generation. The release concentrates per-token compute in a small active footprint while keeping a much larger sparse parameter base available, a structure that distinguishes it from dense models of comparable total size.
The parameter accounting is unusual. Qwen3.8-Flash-Next totals 180B parameters, of which only 6B are active per token. The rest is distributed across a 125B backbone, a 51B N-gram embedding table, and a 4B multi-token prediction module. The embedding table alone holds more than a quarter of all parameters, making the N-gram component one of the defining features of the design.
Four architectural changes define the model: a hybrid design combining Gated DeltaNet with Qwen Sparse Attention, a Gated Residual mechanism, N-gram Embedding, and the Muon optimizer. Together these components form the substance of the Qwen4 architecture preview, giving developers and infrastructure planners an early, runnable reference point for the direction Alibaba's model stack is taking.
On the efficiency side, the most consequential claim is a reported training cost equal to one-ninth of Qwen3.7-Plus. That comparison targets the economics that matter most to organizations training or fine-tuning at scale, and benchmark results released with the model will be read against it.
Deployment is the other headline for compute buyers. The FP8 checkpoint for self-hosting weighs 172.78 GiB, a footprint that immediately narrows the field of GPU configurations capable of serving the model locally. For teams evaluating open-weight multimodal alternatives, that memory requirement, combined with the sparse 6B active-parameter profile, frames the deployment calculus.
Tags: Alibaba · Qwen3.8-Flash-Next · Mixture-of-Experts · Open-Weight Models · Multimodal AI · Muon Optimizer
萬安算交所 AIXX · AIXX · Exchange verificado · Ir ao mercado