A developer has spent six weeks running Qwen3.8-Flash-Next, a 125-billion-parameter MoE model, as the only production LLM on a desktop NVIDIA DGX Spark with 128 GB of unified memory, and has published the image, vLLM configuration and scripts needed to reproduce it on GitHub. The setup shares the machine with an embeddings model and Whisper, reaches about 55 tokens per second for a single request and, according to its author, the key is not the model but cutting the bits that have to be read at every decoding step.

Qwen3.8-Flash-Next on a DGX Spark in 20 seconds

Qwen3.8-Flash-Next runs in production on a DGX Spark with 128 GB of unified memory.
The author measures about 55 tokens/s per request and about 145 aggregate with 10 users.
NVFP4, FP8 and speculative decoding make a 135 GB model fit.
Everything is on GitHub and works on any GB10 machine.

The project, published by GitHub user aseth0 in the aseth0/qwen3.8-flash-next-gb10 repository under the Apache 2.0 license, is useful for any administrator weighing whether to serve large models locally. It shows with numbers which optimizations work on NVIDIA’s GB10 chip, which do not, and how much memory is left for other services.

The model and the machine

Qwen3.8-Flash-Next is the mixture-of-experts (MoE) model Alibaba released in August 2026. According to the official vLLM recipe and published specifications, it has 125 billion parameters, of which only about 6 billion are active per token, plus a 51-billion-parameter n-gram embedding table. It combines Gated DeltaNet layers, a form of linear attention with recurrent state, with sparse attention layers, and includes a multi-token prediction (MTP) module used for speculative decoding. For the FP8 version, the vLLM recipe recommends one or more GB300 systems or servers with eight H100 or H200 GPUs.

The DGX Spark, by contrast, is a desktop machine with the GB10 superchip and 128 GB of memory shared between CPU and GPU. On it, the author serves the model with vLLM v0.30 behind an OpenAI-compatible endpoint, leaving room for bge-m3, which generates embeddings, and Whisper, which transcribes audio. According to the author, the whole stack uses about 105 GiB.

How a 135 GB model fits in 128 GB

In NVFP4, NVIDIA’s 4-bit format, the model weighs about 135 GB, more than the available memory. The author says it took four measured changes:

NVFP4 n-gram table served via mmap: the PLE table, about 27 GB once quantized, is memory-mapped instead of being fully loaded.
NVFP4 draft head: the MTP-3 speculator was reading 1.27 GB of the output layer in BF16 three times per step. Quantizing it added 13% to 25% speed.
FP8 dense linear layers: the 145 layers moved to FP8, adding 16% speed on code with no HumanEval+ loss.
vLLM v0.30 with a patch for the recurrent block state: according to the author, it adds 5% under concurrent load.

Just as useful is the list of what was tested and rejected. Going below 10 active experts broke quality without any speed gain. Storing the KV cache in FP8 increased capacity but cut decode speed by 11% without allowing more concurrent users. MTP-4 tied with MTP-3 while using more memory. The README adds a practical warning: at temperature 0, the model loops.

Measured performance and differences with the repository

In a LinkedIn post, the author reports these figures: 96.3% on HumanEval+, compared with 92.7% for Qwen3.8-27B in FP8; 0.865 on τ²-bench across 74 long agentic tasks with tools, compared with 0.770 for the 27B model; about 55 tokens per second on code and 52 on prose for a single request, with a ceiling of about 145 tokens per second aggregate at 10 users; and about 310,000 tokens of KV cache, with prefix caching that starts hitting from about 1,600 tokens.

Note that the repository README, with measurements dated between 26 September and 5 October 2026, lists some different numbers: 95.7% on base HumanEval+ and 91.5% on the plus variant, 0.784 on τ²-bench (58 of 74 tasks) and a 239,238-token KV cache, with about 121 GiB of unified memory required. The speed figures do match, between 50.6 and 55.9 tokens per second for a single request and about 145 aggregate. The LinkedIn figures may reflect later tuning, but anyone replicating the setup should check against their own run.

For administrators, the practical requirements are clear: a GB10 machine such as the DGX Spark or ASUS Ascent GX10, about 235 GB of disk for weights and image, and a vLLM image pinned by digest to avoid surprises with new releases. With a discrete GPU, the author estimates about 86 GiB of VRAM and 100 GiB of RAM, although the configuration is only validated on GB10. The author’s conclusion sums up the technical lesson: in local inference in 2026, every gigabyte of BF16 read during decoding is paid for in tokens per second.

Frequently asked questions

Can Qwen3.8-Flash-Next run on a DGX Spark?

Yes. A developer serves it in production with vLLM v0.30 on a 128 GB DGX Spark, combining NVFP4, FP8 and speculative decoding, and has published the configuration on GitHub.

How fast is it?

According to the author’s measurements, about 55 tokens per second for a single request and up to about 145 tokens per second aggregate with 10 concurrent users.