Qwen3.8 Flash Next llama.cpp config tuning
r/LocalLLaMA
September 12, 2026 at 04:18 AM
๐ Hola all. Do you guys mind sharing your LLama.cpp config and system setup details for Qwen3.8 Flash Next? Model's quite big and tryining many combinations of llama.cpp options takes lots of time, so looking for other people setup details. I've attached my current config at the bottom, so if anyone sees something that could be improved please shout. My current best result: - PP within 130...200 tps (limited by cpu?) - TG within 14..22 tps (~15tps on average) Hardware: - Dual RTX 3090 (48GB VR...
Hola all.
Do you guys mind sharing your LLama.cpp config and system setup details for Qwen3.8 Flash Next?
Model's quite big and tryining many combinations of llama.cpp options takes lots of time, so looking for other people setup details. I've attached my current config at the bottom, so if anyone sees something that could be improved please shout.
My current best result:
- PP within 130...200 tps (limited by cpu?)
- TG within 14..22 tps (~15tps on average)
Hardware:
- Dual RTX 3090 (48GB VRAM)
- 128GB DDR4
- Some old Xeon 40 core
- Proxmox VM, pcie passthrough, numa binding to a single phys cpu
Llama.cpp config:
llama-server --port ${PORT} --model /nvme/gguf/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf --mmproj /nvme/gguf/mmproj-Qwen3.8-Flash-Next-F16.gguf --load-mode none --lazy-mode off --parallel 1 --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --fit off --temp 1.0 --min-p 0.0 --top-p 0.95 --top-k 20 --presence-penalty 0.0 --repeat-penalty 1.0 --batch-size 2048 --ubatch-size 512 --split-mode layer -ts 26,10 -ngl 99 -ncmoe 26 --no-mmproj-offload --override-tensor per_layer_token_embd=CPU --chat-template-kwargs '{"reasoning_effort":"xhigh"}' ngl, ncmoe, ts - manually adjusted to fit the model without crashing
[link] [comments]
Read the full article at
r/LocalLLaMA
More in Models & Releases
Models & Releases
NVIDIA Unveils RTX PRO 5500 "Blackwell" Workstation GPU with 84 GB GDDR7 Memory
submitted by /u/Lumpy_Phase_9539 [link] [comments]
What happened to AI browsers?
AI browsers were supposedly going to kill Chrome/Safari, and help people quickly buy stuff, book tickets etc. at reasonable prices while avoiding ads. That was such a great sales pitch, but it has been just radio silence from then on. Ar...
[Bi-Weekly Megathread] Project Showcase
Do you have something you'd like to share with the r/LocalLLaMA community. This is the place for it! Recommendation on presentation: Please share plain english description of what your project does and why people should care about it Ho...