Sub-2-BPW Structural Compression for Mixture-of-Experts (MoE) LLMs Compressing Gemma 4 26B-A4B from 9.92 GB to 4.75 GB (1.48 effective bits/param), enabling 100% In-VRAM GPU Execution on 6 GB laptop GPUs with 45.4 Tokens/Second throughput.
Model Focus: This 4.75 GB checkpoint is profiled and pruned specifically for Python programming, algorithm synthesis, data structures, and formal STEM reasoning. By excising 80 dormant multilingual and general conversational experts, the active 48-expert manifold preserves deductive reasoning and syntax generation while fitting entirely within consumer 6 GB VRAM.
(llama-cli downloads the 4.75 GB GGUF container directly from Hugging Face and allocates all 30 layers into GPU VRAM at 45.4 TPS).
