Sub-1-Bit Quantization: There's Plenty More Room at the Bottom
Alex Wissner-Gross reports that Samsung's NanoQuant model has already broken the 1-effective-bit-per-weight barrier using sparsity, quantization, and low-rank factorization. His extrapolation: sub-1-bit quantization goes mainstream within the next year. The endpoint, per Emad Mostaque, is 0.78 bits per weight — and at that point, models could be literally etched into silicon.
