Posted by u/copingmechanism
More quantization visualization types (repost)
Inspired by this post from u/VoidAlchemy a few months back: https://old.reddit.com/r/LocalLLaMA/comments/1opeu1w/visualizing\_quantization\_types/ Intrusive thoughts had me try to reproduce and extend the work to include more quantization types, with/without imatrix, and some PPL/KLD measurements to see what an "efficient" quantization looks like. MXFP4 really doesn't like to participate in this sort of experiment, I don't have much faith this is a very accurate representation of the quant but oh-well. The (vibe) code for this is here https://codeberg.org/mailhost/quant-jaunt along with a sample of summary output (from lenna.bmp) and some specifications that might help keep the vibes on track. \*reposted to respect Lenna's retirement \*\*Edit: Some more intrusive thoughts later, I have updated the 'quant-jaunt' repo to have (rough) support of the ik\_llama quants. It turns into 110 samples. Have also shifted to using ffmpeg to make a lossless video instead of a gif. https://v.redd.it/o1h6a4u5hikg1
More from r/LocalLLaMA
Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨
I feel personally attacked
Distillation when you do it. Training when we do it.