Qwen 3.6 27B is a BEAST

Tools 590 points 309 comments 1 month ago

I have a 5090 Laptop from work, 24GB VRAM. I have been testing every model that comes out, and I can confidently say I’ll be cancelling my cloud subscriptions. All my tool call and data science benchmarks that prove a model is reliably good for my use case, passed. It might not be the case for other professions, but for pyspark/python and data transformation debugging it’s basically perfect. Using llama.cpp, q4\_k\_m at q4\_0, still looking at options for optimising. Edit - I chose to go with IQ4\_XS at 200k q8\_0, I have not used speculative decoding yet, will get there when I get there. Specs: ASUS ROG Strix SCAR 18 RTX 5090 24GB 64GB DDR5 RAM

More from r/LocalLLaMA