Posted by u/_camera_up
My company just handed me a 2x H200 (282GB VRAM) rig. Help me pick the "Intelligence" ceiling.
My workplace just got a server equipped with 2x Nvidia H200 GPUs (141GB HBM3e each). I've been asked to test LLMs on it since they know "I do that at home". While I have experience with smaller local setups, 282GB of VRAM is a different beast entirely. I want to suggest something more "interesting" and powerful than just the standard gpt oss or something. Im interested in raw "intelligence" over ultra high speeds. So what models / quants would you suggest for them to put on it? EDIT: They were actually a bit more specific about the use case. They want to use the LLM for local coding for the developers IDE (code completion and generation as well as reviews). The person I spoke to was also really interested in OpenClaw and AI agents and that I could set one up for us to evaluate once I found a good model. So its basically a playground for us. EDIT2: So sorry, I cannot reply to all of your comments. Thanks so much for your responses. I will evaluate and try different models. Also I understood I need to learn a lot about these high end Inference machines and the models that I can run on them. Guess I will grow into this role.
More from r/LocalLLaMA
This is where we are right now, LocalLLaMA
the future is now
Anthropic: "We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax." 🚨
I feel personally attacked