Posted by u/inserterikhere
Latest upgrade…A100 40 GB
Originally this was my gaming rig but I went ITX and basically bought a new computer. So I had the case, fans, AIO, 64 GB DDR5, motherboard, PSU, and 3080 (upgraded to 5070ti RIP). I was going to sell these parts, but I started running models on my 5070ti and eventually I wanted to start running larger models. I found a 3090 on eBay for $680, and 7950x for $350. I put that together with the parts and it’s been a great AI rig for me. I really didn’t plan on upgrading this for a while, especially now with the current price surges. Welp, I saw an A100 get listed for $1000 on eBay. The catch? Listed for parts, and the description just said “card reports CUDA error”. So I figured it was worth the risk (for me), I could’ve probably sold it for the price I paid. Well, I swapped out the 3080 and on the first boot it was recognized instantly by nvidia-smi. I was able to run and train models immediately. Nice.
External link:
https://i.redd.it/f66wnmearldg1.jpegMore from r/LocalLLaMA
My story of underestimating /r/LocalLLaMA's thirst for VRAM
zai-org/GLM-4.7-Flash · Hugging Face
NVIDIA's new 8B model is Orchestrator-8B, a specialized 8-billion-parameter AI designed not to answer everything itself, but to intelligently manage and route complex tasks to different tools (like web search, code execution, other LLMs) for greater efficiency
I’ve seen some arguments we’ve reached AGI, it’s just about putting the separate pieces together in the right context....