Best model + setup for remote deployment.
- Published
- 09/06, 18:40
- Ingested
- 09/06, 23:00
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
A user is seeking recommendations for a computer setup to run LLMs locally in a remote village with no internet access, but working phone calls. The budget is around $5,000 for a computer capable of running LLMs for basic knowledge and light Q&A, similar to ChatGPT. The recipient is technically savvy and can learn to run commands. The user is asking for GPU recommendations to meet these specific needs.
Hi everybody, I am in a tricky situation.
My grandfather + grandmother live in a remote village in asia, and were visiting us in the city. My grandfather has absolutely fallen in love with AI, and it is completely new for him. He loves how he can talk to it, and get information from it. The caveat is that I showed him chat gpt, not a localllm.
He wants something like this in his own village, just for basic knowledge + light qa. There is no internet in the village, but phone calls work. I have a budget of about 5k for a computer that can run LLMS locally, for him. (and he specifically requested something like this). He is technically "savvy", where if something needs a command to be run or something else, he can learn to do it. What gpu should I buy him that will satisfy his needs? again, I want some chat gpt level, but for basic q/a stuff.
I am currently running deepseek v4 flash on two 6000 pro blackwells(got them when they were just 8k lol), so I know that this is "possible".
Im thinking amd ai pro 9700 and gemma 31b at a good quantization. (maybe I can do two gpus of 9700s to fit bigger quant).
He is not doing frontier level coding, he just needs a good knowledge base. He implied that people from his village may also come to use the machine, or atleast request queries to be asked (he is some sort of governmental figure in his village).
I want recommendations.