Home Ai Server

[paullas]

As the title says has anyone built there own home Ai server.

I am going to start building one over the next few weeks/months just to see whats what. It isnt for business just hobby/curious really but be interesting to know if anyone here has gone down this rabbit hole.

 

[ben]

No, but you are going to need a LOT of RAM and probably a fair few cores to run any LLM at a usable level, I would imagine. I have heard also that a good GPU will drastically reduce processing time.

 

[Hook]

Rent a server, don't build. You could have your LLM up and running in half an hour.

Running it from home would draw a lot of power, providing you have the correct hardware. You need a good GPU (cluster of them ideally) with a lot of VRAM. Undervolted because they'll be running a lot.

 

[paullas]

thanks for the above all good food for thought cheers. I didnt want to go down rental route as its all a learning curve but i will look into that aswell just to see what is on offer.