• 0 Posts
  • 6 Comments
Joined 1 year ago
cake
Cake day: July 2nd, 2023

help-circle



  • I like that, some YouTube video i came across had the sentiment that it was ok to be a renaissance man, jack of all trades kinda deal. I liked that a lot since i have a few different hobbies and im not particularly great at any of them but im good for the most part.

    Made me feel like i didn’t have to be perfect and it was cool to just be doing stuff.



  • Me personally, i use my AMD 7700XT to run ollama on my main pc. Can be helpful for troubleshooting as the internet gets worse amd worse to search especially if i dont know what the issue is. Thats my main use case for it but id like to set up something with RAG and use it to help me with documentation if i have questions.

    I don’t think its super worth it to use a VPS for an LLM if you already have a decrnt gpu that you can run it on. If yoy dont already have the hardware, plenty of older gpus can run the models pretty well. My 1070ti still kicks ass all these years later and you can find them for $100 bucks or less on ebay used. Ive used it for ollama as well and it does just fine.

    Will it be super fast or a really big model? No, but if its for personal use, i dont see any benefit to paying a monthly subscription for it and like i said, works well enough for me.

    Its also more secure and private to host it yourself if thats worth anything to you. Thats one of the biggest reasons i self host.