I’d like to self host a large language model, LLM.
I don’t mind if I need a GPU and all that, at least it will be running on my own hardware, and probably even cheaper than the $20 everyone is charging per month.
What LLMs are you self hosting? And what are you using to do it?
You can run this right from Windows: https://jan.ai/
You’ll need a lot of RAM, and processing is decently fast, even on a basic laptop.
edit: holy hell. Grammar.
This is pretty cool!