Assuming they already own a PC, if someone buys two 3090 for it they’ll probably also have to upgrade their PSU so that might be worth including in the budget. But it’s definitely a relatively low cost way to get more VRAM, there are people who run 3 or 4 RTX3090 too.

For LLMs it entirely depends on what size models you want to use and how fast you want it to run. Since there’s diminishing returns to increasing model sizes, i.e. a 14B model isn’t twice as good as a 7B model, the best bang for the buck will be achieved with the smallest model you think has acceptable quality. And if you think generation speeds of around 1 token/second are acceptable, you’ll probably get more value for money using partial offloading.

If your answer is “I don’t know what models I want to run” then a second-hand RTX3090 is probably your best bet. If you want to run larger models, building a rig with multiple (used) RTX3090 is probably still the cheapest way to do it.

Hooray, I can finally play it. Had it on my wish-list for years, when I finally bought it I found out that neither the native Linux or the Windows+Proton version was working.

A static website and Immich

There are tons of options for running LLMs locally nowadays, though none come close to GPT4 or Claude 2 etc. One place to start is /c/localllama@sh.itjust.works

I asked someone about this a few days ago, and they claimed to have over 30000 photos in Nextcloud without issues

I suppose “a few” is quite open to interpretation, but I have 50k photos now so if it can handle 100k without getting sluggish it’ll probably be fine for the foreseeable future.

Does Nextcloud handle large numbers of photos nowadays? IIRC when I was comparing programs some years ago I read that both it and Owncloud struggled when you got to a few 10000s of photos.

Ah, nice.

Btw. perhaps you’d like to add:

build: .

to docker-compose.yml so you can just write “docker-compose build” instead of having to do it with a separate docker command. I would submit a PR for it but I have made a bunch of other changes to that file so it’s probably faster if you do it.

Awesome work! Going to try out koboldcpp right away. Currently running llama.cpp in docker on my workstation because it would be such a mess to get cuda toolkit installed natively…

Out of curiosity, isn’t conda a bit redundant in docker since it already is an isolated environment?

I downloaded them from here: https://huggingface.co/TheBloke

I think they’re seeing this as a traditional copyright infringement issue, i.e. they don’t want anyone to be able to make copies of their work intentionally either.

It’s not an easy question, but I don’t think it’s about just choosing to stand idly by while they’re dying. Finding and recovering the sub would be an incredibly difficult and expensive operation. It might not apply to you specifically, but if someone thinks that the government should try to save these people, regardless of the cost, I think it raises the question why we’re letting other people die from preventable causes. Perhaps you disagree with current politics and think the government should do everything to save both rich and poor alike, but IMO if a multi-million $ rescue operation had been launched, it would’ve been a reminder that we, as a society, are letting other people die.