I’ve been trying to upgrade from 10.10 to 10.11 for a while now, as the Android TV app keeps nagging me, and every attempt ended with impossibly long library scan times.
After some thorough investigation, it appeared that a Home Videos type collection causes unending scan (yet to be solved), but also that Jellyfin does a lot of writes to the config directory (either database or metadata or both). Mine’s on spinning media part of a ZFS pool. I tried a few performance tuning options, such as testing the config dir with recordsize (similar to block size) of 4K, 8K, 64K, 128K and library scans fell from 30-40 minutes down to 8-13min with 4K-64K. The ZFS tuning wiki suggest 64K recordsize with LZ4 compression for SQLite workloads such as Jellyfin. That seems to work as well as 4K and 8K but likely is faster when reading thumbnails and such.
Note that upgrading to 12-rc3, which is supposed to speed up library scans did not improve scan times for me. Optimizing config/database write speed did. I cross-checked the culprit by experimenting with moving the config dir to NVMe and RAM. Both of those got the scan times down to 8-9 minutes compared to the optimized spinning media’s 12-13.
So if you had upgraded (or about to) to 10.11 your library scans are (about to get) dog slow and your Jellyfin’s config dir resides on spinning media, optimize its write performance for SQLite.
Isn’t it ironic that you post this meme along with complex instructions to fixing a problem in jellyfin?
With great power come great … uuh … :D
Also the meme is just bait.
complications 😅
Complications, yes!
Excellent bait
Yeah but tell me about this tool for Plex remote streaming…
It’s a wrapper called jellyfin
No, I said remote…
Anyone using jellyfin should not be exposing it to the public internet.
Hi, could you tell me why? I mean, I think I understand the basics of the risk of brute force attacks. My mitigation for the admin account is, not to allow login outside the local network. Users are allowed from anywhere, as my family uses the library. Also my service (there are others) are routed through NGINX.
Do you have tips for external users using, for example, a VPN only for specific addresses / IPs? At least that was an idea I had, but didn’t got got around to dig for solutions.
Thanks in advance.
VPN is the only way I would ever be doing it.
Why not? Unless you’re talking about unauthenticated steaming of media if you reverse engineer a servers folder structure.
It’s not secure. It’s a horrible thing to do
Do you have any proof that it’s not secure?
That’s the nice thing about Plex, you just stream your Plex library using Plex, and Plex is also the only place you and anyone you share with need to log in.
Also doesn’t jellyfin not support remote streaming?
It’s a nonsensical statement. Jellyfin is accessed through a website. You’re responsible for routing to it.
through a website
I access it via the app on my LG TV with WebOS, so not technically a website but yeah, over the network via an API at least 👍
It’s a very sensical statement. I’m not exposing a whole ass website on my own network just for remote viewing. With plex you don’t need to do that.
Yes you do. It just does it for you including handling SSL.
Inside the network you do, but not for remote access outside the network.
Outside the network you still have to open your server to allow plex cloud to access it and potentially tunnel your traffic through plex.tv if a direct connection doesn’t work. This happens automatically but it does happen
With Jellyfin you just have to supply your own relay, like Tailscale or wireguard, which does require more setup but is completely free, means that you can stop your media traffic from being funneled through services that harvest your data (which plex absolutely does), and doesn’t go to shit if plex.tv goes down
You can do the exact same thing with Plex, you just don’t need to since automatic remote streaming is built in. In fact, the article’s solution is literally Tailscale.
You can also bypass plex authentication on any ip range you choose, so if you add your local network and plex’s auth service goes down, it’s no big deal.
This happens automatically but it does happen
And that’s the key, it happens automatically and it just works. With jellyfin you have to expose the web server to the internet, or point an app to something else exposed on the internet. With plex you don’t. If you don’t want to (or can’t) a direct connection from your server to their servers, then their servers to your device is established. No VPNs, no reverse proxies, no port forwarding, nothing exposed to the rest of the internet.
What I’m praying for is for Jellyfin to add a tailscale like direct peer to peer system. They can skip the backup funneling traffic through their servers. But just have something that coordinates a direct connection with just basic NAT.
Let’scrypt and port forward. Sprinkle on some free ddns with a sync script/job to keep ddns pointing to your real public IP. Can even roll in some headscale if you’re feeling adventurous
Feeling adventurous is exactly what you need if you decide to expose Jellyfin to the Internet (a reverse proxy adds nothing to security)
You just use a VPN
You just need to setup a reverse proxy. My Jellyfin instance has been accessed from all over the world
Jellyfin does support remote streaming and it doesnt have to be through the web ui.
If you use folders a lot, the library will get faster with v12 https://github.com/jellyfin/jellyfin/issues/15141
OP said this didn’t improve their situation though.
How large is the database? Because there is a way to fully load SQLite into memory, which would speed everything up drastically for a scan. At the end of the scan, you could just write back everything to the disk, then it is just one blob of sequential write, which is fast enough on spinning rust.
Interesting, I figured people would use MySQL instead if possible.
600MB. Would that require changes to Jellyfin? They’re on the path to introducing Postgres support.
I just checked the code and unfortunately, they do not support loading the SQLite file into memory yet. Even worse, the change would be substantially. It appears they currently abuse the database connection a bit. They basically create a new connection for each operation. This does not matter if you can read/write in parallel, but that’s only the case with NVMe and certain drivers. So all SATA (SSD and HDD) suffer from the current mechanic. So just loading it into memory is not going to solve the issue. Jellyfin needs to change architecturally, so that a database connection is reused. I don’t know why or who implemented the current logic, but it is clearly done by someone not understanding how databases work.
Depending on the mood and time, I will come up with a solution that is solid. For now, you are stuck with the current logic.
Thanks for looking into it. As far as I read they moved to an ORM with 10.11 as a prereq to supporting other db engines. I’m guessing the db connection code would be undergoing major changes when they do that so I wouldn’t rewrite the current implementation. I’d either follow that work and help them get it right or rework the implementation after they’re done. We’ll just suffer through the mean time. 😄
Hey not bad for the developer…he vibe coded it right
Just the desktop client. Not the server.
Thanks, will take a look a my setup
I’ve always kept my apps and databases on solid state media, putting them on spinning rust, even in a well tuned ZFS pool with lots of disks is way too slow for their (admittedly inefficient) database.
This is true but it’s nice to have the whole app, data and database in one place, going together, snapshotted together, “backupable” together. It’s slower for sure. That said it can be reasonably fast with a large pool (more disks). With the magic of SSD cache, database reads fly and reads are the majority of the loads in my heads. In the future I would put root on ZFS as well and either do SSD cache or have root on ZFS SSD pool that gets send/recved regularly onto the spinning pool so it’s easy to restore when needed.
Removed by mod
Doesn’t jellyfin need a static IP.
Static IP for what? As long as you know the ip address of your jellyfin server you can connect to it. A static IP is just good to have for ease of remembering, using your own domain (not that you can’t do it with Dynamic addresses) or if you are integrating other services with it. This applies to any service you host.
Most ISP connection are behind CGNAT(ipv4), and in case of ipv6 to have a secure connection wouldn’t that require SSL certificate and a domain?
I have Pangolin setup on a VPS so I and my family can access it externally. It “bypasses” my router’s firewall. Tailscale was a stupid solution because a lot of people will be using it and I don’t want to maintain another service and be tech support for their tailscsle client. So the only thing I did to restrict access was by geoblocking.
No.
To function? No.
Useful? Yes.
