My media pool was quietly filling up. Not from new downloads — I hadn't grabbed anything big in weeks — but the free space kept sliding down anyway, a few gigabytes at a time, like a slow leak. When I finally went looking for where it was all going, I found more than five hundred gigabytes of the exact same movies I already had. My own library was haunting itself.
The slow leak
The first sign was mundane: a disk-usage alert on my bulk pool, the one that holds all the media. I hadn't added a big library, so my assumption was some runaway log or a snapshot gone wild. I went in expecting to delete a few stray gigabytes and move on.
Instead I found that basically every film sat on disk twice. Once in the downloads directory — where the torrent client was still happily seeding it — and once, byte-for-byte identical, in the media library where the app had "imported" it. Add up a media library's worth of that and it came to roughly 543 GB of pure duplication. Half a terabyte of déjà vu.
Why "import" meant "copy"
Here's the mechanism. My download manager grabs a file into a downloads folder. Then the automation layer — the *arr apps that organize movies and shows — takes that finished download and puts a nicely-named version into the media library the media server actually reads from. In a healthy setup, that second step is a hardlink: the seeding file and the library file are two names pointing at the same bytes on disk. The download costs you the file once, and seeding it afterward is effectively free.
Mine wasn't hardlinking. It was copying. And the reason was a decision I'd made months earlier without thinking about this consequence at all: my downloads lived on one ZFS dataset, and my media lived on a different one.
A hardlink is just a second name for the same inode — and an inode belongs to exactly one filesystem. You cannot link across a filesystem boundary. Ask to, and the tool silently falls back to a full copy.
Two ZFS datasets look like two folders, but they are two separate filesystems. So when the app tried to hardlink from downloads to media and couldn't, it did the only other thing it could — it copied. Quietly. Every single import. No error, no warning, just a second full-size clone of everything, forever.
The client that wouldn't clean up
Copying would have been survivable if the download side ever cleaned itself out. It didn't. My torrent client has a setting that's supposed to remove a torrent and delete its files once it hits a seeding ratio or time limit. On the version I was running, that action simply never fired. Torrents sailed past their ratio, past their time cap, and kept seeding indefinitely — which meant the download-side copies never went away. So the copy problem and the never-delete problem compounded: every grab doubled my storage, and nothing ever reaped the second half.
My first response was a band-aid, and I'll admit it was a band-aid: a small scheduled script that does the cleanup the client wouldn't — walk the torrents, delete the ones past their ratio or age. But with one important piece of paranoia baked in. Some content — full disc structures, that kind of thing — never gets imported by the automation at all, which means the only copy is the one still in downloads. Reap that blindly and you've deleted the actual file. So the reaper explicitly skips anything that looks like a sole surviving copy. A cleanup script that can eat your only copy is worse than the mess it's cleaning.
The actual fix
The band-aid kept the disk from overflowing, but it was treating a symptom. The real problem was the filesystem boundary, and the real fix was to remove it: put downloads and media under one dataset.
Once they share a filesystem, hardlinking works. The import step stops copying and starts linking, and the seeding torrent and the library file become two names for one set of bytes. I migrated everything onto a single dataset, pointed both sides at it, and checked the thing that actually proves it worked — the inode and its link count:
# same inode number, link count of 2 = one file, two names
stat -c '%i %h' /data/torrents/<movie>
stat -c '%i %h' /data/media/movies/<movie>Same inode, link count 2. A brand-new grab landed, imported, and cost me the file exactly once. Seeding it afterward now costs essentially nothing. The 543 GB of duplication was never a quota problem or a client bug at heart — it was a filesystem boundary I'd drawn without realizing what it would cost me.
The lesson
The headline is one sentence: hardlinks live and die inside a single filesystem, so your downloads and your library have to be on the same one — or every import is a full copy and you pay for every file twice. If you run any download-then-organize pipeline, go check right now whether your import step is actually linking; the giveaway is a link count of 1 on files you know were seeded, or a used-space number that's suspiciously close to double what your library should be.
And the meta-lesson, the one I keep paying tuition on: a cleanup script is a comfortable answer that lets you avoid fixing the thing that's actually broken. The reaper made the alarm stop, which felt like winning. The one-dataset migration made the alarm impossible, which was actually winning. It's worth knowing the difference before you reach for cron.
Comments