Just felt the need to express somewhere how impressed I am. I’m using too many drives in a homelab setting, so there are SATA power splitters involved. When I added zlogslog devices, I had one drive fault out because I ended up with too many drives on a five-way splitter (which is probably something we shouldn’t even manufacture… but my fault for buying), rebalanced the drives and restarted, no issues since nothing was being written. Did my first major transfer to the server today and a drive faulted out again (just resetting repeatedly), so I spent an hour making another modular cable for my power supply and eliminated that splitter entirely.

I expected a corrupt drive and a full resilver; what I got was only 150GB out of a 450GB transfer needing resilvered. It only had to recalculate the data since the drive faulted out.

Amazing engineering, that’s all I’ve got to say. Have a good day, everyone!

EDIT: not everything is a z

Recently I've been doing a lot of cleanup of files on my NAS which has involved moving/copying around a lot of data. After a while the pool was freezing up and becoming unresponsive and I figured out ZFS's memory usage has been reaching a point where its maxing out my system. According to arc_summary and cat /sys/module/zfs/parameters/zfs_arc_max, ARC is configured to use at most 16GiB of memory, but I witnessed arcstat reporting ARC as using over 200GiB of memory (system has 256GiB total), even causing one of my VMs running on the system to hard crash. Whats going on? How can I limit ARC memory usage?

I've to sell my old laptop before getting new one. Old laptop is running with zpool mirror 2 disks, data /home only. I cannot keep all 2 disk later, also no more spare disk to backup. Could i just unplug any disk from this pool, plug into new PC, and read/copy data from this disk? I wanna create zpool mirror on new laptop too. I guess I will make pool with new disk 1st, then copy data from old disk into it. And attach disk become new mirror pool. Is there any point I need to prepare before migration?

Edit, I went ahead with the copy and the system boots and runs, I do not know why it changed but it is aparently OK.

I am in process of rebuilding a pool per instructions semi-derived from.

https://docs.zfsbootmenu.org/en/v3.1.x/guides/debian/uefi.html

I am following notes I have used before.

create command

sudo zpool create -f -o ashift=12 \ -O compression=lz4 \ -O acltype=posixacl \ -O xattr=sa \ -O relatime=on \ -o autotrim=on \ -m none suwannee /dev/disk/by-id/nvme-eui.0025384931406ac8

in the past the step looked like this

mount | grep mnt suwannee/ROOT/Void on /mnt type zfs (rw,relatime,xattr,posixacl) but today

sudo zpool export suwannee sudo mkdir /mnt/suwannee sudo zpool import -f -N -R /mnt/suwannee suwannee sudo zfs create -o mountpoint=/ -o canmount=noauto suwannee/ROOT/LMDE7 sudo mkdir /mnt/870/donor sudo mount /dev/sdd3 /mnt/870/donor sudo zfs mount suwannee/ROOT/LMDE7 mount | grep mnt suwannee/ROOT/LMDE7 on /mnt/suwannee type zfs (rw,relatime,xattr,posixacl,casesensitive)

My pools being for Linux have always been case sensitive as the pool property, but it has never been displayed as mount property.

example ``` zfs get all | grep casesensitivity NAME PROPERTY VALUE SOURCE

lagoon/suwannee/ROOT/Void_Xfce@autosnap_2026-03-03_05:00:03_monthly casesensitivity sensitive - ```

I will continue on, but will this be an issue? Why did it change? did I make a mistake somewhere? or has ZFS changed? this is the first time I have created a pool under Debians ZFS v2.4.3-1~bpo13+1

Just ran into a weird case I've never seen before, but I have a pool reporting devices experiencing corruption, except zpool status shows no READ, WRITE or CKSUM errors for any device.

zpool status -v does show one corrupted file but I'm just confused how the corruption occurred or has been detected. Here's the full status output (minus the file path):

``` $ zpool status -v zdata pool: zdata state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A config:

NAME                                      STATE     READ WRITE CKSUM
zdata                                     ONLINE       0     0     0
  mirror-0                                ONLINE       0     0     0
    fb899172-3886-4a7d-bc05-93c6bcdfeada  ONLINE       0     0     0
    4b935244-7f9a-41b4-8b55-d0e9e3090458  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files: ```

The system in question has non-ECC RAM which was my first thought, but I'm not sure that tracks as I would except a flip in RAM to result in either silent corruption (ZFS writes out corrupted data with a valid checksum for that data so it verifies) or at least a CKSUM error (ZFS wrote out corrupted data with checksum for the original, or vice versa). I checked SMART stats for both drives and I'm not seeing an sign of any errors (nothing reallocated, unrecoverable, no CRC errors etc.).

In terms of recovery this is a non-issue as the file in question has already been overwritten so it only exists in an automatic snapshot which I can just discard. I'm just confused as to how this corruption might have occurred in the first place, and why it isn't registering as an error for any of the devices? What else is there?

AI performance depends on the efficiency of the AI (Anthropic Claude is currently leading) and on how well it is trained for a given environment (my napp-it cs comes with extensive training data).

Helpdesk process (1h)

• unclear error message
• initial analysis by Claude AI, with follow-up question for logs
• code analysis
• code improvement

That was for an error message reported in a forum. With a local Claude installation including local and remote access (file and shell to cluster member servers), analysis + fix + test would have taken 5 minutes.

(use Chrome to translate on the fly)

https://www.hardwareluxx.de/community/threads/napp-it-cs-web-gui-für-fast-jeden-zfs-server-oder-servergruppen.1349039/page-11#post-31228989

EDIT:

zhack label repair -u /dev/sdg1zhack label repair -u /dev/DISK

Solved it for me!

Hi...

Right after I detached one of my drives from my mirror (2 total) I accidentally secure erased (impossible to recover) the drive still in the mirror instead of the detached drive.

Now I can run zdb -l /dev/disk/by-id/DISK-ID and I get a bit of data, including 1 uberblock, but all txg are 0.

When I try zdb -d /dev/disk/by-id/DISK-ID I always get not found errors.

zpool import never find the pool.

Is recovery possible?

Yes I have backups for some stuff, but not all (notably Windows VM with save files)

Thank you for the help

I needed to reshuffle some disks in TrueNAS on a mirrored pool, was trying to do it online.

I offlined one disk, moved it, onlined it, checked the resiliver status.

Resiliver is showing as scanning 0%, zpool status shows 0B scanned and it's been two hours. All disks look healthy

zpool status shows all disks online no degraded status.

Only other thing of interest is there was on another pool a Z2 pool doing a disk expansion.

So it's like it's stuck. Interestingly enough the expansion which was still on going on the Z2 also dropped off the TrueNAS UI earlier in the week but I could see it was still moving so just left it.

I think I'm going to need a reboot which I was trying to avoid.

Give all disks showed as online and nothing had happened in hours I tried a zpool clear but the resilvering is still there and stuck.

Any ideas?

TrueNAS v25.04.2.1

Edit:

zfs-2.3.0-1
zfs-kmod-2.3.0-1

Edit:

Reboot although very inconvenient resolved this all for me. TrueNAS has lost track of the expansion which is annoying, but I can track it via zfs tools.

Hi all, I'm wondering and hoping that there's a better solution than what I'm currently using.

I have a ZFS server (with sanoid snapshots) at my place, and I also have a Synology machine at a friend's that I use as my offsite backup, and that also has its own snapshots. They're tailscaled so they can talk to each other.

Currently, I have a nightly cron job to rsync any changed files over, which does work, but I have a few issues with it:

• If a file is renamed on the main (ZFS) server, it's fully re-sent to the backup server and the old copy is deleted there. This causes the snapshots on the Synology server to be slightly bloated, but it'd be a bigger issue if I do any larger-scale file reorganization.
• I have a lot of files so rsync just... takes a while to look for changes. Not a dealbreaker, but I'd rather save ~3 hours of HDD read time per day at these drive prices if possible XD

Aside from ditching the Synology machine and setting up another ZFS server offsite instead, is there a way to solve one/both issues?

I moved this pool from an old server i was selling to a new one, going from having a backplane to cables. I can never seem to get all 16 disks to show up despite trying a different HBA card, but i've individually checked all disks and they are all functional without any issues. Every time i've tried to import the thing it locks up the OS and i have to hard power down the system. I have a theory that if i can image all the drives and try and put them together again with disk images on a larger pool instead of the physical drives that might work, but i don't have access to something that has 64TB of free space.

I formatted an external USB drive that I’m intending to use in a single disk pool. I have other backups, the only reason I’m using ZFS here is for encryption and data integrity checks.

When I formatted the disk, I forgot to specify ashift=12 and it used ashift=9 sectors by default because the disk reported as a 512b…

I already have some data in the disk and I don’t want to sync everything together to this machine because it’s in a remote location and connection is not very fast. How bad is it going to be if I keep it as it is? Has anyone quantified this in practice?

I’ll only have occasional writes. Files will be mostly read only.

Here's the problem I'm struggling with:

I'm trying to setup a Jellyfin server on FreeBSD 15.1 with 2 drives with ZFS filesystems on them - one is the '`zroot`' boot volume, the other is the data drive '`zjail`', totaling 2 pools.

I want to keep the zjail writable on the server to save new movies there, but read-only in the jail.

I cleared the 2nd drive with '`zpool labelclear -f ada1`' and '`gpart destroy -F ada1`', so it's empty. Then I created a GPT Partition Table, added a ZFS Partition, created the pool and the ZFS filesystem, and stuck a file in it during testing to see something whenever I finally get it loaded.

My `/etc/jail.conf` includes:

`path = "/jails/jellyjail";`

Inside the jail, I created the path for the read-only movies to mount to:

`mkdir -p /media/jaildata`

At this point everything loads fine.

Now I try to configure things to use the RO 'zjail' drive.

Edited `/etc/jail.conf` to use `nullfs`: `mount.fstab = "/jails/jellyjail/etc/fstab";`

Edited that `fstab` file: `/zjail/jaildata/ /media/jaildata nullfs ro 0 0`

Each of these paths can be seen in the appropriate place (ie `/zjail` in the server & `/media` in the jail).

I know the fstab needs to be:

`/path/on/host/ /path/in/jail nullfs ro 0 0`

but no matter how I list `/path/on/host` or `/path/in/jail` , when I restart the jail I get an error that it can't find that file or directory (typically the host directory).

I'm lost. I've read multiple pages on how to set this up, and I think I doing it right - but obviously not.

Thanks for any and all help on this.

My zpool output gives

zroot     ONLINE    75%

How would this much fragmentation impact the disk space?

I find the disk space almost 100% consumed at inopportune moments.

How do I prevent this?

Zfs record sizes for various data sets are default 128k, except for mysql dataset.

Hi everyone,

I wanted to share my custom workstation implementation of ubuntu-server-zfsbootmenu tailored specifically for engineering workflows.

The Problem

I recently migrated from Windows to Linux for my engineering work. I quickly realized that proprietary EDA and embedded tools—like Xilinx Vivado or ST STM32CubeIDE—can absolutely wreck a Linux system. They often introduce untraceable, undeletable configuration changes, broken dependencies, and global environment mess.

When a hardware compile fails or a board disconnects, it creates a massive headache: Is my PC's OS failing, or is it an actual hardware/board failure?

The Solution: Disposable Bare-Metal Environments

To solve this, I wanted an environment where the entire operating system behaves like a disposable "package," but without the performance overhead or limited hardware access of a traditional Virtual Machine.

Disk partitioning

Using ZFS and ZFSBootMenu, I built a "Jenga tower" architecture:

1. Swappable OS Datasets: I can instantly snapshot, roll back, or clone entirely separate Ubuntu OS environments. They act like VMs, but run on bare metal with 100% direct access to the computer's internals (GPU, PCIe, USB JTAG debuggers, etc.).
2. Non-Volatile Data Partition: A separate ZFS data dataset that persists independently. No matter which OS version or snapshot I boot into, they all securely mount and access the same project/data files.

If a tool update breaks my environment, I just blow the dataset away and roll back to a clean state in seconds.

I put together a guide and the installation scripts to deploy this alongside an existing pool here: 👉https://github.com/ArthorH/Ubuntu-ZFS-Install-existing-pool

I tested this system for about half a year now and its really helpful to just toss whole system into the bin and revert it to clean state (without ruining every other setup). It helps me so i wanted to share :).

I'd love to hear your thoughts, or how other engineers are tackling the "messy proprietary IDE" problem on Linux workstations!

I'm retired and out of date, I know raid from the days of battery backed ram in hardware controllers and stacks of 9.1gb scsi drives going cronch cronch cronch...

I have 4 2TiB magnetic disks from an obsolete WD mycloud EX4 in a raidz1 pool. It's already decently improved but I'm wondering if I can bolt more bits of scavenged tech together without trashing stuff.

I can run one extra drive that isn't my zpool, it's my boot/system drive and is currently a dirt cheap sata BX500 of a pcie sata controller but I think i can drop in an pcie nvme controller instead.

I'm considering putting a ZIL and L2 ARC onto a chunk of whatever domestic SSD i end up using for a bit of a performance boost, but I'm kind of aware i might need to think about wear levelling particularly for the ZIL partition

• is there a locality effect of partitions on 'modern' ssds or is this completely abstracted away?

• Do i need to leave un-partitioned space for wear levelling to work properly or just not fill every last mb of every partition?

• Is this still a performance boost doing this with ZFS in 2026 or am I overcomplicating this? My main storage disks aren't fast.

Do I have to run a full scrub or does onlining it bring it back up to speed relatively quickly?

Basically, if I want to try again to pull the bad drive out, is it safe to do so right now or should I wait for the scrub I started to finish?

I'm increasing the capacity of my pool by replacing ten 4TB drives with 8TB ones.
So, I take a drive offline, remove it, and replace it with an 8TB drive.
---mod. I only have this system and only have a 10 slot SATA card---
Then I wait for the resilvering to finish.

Well, I've never encountered a situation like this before—let's hope it holds up for half a day!

Pray for me!

** rc13

• Previous Versions (VSS) fixes with automounts, Windows Server 2025 and more.
• ZIL replace BSOD fix
• Fix snapshot mounts, browsing

https://github.com/openzfsonwindows/openzfs/releases
https://github.com/openzfsonwindows/openzfs/issues

If you run ZFS you know the pain before buying drives for a build: first you have to figure out whether a drive is CMR or SMR, then you try to guess how likely it is to die. It is not easy to find reliable drives, and getting the first one wrong can cost you money and data.

Getting an SMR drive instead of an CMR (sneaking into NAS lines) is something you don't want by any means: Those drives will drag out a ZFS resilver for days, and some just fall out of the array! That's basically the whole WD Red SMR mess from a few years back. And you usually can't tell from the box. The model numbers seem designed for you to make a mistake: WD40EZAX is CMR but WD40EZAZ is SMR - just one letter difference.

I started keeping my own list in a spreadsheet of what's actually CMR, built from the official datasheets and from decoding model numbers. And over a few months it grew into a real dataset. Then I added failure rates on top, computed from Backblaze's raw 2025 Drive Stats. That was a huge amount of raw data, so I summed up the drive lifetime and failures of every NAS drive model.

A few things that surprised me:

• Plain WD "Red" (WD60EFAX and friends) is SMR. Crazy, huh? "Red Plus" and "Red Pro" are CMR. That one word on the box is the whole difference.
• Bigger and newer doesn't automatically mean more reliable drive. The WD Ultrastar HC590 26TB sits at 0.4%, but the current Exos X24 24TB is at 2.75%, and the older Exos X10/X12/X14 are up to around 5% failure rate.
• The safest CMR drives with real data right now, the ones I would trust in a ZFS pool, are the WD Ultrastar HC5x0/HC590 line and the newer Exos X16/X18, all under about 0.6%.

Fair warning: only 36 of the 150 drives have Backblaze data, since they only run so many models. So I left the failure rate column blank, rather than guess. The rest of the list have full specs and CMR/SMR flag.

If you catch a wrong CMR/SMR flag, let me know and I'll fix it. Those are the easiest ones to get wrong.

Everything's here:

Website (has sortable tables: CMR/SMR, failure rate, price per TB, drive noise, SATA/SAS, RPM): https://www.nasdisks.com
Dataset (CSV/JSON, CC BY license - Backblaze data keeps their own attribution): https://www.nasdisks.com/data/
Full disclosure: the site has Amazon affiliate links, that's how I cover hosting bills.

https://www.napp-it.org/index_en.html

Sync: "Server 1 ZFS Filesystem 1" -> "Server 2 Filesystem 2"

Before everyone says this has existed for ages with rsync, robocopy, and zfs send — here's the feature list:

• Direct sync any-to-any, including e.g. Windows <-> FreeBSD <-> Solaris (daisy-chain capable)
• Encrypted sync via z-stream tunnel with one-time key (https://github.com/guenther-alka/z-stream/releases)
• Restartable (file-based), unlike incremental replication
• Creates ZFS snapshots (source and destination)
• ACLs are preserved (within a single OS)
• Practical up to about 1TB — beyond that, zfs send is unbeatably faster

Since the last version

• Single folder (/opt/csweb-gui), copy and run, no dependencies (https://www.napp-it.org/downloads_en.html)
• Frontend web GUI for managing the local server (free) or a group via an encrypted backend connection
• Frontend and backend on any OS (FreeBSD, Illumos, Linux, OSX, Solaris, and Windows)
• 2FA with mobile authenticator
• S3, Storage Spaces, and ZFS with ACL, user, and share management
• Encrypted replication of unencrypted filesystems via z-stream
• Easily extensible with custom menus (optimized for Claude AI-assisted work, https://www.napp-it.org/pdf/csweb-gui_use_ai.pdf)