btrfs forces the fs read-only on freeze/thaw while a device is failed

Found: 2026-08-11, during the 6.18 octopus validation dry-run (bhive, SUT zygo-6.18.x-zb64 @ 47891c21c94f, 6.18.42+, BTRFS_DEBUG=y). raid5 data / raid1 metadata.

Symptom

With one member device failing all IO (dm error target), the next freeze/thaw of the filesystem forces the whole fs read-only:

BTRFS: error (device dm-14) in btrfs_unfreeze:2481: errno=-5 IO failure
       (super block on devid 6 got modified unexpectedly)
BTRFS info (device dm-14 state E): forced readonly

A single failed device is within the redundancy of both profiles (raid5 data tolerates one lost column; raid1 metadata keeps a good mirror), so the fs should stay writable degraded, not go read-only.

Root cause

btrfs_unfreeze() (fs/btrfs/super.c:2464) re-reads every device's primary super block on thaw via check_dev_super() (super.c:2412) and, on any error, calls btrfs_handle_fs_error(... "super block on devid %llu got modified unexpectedly") -> forced read-only.

check_dev_super() does not distinguish unreadable from tampered:

sb = btrfs_read_disk_super(dev->bdev, 0, true);
if (IS_ERR(sb))
        return PTR_ERR(sb);        /* <-- -EIO from a failed device -> RO */

btrfs_read_disk_super() (fs/btrfs/volumes.c:1352) returns:

• -EIO when the read itself fails (read_cache_page_gfp, line 1385) -- i.e. the device is failed/unreadable;
• -EINVAL when the read succeeds but magic/bytenr are wrong (line 1391), and check_dev_super() then also rejects csum/transid mismatches (-EUCLEAN) -- i.e. the super block was genuinely modified.

So the two cases are cleanly separable, but btrfs_unfreeze treats an unreadable device (a tolerated failure) exactly like a tampered super block (a real integrity problem) and takes the whole fs down.

Why it matters (the trigger is common)

btrfs_unfreeze runs after any fs freeze/thaw:

• fsfreeze/FIFREEZE (LVM/backup snapshots take one routinely),
• hibernate/resume,
• a dmsetup suspend (default lock_fs) on any member bdev -- btrfs is the holder of every member, so freezing one freezes the fs.

Any of these, while one device is failed, needlessly kills a redundant fs.

The normal write path does the right thing

Injecting the same failure without a freeze (dmsetup suspend --nolockfs) leaves the fs writable degraded -- btrfs tolerates the failed device and only warns:

BTRFS warning (device dm-14): lost super block write due to IO error on ror-3 (-5)
BTRFS error (device dm-14): error writing primary super block to device 4
   ... (fs stays rw, further writes succeed)

So only the unfreeze re-check is over-strict; the runtime path already handles a failed device gracefully.

Reproduce (SUT, ~30s)

mkfs.btrfs -f -K -d raid5 -m raid1 <4 dm-linear members>
mount -o nodiscard <m0> /mnt ; echo hi > /mnt/before ; sync
sect=$(dmsetup table m3 | awk '{print $2}')
# default suspend freezes the fs:
dmsetup suspend m3; dmsetup reload m3 --table "0 $sect error"; dmsetup resume m3
# -> BTRFS ... btrfs_unfreeze:2481 ... forced readonly ; writes now EROFS
#
# with --nolockfs (no freeze): fs stays writable degraded, only warns.

The error set (why an -EIO-only fix is too narrow)

check_dev_super() can reject a device's super block three ways:

The realistic dying-disk case is the last row: a disk that fails writes first keeps a perfectly valid but stale super block on disk (this is the lost super block write due to IO error warning). On the next freeze/thaw check_dev_super() reads it fine and rejects it on the transid check (btrfs_super_generation(sb) != last_trans) -> -EUCLEAN -> forced RO. A pure -EIO (fully unreadable) is actually the least common mode. So tolerating only -EIO fixes almost none of the real exposure.

Proposed fix (broader)

btrfs_unfreeze's purpose is to detect the whole fs being modified externally while frozen (hibernate + another OS). A single device's super block being unreadable, stale, or garbage is a device failure the profile already tolerates -- not fs-wide tampering. So the unfreeze re-check should tolerate up to the profile's redundancy worth of bad device super blocks (for any errno -- -EIO / -EINVAL / -EUCLEAN), warning and bumping the device error counter the way the runtime write path does, and force the fs read-only only when more than that are bad (the mounted fs's own consistency is genuinely in doubt):

/* in btrfs_unfreeze(): count bad device supers, don't RO on the first */
int bad = 0, tolerated = btrfs_calc_...max_missing(fs_info); /* raid1->1, raid6->2, ... */
list_for_each_entry(device, &fs_info->fs_devices->devices, dev_list) {
        ret = check_dev_super(device);
        if (ret < 0) {
                btrfs_warn(fs_info,
                        "super block on devid %llu unreadable/stale (%d) after thaw",
                        device->devid, ret);
                btrfs_dev_stat_inc_and_print(device, BTRFS_DEV_STAT_...);
                if (++bad > tolerated) {
                        btrfs_handle_fs_error(fs_info, ret,
                                "too many device super blocks bad after thaw");
                        break;
                }
        }
}

(Exact "tolerated" derivation and which dev_stat counter are Zygo's call; the single-metadata-profile redundancy -- raid1 = 1 tolerated -- is the floor. A narrower first step, tolerating only -EIO, fixes the dm-error case but leaves the common stale-super/transid and the silent-corruption cases still forcing RO.)

Test-harness note

The dry-run's injector used a plain dmsetup suspend, which froze the fs and tripped this path -- masking the real degraded-write behaviour. The runner now uses dmsetup suspend --nolockfs so the online-FAIL model exercises a genuine disk death (the runtime path), which btrfs handles. This bug remains real and independently reachable via fsfreeze/hibernate on a degraded array.