stripe_alloc: reservation accounting bug, balance reclaim, minimal geometry

Findings from the fill-to-ENOSPC test generation (2026-08-02), all on bhive with the full series.

Bug: data reservation ignored trapped free space (fixed, two rounds)

btrfs_space_info_used() counted bytes_zone_unusable but not bytes_stripe_unusable, so the data reservation path admitted write()s against trapped sub-stripe free space the claim rule can never hand out. cow_file_range() then failed the real allocation at writeback and discarded the pages with the reservation -- silently: no message (the "dropping unwritten extent" log is on the ordered-extent path; this failure is earlier), no EIO for a non-fsyncing writer, and the freed reservation let the writer continue indefinitely.

Measured on kernel #77 (unfixed): the fill workload "wrote" 12.5 GiB into a 5 GiB filesystem; 1,008,802 blocks (~3.9 GiB) read back as short/zero on an UNDAMAGED fs.

• Round 1 (#78, fixup! 1c548be): count bytes_stripe_unusable in btrfs_space_info_used(), like zoned's bytes_zone_unusable. Cut the loss from unbounded to 3.3% ('b') / 30% ('c'): the counter is only recomputed at commit, and reservations race each commit's retire-time trapping.
• Round 2 (#79, fixup! cada874): (a) new bytes_stripe_open = Σ open runs' unallocated remainders, maintained per-bg under stripe_run_lock and synced into the space_info at every open/alloc/grow/close -- claimed bytes are invisible to the free space cache and must not back reservations; (b) a closed tail's sub-stripe head is added to bytes_stripe_unusable immediately (the commit rescan overwrites with the authoritative total, so the increment self-reconciles). Cut 'c' 30% -> 3.5%; 'b' flat.
• Round 3 (#81): one-full-stripe margin per open run (open to close; released when a run fills). No measurable change -- correctly conservative but aimed at the wrong window.
• Round 4 (#82): a writeback ENOSPC retry loop (flush parked rbios + periodic commit kick, 3s budget) that got the loss down to one warned 16K file. enospc_debug had shown the anatomy: accounting and allocator AGREE at every failure while ~19 MiB of admitted bytes_may_use is orphaned (reservations outlive commits; cumulative trapping starves them; ~470 MiB pinned that a commit partly returns as claimable stripes). Rounds 3 and 4 BACKED OUT at Zygo's direction ("no need to keep that dead end in the patch series") in favor of:
• Round 5 (#85, the root-cause fix): PESSIMISTIC RESERVATION, OPTIMISTIC ALLOCATION. A byte-counted reservation holds no collateral against the claim rule -- commits between write() and writeback trap sub-stripe tails, leaving old admissions holding air. Three pieces: (a) bytes_stripe_margin: one full stripe width per outstanding delalloc extent, charged/released inside btrfs_mod_outstanding_extents (exact across merges/splits; compression prices per-piece automatically via the io-tree split hook), counted in btrfs_space_info_used(); admission probes len+margin through the ticketed FLUSH_DATA path, so a writer that cannot be covered WAITS for flushing (commits convert pinned deletions into claimable stripes) and gets an honest ENOSPC at write(2) if that cannot help. The allocator keeps non-final split pieces whole-stripe multiples so at most one tail per extent can strand. (b) The #84 test of (a) alone still dropped 4 writebacks and exposed the mechanism's second half: mid-transaction returns of free space (unpinned deletions, same-transaction frees, drained run tails, freed reservations) sat in the free space cache as trapped fragments invisible to bytes_stripe_unusable until the commit rescan -- admitted may_use+margin (49.6 MiB) exactly equaled the phantom, and the next rescan jumped by exactly that amount. Fixed centrally: every btrfs_add_free_space() into an armed group counts the added range's sub-stripe head/tail fragments as trapped immediately (interior whole stripes are trivially fully free and stay claimable; bounded conservative overcount, rescan reconciles). This replaced the per-callsite stripe_run_tail_returned, which missed two drain-return paths AND miscomputed fragments (round_up() on the non-power-of-two 448K stripe width). (c) #85 fill-edge: PERFECT for the first time. RC=0, verify short=0, ZERO allocation failures, ZERO drop warnings, margin drained to 0 at idle, fill capacity unchanged (4804 MiB). Fullness now surfaces only as honest ENOSPC at write(2).

Full rolling scenario on #85: FIRST FULLY CLEAN PASS. Tree 'a' correct-or-EIO (0 wrong, 0 silent); trees 'b' and 'c' all-zero at every step (#82's 216 warned 'b' edge drops: gone); both scrubs "no errors found". The formerly-separate deterministic 'b' EIO cluster is also gone -- diagnosed as stale rbio-cache pages poisoning the parity of a claimed stripe's first partial write (see NOTE-b-eio-cluster.md) and fixed by making rmw_try_pad_full() win over the cached-pages shortcut in rmw_rbio().

(Kernel #84 also crashed once, via the hang watchdog, during its fill-edge run -- no trace reached disk; not reproduced on #85 with the phantom fixed and zero enospc_debug dumps. Suspected: the free-space-entry dump printing hundreds of lines to the 115200-baud serial console under locks, while the margin code raised sinfo lock traffic. Moot while allocation failures are zero; remember it if dumps ever return.)

Sanity datum: with the fix, stripe_alloc's fill capacity equals legacy's within 2 MiB (4582 vs 4584) -- pure sequential fill traps nothing measurable; trapping starts with deletion holes.

Honest statfs arithmetic confirmed

After deleting 2297 MiB from the full fs: df-avail 1112 MiB + stripe_unusable 1187 MiB = 2299 MiB. The counters add up.

Balance reclaims trapped space (RESOLVED: headroom recipe)

Original finding: on a fill-to-ENOSPC fs, btrfs balance start -d collapsed stripe_unusable 1187 -> 51 MiB and the refill recovered 93% of the deleted bytes, but balance itself exited ENOSPC before the last chunks. Zygo's diagnosis: NOT series work -- balance of a block group requires enough free space outside it to relocate all its extents at their original sizes, and a filesystem filled to ENOSPC has none; correct behavior, wrong test setup.

Test v2 (his recipe): multi-GiB devices, shrink each by 1G after mkfs (btrfs fi resize <devid>:-1g), fill, grow back (btrfs fi resize <devid>:max) before balancing -- the regrown space guarantees one block group's worth of destinations and each freed group cascades more; the same technique admins already use against metadata ENOSPC. Kernel #86 result: BOTH allocators PASS. stripe_alloc: balance completes ("9 out of 11 chunks"), stripe_unusable 1176 -> 1 MiB, tree 'a' byte-perfect through balance and refill, refill absorbs the full regrown capacity.

Metadata raid56 under stripe_alloc (analysis, 2026-08-03)

The machinery below find_free_extent (runs, claim, padding, rescan, central fragment accounting, balance reclaim) is allocator- and content-agnostic and would cover metadata unchanged; metadata is simpler in one way (always CoW: no nocow class, no run persistence). The real work is reservation: metadata reserves per-item via block rsvs and overcommit, not per-byte delalloc, so the pessimism has to move there (roughly one stripe width per active run folded into the overcommit check) -- and the stakes are higher, because a mid- transaction metadata allocation failure is a transaction abort, not a warned dropped writeback.

Performance positioning (Zygo): stripe-alloc'd raid5 metadata would have NO RMW at all -- every write is a full stripe or padded to one -- putting it between upstream raid56 metadata (full RMW) and raid1c3/c4. The mirrors write fewer bytes when a stripe isn't full (3-4 copies of a node vs a padded full stripe) and burn no parity CPU, but cost more space than full-stripe raid5/6 parity. Padded tails also strand run remainders at high fsync rates (log tree), so log blocks likely want a dedicated run.

Mixed block groups: btrfs_is_stripe_alloc_bg() tests the DATA flag and do_allocation() dispatches per block group, so a mixed (DATA|METADATA) raid56 group already routes BOTH classes through the claim rule today -- mechanically complete, but metadata reservations there get no margin pricing, risking abort near-full. Until metadata reservation support exists, mixed groups should be excluded from the gate explicitly (!(flags & METADATA)) or added to the test matrix.

Minimal write-hole geometry

The stock "magic 256K" that lets narrow-array tests pass by accident is find_free_space()'s full-stripe alignment: allocations >= full_stripe_len are stripe-aligned (since the original raid56 merge 53b381b3abeb), so byte-dominant large files tile whole stripes and opt out of every shared-stripe failure mode. Sub-stripe files get no escape at any width.

Falsification suite (two-drive-writehole.sh, now misnamed):

• 2 devices (1 data + parity): the write hole is GEOMETRICALLY impossible -- parity is a per-row mirror, a torn row desyncs only its own uncommitted data. Confirmed clean on both allocators under both crash models (45/45 controls, 0/72 armed, everything mountable under barrier). Kept as the structural control.
• 3 devices statistical: low power. Under the entries model most torn images are unmountable ("couldn't test": 28-32 of 54 armed; 1 genuine mounted hit); under the barrier model an rbio's data and parity log entries are nearly adjacent, so independent suffixes rarely separate them (0/108).
• 3 devices DIRECTED (directed-writehole.sh): deterministic and conclusive. Sixteen 4K files fill column 0 (txn 1, committed); file 17 lands at column 1 row 0 of the same stripe (txn 2, verified by fiemap+chunk-map and by content search); image surgery reverts f17's data block while keeping the txn-2 parity ("parity landed, data write lost" -- honest, both writes share one unflushed window); drop the member holding column 0. Stock allocator: committed f1 reads EIO -- the write hole, 100% reproducible. stripe_alloc: f17 provably placed in the next stripe, f1 reconstructs. (Two earlier false "survivals" were harness bugs: a greedy chunk-map regex stitching the SYSTEM chunk's start onto the data chunk's stripes, then the original members' loop devices staying registered so the "degraded" mount silently assembled the pristine member. Both fixed; the final run is clean end to end.)

SOLVED: the unrecoverable tree-'b' EIO cluster

In fill-mode rolling runs, tree 'b' (written after wiping devs 1+2, before wiping dev 3) ends with a deterministic EIO cluster reconstruction and scrub cannot fix (112 blocks on #77-78, 48 on #79-80, 34 on #81, 32 on #82). Root cause found by stripe-level post-mortem of the preserved #82 images: a stale cached rbio from the stripe's previous ('a'-era, pre-wipe) life is stolen by the first partial write into the re-claimed stripe; the all-uptodate stolen pages skip both the RMW read and rmw_try_pad_full(), so parity is computed from the cache's memory of columns whose real on-disk content the wipe had replaced. Full evidence chain, fix directions, and forensic method: NOTE-b-eio-cluster.md.