BTRFS – full filesystem not mounting – Good btrfs balance technique as well
#####################################################

For good balance Technique read “BALANCE STEP” below

ISSUE
######

So here is the issue, Your partition filled up and you cant mount it because its too full. But you need to mount it to empty up. Maybe it mounts as read only, however in readonly you cant really delete.

Here is how it would look like…

If your trying to mount something

# mount /dev/sda1 /mnt
mount: mounting /dev/md0 on /sysroot failed: No space left on device

Here is what dmesg says

# dmesg
btrfs: device label myvolonsda1 devid 1 transid 353730 /dev/sda1
btrfs: disk space caching is enabled
btrfs: could not do orphan cleanup -28
btrfs: open_ctree failed

FIX METHOD 1 – SINGLE COMMAND
###############################

# mount /dev/sda1 /mnt -t btrfs ‘-onospace_cache,clear_cache,enospc_debug,nodatacow’

UPDATE 2015-12-03: Read up about running the above command differently now http://www.infotinks.com/btrfs-recovery-mount-ultimate-mount-command/
# dmesg
btrfs: device label myvolonsda1 devid 1 transid 353730 /dev/sda1
btrfs: disabling disk space caching
btrfs: force clearing of disk cache
btrfs: setting nodatacow, compression disabled

# btrfs fi df /mnt
Data, single: total=3.32GiB, used=3.32GiB
System, DUP: total=8.00MiB, used=4.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=332.69MiB, used=25.08MiB
Metadata, single: total=8.00MiB, used=0.00

Now thats mounted as read/write, delete the big files. Dont forget to balance FS after..

BALANCE STEP:
==============

Dont forget to balance the system afterwards, to reclaim space. A balance reallocated blocks across the FS to make better usage of the space (its very beneficial also if you have multi devices in the FS, but if you have 1 thats fine)

If small FS (note for balance to work the FS needs to be read/write mode):
# btrfs balance start /mnt

If big filesystem the above will take forever, so only balance blocks that are smaller like 5% used blocks:
# btrfs balance start -dusage=5 /mnt

Or run it on 1% thru 25%, it should go really quick even on big FS:

# for i in 1 2 3 4 5 10 20 25; do echo “Balancing full FS but only on blocks that are at least ${i} % full – [`date`] – [`date +%s`]”; echo btrfs balance start -dusage=$i /mnt; done

NOTE: you can in another shell do “btrfs balance status /mnt” to get status

After balance, unmount

# umount /mnt

Then remount as normal mount

# mount /dev/sda1 /mnt

And run the same balance command again, it should go much quicker as its already balanced (but it insures a good balance)

FIX METHOD 2 – MORE COMMANDS AND REMOUNTING
############################################

This method might not always work, but if it does, its also legit method.

(STEP1) mount as recovery and readonly, this way it will mount

# mount -o recovery,ro /dev/sda1 /mnt

(STEP2) identify too big file/folder/files and back it up to data or where ever

# cd /mnt
# du -hsc *
etc…

now to delete it or truncate it (make file 0 size, but dont delete file and its permissions are kept the same, that way dont have to recreate the file with the same permissions)

(STEP3) to delete the big file remount the fs as rw

# mount -o remount,rw /mnt

if command fails repeat it and it will work (Repeat a few times)
# mount -o remount,rw /mnt
# mount -o remount,rw /mnt
# mount -o remount,rw /mnt

(STEP4) reclaim the space that you need
now
# rm /mnt/bigfile
or
# :> /mnt/bigfile
Repeat for every big file
NOTE: :> just emptys out a file, so its size 0, but file still exists with its permissions and attributes preserved, just the contents are erased and size goes to 0 bytes

(STEP5) balance the fs
Read “BALANCE STEP” section above

FIX METHOD 3 – READONLY AND BACKUP AND RESTORE
################################################

Also known as the cp/format/cp game

1. mount with readonly flag as that should mount no problem
2. copy the FS out some where, maybe data partition
3. format the FS
4. Copy back files back from backup location to the new FS

Dont forget to balance the NEW FS when your done

 

Leave a Reply

Your email address will not be published. Required fields are marked *