Create an uneven used space multi-device bcachefs file system like this,
where sdb1 is about 1/2 used and sdb2 is almost empty:
# bcachefs format /dev/sdb1
# mount /dev/sdb1 /mnt/1
# dd if=/dev/zero bs=1M count=512 of=/mnt/1/zeros.bin
# bcachefs device add /mnt/1 /dev/sdb2
GParted reports both devices as having the same usage:
Partition File System Mount Point Size Used Unused Flags
/dev/sdb1 bcachefs /mnt/1 1.00 GiB 526.00 MiB 498.00 MiB
/dev/sdb2 bcachefs /mnt/1 1.00 GiB 526.00 MiB 498.00 MiB
When in fact the used space is significantly different for each device:
# bcachefs fs usage /mnt/1 | egrep ' \(device |free:|capacity:'
(no label) (device 0): sdb1 rw
free: 522190848 3984
capacity: 1073741824 8192
(no label) (device 1): sdb2 rw
free: 1061945344 8102
capacity: 1073741824 8192
This is because bcachefs::set_used_sectors() is always searching for the
first "free:" and "capacity:" figures. Fix by reading the figures for
the correct device.
[Only the options being used by GParted are quoted here from the help
output. More options are available.]
$ bcachefs fsck --help
bcachefs fsck - filesystem check and repair
Usage: bcachefs fsck [OPTION]... <device>
Options:
-y Assume "yes" to all questions
-f Force checking even if filesystem is marked clean
-v Be verbose
Closes!123 - Add support for bcachefs, single device file systems only
When the size of the partition being cleared is an exact multiple of
MiBs, likely given that GParted aligns partitions to whole MiBs by
default, then the same range will be zeroed 4 times in a row. Example
operation details from clearing a partition look like this:
Format /dev/sdb1 as cleared
+ calibrate /dev/sdb1
path: /dev/sdb1 (partition)
start: 2048
end: 2099199
size: 2097152 (1.00 GiB)
+ clear old file system signatures in /dev/sdb1
write 512.00 KiB of zeros at byte offset 0
write 4.00 KiB of zeros at byte offset 67108864
write 512.00 B of zeros at byte offset 1072161280
>> write 4.00 KiB of zeros at byte offset 1072693248
>> write 4.00 KiB of zeros at byte offset 1072693248
>> write 4.00 KiB of zeros at byte offset 1072693248
>> write 4.00 KiB of zeros at byte offset 1072693248
write 512.00 KiB of zeros at byte offset 1073217536
+ set partition type on /dev/sdb1
This is because the bcachefs backup super block is located at -1 MiB
from the end of the device, rounded down by the bcachefs bucket size.
The bucket size can be any of 128 KiB, 256 KiB, 512 KiB or 1 MiB,
depending on the size of the bcachefs file system. So when the
partition size is an exact multiple of MiBs all 4 possible rounding
sizes result in the same offset.
Avoid writing the same range of zeros multiple times by skipping a range
if it is identical to the previous range.
Closes!123 - Add support for bcachefs, single device file systems only
GParted doesn't clear all bcachefs file system signatures.
# truncate -s $((1*1024*1024*1024)) /tmp/disk.img
# losetup --find --show /tmp/disk.img
/dev/loop0
# bcachefs format /tmp/disk.img
# wipefs /dev/loop0
DEVICE OFFSET TYPE UUID LABEL
loop0 0x1018 bcachefs 15304edb-6dc4-476c-989e-74eaea6660e8
loop0 0x3ff00018 bcachefs 15304edb-6dc4-476c-989e-74eaea6660e8
Use GParted to format to cleared /dev/loop0.
# wipefs /dev/loop0
DEVICE OFFSET TYPE UUID LABEL
loop0 0x3ff00018 bcachefs 15304edb-6dc4-476c-989e-74eaea6660e8
The backup super block is located at -1 MiB before the end of the
device, rounded down to the file system's bucket size [1], where the
bucket size is one of: 128 KiB, 256 KiB, 512 KiB, 1024 KiB [2].
location = device size - 1 MiB
= hex(1*1024*1024*1024 - 1*1024*1024)
= 0x3ff00000
Bcachefs magic is at byte offset 24 (0x18) into the super block [3].
The backup super block was not erased because GParted only writes zeros
from -512 KiB to the end of the device. It is necessary to clear old
signatures before formatting with a new file system to avoid confusion
from detection of multiple incompatible signatures [4][5]. Fix this by
writing zeros over all possible locations of the bcachefs backup super
block.
[1] bcachefs-tools v1.6.4: c_src/libbcachefs.c:bch2_format()
https://evilpiepirate.org/git/bcachefs-tools.git/tree/c_src/libbcachefs.c?h=v1.6.4#n313
[This code locates the backup super block at device size - 1 MiB
rounded down to the bucket size.]
321 struct bch_sb_layout *l = &sb.sb->layout;
322 u64 backup_sb = size_sectors - (1 << l->sb_max_size_bits);
323
324 backup_sb = rounddown(backup_sb, i->bucket_size >> 9);
[2] bcachefs-tools v1.6.4: c_src/libbcachefs.c:bch2_pick_bucket_size()
https://evilpiepirate.org/git/bcachefs-tools.git/tree/c_src/libbcachefs.c?h=v1.6.4#n85
85 /* Want a bucket size of at least 128k, if possible: */
86 bucket_size = max(bucket_size, 128ULL << 10);
...
94 /* max bucket size 1 mb */
95 bucket_size = min(bucket_size * scale, 1ULL << 20);
[3] bcachefs-tools v1.6.4: libcachefs/bcachefs_format.h:struct bch_sb
https://evilpiepirate.org/git/bcachefs-tools.git/tree/libbcachefs/bcachefs_format.h?h=v1.6.4#n907
[4] 743968ef68
Add clearing of SWRaid metadata 0.90 and 1.0 super blocks (#756829)
[5] 3c75f3f5b1
Use wipefs to clear old signatures before creating new file systems (#688882)
Closes!123 - Add support for bcachefs, single device file systems only
Add reading of the bcachefs file system label, not the per device label,
and the external UUID. These match what blkid reports.
Example without a label:
# bcachefs format /dev/sdb1
# bcachefs show-super /dev/sdb1 | egrep -i 'Label:|UUID:|Device:'
External UUID: 3316bc9a-d129-42b6-a80e-9649874bca73
Internal UUID: 656eebe5-10a9-4f12-94c8-aab2fdc54732
Label:
Device: 0
Label: (none)
UUID: cd436a8d-82eb-4993-a317-b39ea0d6bd2e
# blkid /dev/sdb1
/dev/sdb1: UUID="3316bc9a-d129-42b6-a80e-9649874bca73" BLOCK_SIZE="512" UUID_SUB="cd436a8d-82eb-4993-a317-b39ea0d6bd2e" TYPE="bcachefs" PARTUUID="7962e584-34c9-4088-8a00-a651af517089"
Example with a label:
# bcachefs format --force -L 'test label' /dev/sdb1
# bcachefs show-super /dev/sdb1 | egrep -i 'Label:|UUID:|Device:'
External UUID: 3d7bdabe-2616-4545-affc-1aba0f8fb4a7
Internal UUID: 9cc95d3e-7991-4f78-9dd0-850cb9749e34
Label: test label
Device: 0
Label: (none)
UUID: 784d1bd0-5769-4fbb-ad32-07894d381bba
# blkid /dev/sdb1
/dev/sdb1: UUID="3d7bdabe-2616-4545-affc-1aba0f8fb4a7" LABEL="test label" BLOCK_SIZE="512" UUID_SUB="784d1bd0-5769-4fbb-ad32-07894d381bba" TYPE="bcachefs" PARTUUID="7962e584-34c9-4088-8a00-a651af517089"
Closes!123 - Add support for bcachefs, single device file systems only
For bcachefs file systems 19 MiB and smaller, the available space is
reported as a very large value. The calculation went negative in a
64-bit signed value but then was interpreted as an unsiged value.
Writing any significant amount of data to the file system hangs.
# truncate -s $((19*1024*1024)) /tmp/test.img
# losetup --find --show /tmp/test.img
/dev/loop0
# bcachefs format /dev/loop0
# mkdir /mnt/0
# mount /dev/loop0 /mnt/0
# strace -e statfs df -k /mnt/0
statfs("/mnt/0", {f_type=0xca451a4e, f_bsize=4096, f_blocks=2305843009213693856,
f_bfree=2305843009213693600, f_bavail=35474507834056483, f_files=18446744073709529090,
f_ffree=18446744073709529088, f_fsid={val=[0xddb6645d, 0x8560584]}, f_namelen=512,
f_frsize=4096, f_flags=ST_VALID|ST_RELATIME}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 9223372036854775424 1024 141898031336225932 1% /mnt/0
For a 20 MiB bcachefs the available space is 0 so the file system
overhead is 100%.
# umount /mnt/0
# truncate -s $((20*1024*1024)) /tmp/disk.img
# losetup --set-capacity /dev/loop0
# bcachefs format --force /dev/loop0
# mount /dev/loop0 /mnt/0
# strace -e statfs df -k /mnt/0
statfs("/mnt/0", {f_type=0xca451a4e, f_bsize=512, f_blocks=1280, f_bfree=0, f_bavail=0,
f_files=2, f_ffree=0, f_fsid={val=[0x6b3e4926, 0x33f99a32]}, f_namelen=512, f_frsize=512,
f_flags=ST_VALID|ST_RELATIME}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 640 640 0 100% /mnt/0
For a 32 MiB bcachefs the file system overhead is a more reasonable 9%
so use this as the minimum bcachefs file system size.
...
# truncate -s $((32*1024*1024)) /tmp/disk.img
...
# strace -e statfs df -k /mnt/0
statfs("/mnt/0", {f_type=0xca451a4e, f_bsize=512, f_blocks=24832, f_bfree=22784,
f_bavail=22433, f_files=182274, f_ffree=182272, f_fsid={val=[0xfdddedd3, 0xe90be3cb]},
f_namelen=512, f_frsize=512, f_flags=ST_VALID|ST_RELATIME}) = 0
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/loop0 12416 1024 11217 9% /mnt/0
Closes!123 - Add support for bcachefs, single device file systems only
Currently bcachefs-tools only provides a method to report the file
system usage while it is mounted. We won't make GParted mount a
bcachefs to read it's usage as we want to keep GParted's scanning as a
read-only activity. Therefore GParted can't report the usage of an
unmounted bcachefs.
# bcachefs format /dev/sdb1
# bcachefs fs usage /dev/sdb1
error opening /dev/sdb1: not a bcachefs filesystem
# echo $?
1
# bcachefs fs usage --help
bcachefs fs usage - display detailed filesystem usage
Usage: bcachefs fs usage [OPTION]... <mountpoint>
...
# mount /dev/sdb1 /mnt/1
# bcachefs fs usage /mnt/1
Filesystem: a61a8302-9a79-4c24-a9e6-486e7fcc78f5
Size: 987842560
Used: 12713984
Online reserved: 0
Data type Required/total Durability Devices
btree: 1/1 1 [sdb1] 1048576
(no label) (device 0): sdb1 rw
data buckets fragmented
free: 1061027840 8095
sb: 3149824 25 126976
journal: 8388608 64
btree: 1048576 8
user: 0 0
cached: 0 0
parity: 0 0
stripe: 0 0
need_gc_gens: 0 0
need_discard: 0 0
capacity: 1073741824 8192
# echo $?
0
Closes!123 - Add support for bcachefs, single device file systems only
Set the minimum file system size to 16 MiB as creating a bcachefs that
size succeeds:
$ truncate -s $((16*1024*1024)) /tmp/disk.img
$ bcachefs format /tmp/disk.img
...
initializing new filesystem
going read-write
initializing freespace
$ echo $?
0
Where as creating a smaller file system fails for most sizes below that:
$ rm /tmp/disk.img
$ truncate -s $((15*1024*1024)) /tmp/disk.img
$ bcachefs format /tmp/disk.img
...
mounting version 1.6: btree_subvolume_children
initializing new filesystem
going read-write
bch2_trans_mark_dev_sb(): error ENOSPC_disk_reservation
bch2_fs_initialize(): error marking superblocks ENOSPC_disk_reservation
bch2_fs_initialize(): error ENOSPC_disk_reservation
bch2_fs_start(): error starting filesystem ENOSPC_disk_reservation
error opening /tmp/disk.img: ENOSPC_disk_reservation
$ echo $?
1
Closes!123 - Add support for bcachefs, single device file systems only
Bcachefs [1] has many of the same capabilities as Btrfs [2] and ZFS [3]:
COW (Copy-on-Write), multi-device, multi-volume, snapshotting and many
more. Therefore when adding bcachefs use the same range of Orange
colours already used by Btrfs and ZFS [4]. As bcachefs is a native
Linux file system and ZFS is not, move ZFS to a darker shade of Orange
to allow bcachefs to be added in the middle:
Btrfs - Orange Medium (#E58749)
bcachefs - Orange Dark (#C26825)
ZFS - Orange Shadow (#984F18)
[1] bcachefs
https://bcachefs.org/
[2] Welcome to BTRFS documentation! > Introduction
https://btrfs.readthedocs.io/en/latest/Introduction.html
[3] ZFS
https://en.wikipedia.org/wiki/ZFS
[4] 8a4f9ad205
Adjust shades of aquamarine, cyan and orange
Closes!123 - Add support for bcachefs, single device file systems only
When preparing the GParted Live 1.6.0 distribution, which is based on
Debian unstable ("sid"), compiling GParted failed like this:
$ make
...
/usr/bin/msgfmt --desktop --template gparted.desktop.in -d ./po -o gparted.desktop
chmod +x gparted
/usr/bin/msgfmt --xml --template org.gnome.gparted.policy.in -d ./po -o org.gnome.gparted.policy
/usr/bin/msgfmt: cannot locate ITS rules for org.gnome.gparted.policy.in
make[3]: *** [Makefile:1060: org.gnome.gparted.policy] Error 1
make[3]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/root/gparted/gparted-1.6.0-beta1'
make[2]: *** [Makefile:618: all-recursive] Error 1
make[2]: Leaving directory '/root/gparted/gparted-1.6.0-beta1'
make[1]: *** [Makefile:452: all] Error 2
make[1]: Leaving directory '/root/gparted/gparted-1.6.0-beta1'
dh_auto_build: error: make -j16 returned exit code 2
make: *** [debian/rules:9: build] Error 25
dpkg-buildpackage: error: debian/rules build subprocess returned exit
status 2
debuild: fatal error at line 1184:
dpkg-buildpackage -us -uc -ui failed
This was also previously reported in the GParted Forum [1]. Future
Debian 13 ("trixie") and Ubuntu 24.04 LTS ("nobel") releases have moved
the needed gettext translation rules for .policy XML files:
/usr/share/gettext/its/policy.its
/usr/share/gettext/its/policy.loc
to new package libpolkit-gobject-1-dev not installed by default.
Document this new build time dependency.
Also see commits [2][3] where the equivalent change was needed in the
Alpine Linux and CentOS continuous integration images.
[1] GParted forum / [SOLVED] Unable to build "msgfmt: cannot locate ITS
rules for org..."
http://gparted-forum.surf4.info/viewtopic.php?id=18136
[2] 57ae8f888b
Fix .policy file translation failure in Alpine Linux CI image (!107)
[3] 8450d8c605
Fix .policy file translation failure in CentOS CI image (!107)
Closed!121 - Document future Debian/Ubuntu build time dependency in
README
... now Attempt Data Rescue has been removed and GParted no longer has
code to show "file:/tmp/gparted-roview-XXXXXX" URIs. Missed in earlier
commit:
8ce9074ac6
Remove Attempt Data Rescue and use of gpart (!118)
As the AppStream 1.0 [1] specification no longer describes them as
appdata files, but instead as metainfo files, rename the Makefile.am
variables for consistency with the name of the standard.
[1] AppStream 1.0
https://www.freedesktop.org/software/appstream/docs/index.htmlCloses#241 - Move appstream metadata out of legacy path
AppData files always were a subset of the AppStream specification
[1][2]. AppStream 0.12 specification [3] onwards says the metainfo
files will be found when placed in /usr/share/metainfo/ *AND* that
/usr/share/appdata/ is a legacy location *AND* a future release of
AppStream will likely drop support for it [4].
Debian 10, RHEL 7 and Ubuntu 18.04 LTS distributions all have the
/usr/share/metainfo/ directory containing application .appdata.xml and
.metainfo.xml files. Ubuntu 16.04 LTS does not have the directory
despite the AppStream specification [3] claiming it does. As old
supported distributions do have the directory, unconditionally update
this.
For reference are these commits in projects GNOME System Monitor [4] and
Evince [5] from 2017 making the same change.
[1] AppData Specification [circa 2016]
https://web.archive.org/web/20160903181519/https://people.freedesktop.org/~hughsient/appdata/
"Rather than create a new schema from scratch, we'll be using a
subset of the AppStream metadata proposal.
Applications wishing to have long descriptions, screenshots and
other useful things are required to ship one or more files in
/usr/share/appdata/%{id}.appdata.xml.
"
[2] AppStream 0.4, 2.2 AppData XML files [circa 2013]
https://web.archive.org/web/20131204004054/http://www.freedesktop.org/software/appstream/docs/sect-AppStream-Metadata-AppData.html
[3] AppStream 0.12, 2.1.2 Filesystem locations [circa 2020]
https://web.archive.org/web/20200615042130/https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#spec-component-location
"2.1.2 Filesystem locations
Upstream projects can ship one or more metainfo files in
/usr/share/metainfo/%{id}.metainfo.xml, where id is a unique
identifier of this specific component.
(>) Note
Component metadata of type desktop-application as described in
Section 2.2, "Desktop Applications" can be installed with an
.appdata.xml extension as well for historical reasons. AppStream
implementations will read the XML files as long as they end up in
the right location on the filesystem.
(!) Important: Legacy Path
AppStream tools scan the /usr/share/appdata/ path for legacy
compatibility as well. It should not be used anymore by new
software though, even on older Linux distributions (like RHEL 7 and
Ubuntu 16.04 LTS) the metainfo path is well supported. Support for
the legacy path will likely be dropped completely with a future
AppStream 1.0 release.
"
[4] [GNOME System Monitor] Install appdata to the new location
(bgo#790146)
43dc057771
[5] [Evince] build: Install appstream metadata to non-deprecated
location
8cae24ea48Closes#241 - Move appstream metadata out of legacy path
... code pattern. This is to make the code easier to understand by not
having to remember if condition context for indented code over longer
distances. This has been done before. Here are just 2 examples:
[1] 75bda733bb
Refactor run_blkid_load_cache() into if fail return early (#131)
[2] 407e0ac6e3
Refactor fat16::read_label() into if fail return early pattern (!104)
Closes!119 - Tidy-ups for file system interface classes
Restructure the variable parsing code into "if leading text found then
scan the number" pattern.
Closes!119 - Tidy-ups for file system interface classes
Restructure the variable parsing code into "if leading text found then
scan the number" pattern.
Anchor leading text matches to the start of a new line in the output.
Closes!119 - Tidy-ups for file system interface classes
Restructure the variable parsing code into "if leading text found then
scan the number" pattern.
Closes!119 - Tidy-ups for file system interface classes
And restructure the variable parsing code into "if leading text found
then scan the number" pattern.
Anchor leading text matches to the start of a new line in the output.
Closes!119 - Tidy-ups for file system interface classes
FileSystem member variables T, N & S are being used like local variables
in many of the file system specific set_used_sectors() methods. They
are only used within each set_used_sectors() call and not used to
represent persistent information of a FileSystem interface class or to
pass information between separate methods. Therefore stop using them
and replace them with local variables instead.
This block of code finds a field in the output and scans the number:
Glib::ustring::size_type index = output.find("Block count:");
if (index >= output.length() ||
sscanf(output.substr(index).c_str(), "Block count: %lld", &T) != 1)
T = -1;
The if statement says "if leading text is not found or scanning the
number fails then assign -1". A sequence of two negatives leading to
assigning an error value is hard to understand. Instead this an
equivalent block from btrfs::set_used_sectors():
long long total_bytes = -1;
Glib::ustring::size_type index = output.find("\ntotal_bytes");
if (index < output.length())
sscanf(output.substr(index).c_str(), "\ntotal_bytes %lld", &total_bytes);
This assigns a default error value and the if statement says "if leading
text found then scan the number". Much simpler to understand.
Therefore change the code around to use this same pattern.
Anchor the leading text matches to the start of a new line in the
output where possible. Just because it's what some of the other file
system's set_used_sectors() methods do (btrfs, reiser4 and xfs) and it
seems like more robust text matching.
Closes!119 - Tidy-ups for file system interface classes
Replace floating point calculation to convert size and space figures
from file system block sized units to sectors with an integer
calculation. Do this for the same reasons discussed in commit "Stop
using floating point calculations in FS resize() methods" earlier in
this patchset. This will limit the largest file system that GParted can
read the usage of to 8 EiB - 1 bytes.
There is still a floating point calculation in btrfs::set_used_sectors()
which is being left because that is apportioning used space figure
between multiple devices.
Closes!119 - Tidy-ups for file system interface classes
A number of the file system specific resize() methods use floating point
calculations to convert from the new partition size in sectors to the
new file system size to be passed to the resize command in bytes or
kibibytes. This is bad because there could be rounding errors
converting from integer to floating point, performing the calculation
and converting back. Replace with integer only multiply and divide
calculations. Integer division always truncates [1] which is exactly
what is needed. The largest integer will be the size of the file system
in bytes held in a signed 64-bit long long, or Sector or Byte_Value
typedef of the same type. This will limit the size that a file system
can be shrunk to, to 8 EiB - 1 byte.
[1] C++ Arithmetic operators
https://en.cppreference.com/w/cpp/language/operator_arithmetic
"the algebraic quotient of integer division is truncated towards
zero (fractional part is discarded)"
Closes!119 - Tidy-ups for file system interface classes
