- Update to version 0.14.1:
Bugfixes:
* The metadata_builder now recomputes all total sizes
(total_fs_size, total_allocated_fs_size, and
total_hardlink_size) as part of the build() function. This not
only ensures that the totals are correct even if the allocated
size changes between scanning and segmenting (which has been
happening at least on ZFS volumes), but it also allows images
affected by a related bug in Windows builds of DwarFS to be
fixed by rebuilding the metadata.
* Instead of making the FUSE drivers fail hard when seeing the
options that were removed in v0.14.0, they now just log a
warning and ignore them. The options may still be fully removed
in a future release.
gh#mhx/dwarfs#303.
* The pcmaudio categorizer had two minor issues when compressing
a large number of WAV files. One was reporting an unsupported
format: 3/0 or unsupported format: 65,534/3 warning, which
isn't very useful for the end user. These format codes
correspond to IEEE floating point formats, which are indeed
unsupported. However, the format appears to be quite common,
so the warning has been downgraded to an info message that
explicitly mentions the floating point format. The second issue
was an unexpected fmt chunk size of 20 bytes, which caused the
file to be rejected as a PCM audio file (meaning it was added
using a generic compressor instead of FLAC). It turns out that
these non-conforming fmt chunks are also quite common in
practice, so the code has been changed to accept the
non-conforming file, but also logging an info message
mentioning the non-conformance.
OBS-URL: https://build.opensuse.org/request/show/1331760
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/dwarfs?expand=0&rev=7
Bugfixes:
* The metadata_builder now recomputes all total sizes
(total_fs_size, total_allocated_fs_size, and
total_hardlink_size) as part of the build() function. This not
only ensures that the totals are correct even if the allocated
size changes between scanning and segmenting (which has been
happening at least on ZFS volumes), but it also allows images
affected by a related bug in Windows builds of DwarFS to be
fixed by rebuilding the metadata.
* Instead of making the FUSE drivers fail hard when seeing the
options that were removed in v0.14.0, they now just log a
warning and ignore them. The options may still be fully removed
in a future release.
gh#mhx/dwarfs#303.
* The pcmaudio categorizer had two minor issues when compressing
a large number of WAV files. One was reporting an unsupported
format: 3/0 or unsupported format: 65,534/3 warning, which
isn't very useful for the end user. These format codes
correspond to IEEE floating point formats, which are indeed
unsupported. However, the format appears to be quite common,
so the warning has been downgraded to an info message that
explicitly mentions the floating point format. The second issue
was an unexpected fmt chunk size of 20 bytes, which caused the
file to be rejected as a PCM audio file (meaning it was added
using a generic compressor instead of FLAC). It turns out that
these non-conforming fmt chunks are also quite common in
practice, so the code has been changed to accept the
non-conforming file, but also logging an info message
mentioning the non-conformance.
OBS-URL: https://build.opensuse.org/package/show/filesystems/dwarfs?expand=0&rev=10
- Update to version 0.13.0:
Bugfixes:
* Made section index discovery more robust.
gh#mhx/dwarfs#264
* A recent kernel change (https://lkml.org/lkml/2025/5/5/2868)
caused the tools_test to fail on Linux 6.14 and later.
This has been fixed by accepting both EPERM and ENOSYS as
valid error codes for link() calls.
Features:
* Support for big-endian architectures.
This is still experimental, even though all unit tests pass
with QEMU, and the benchmark suite runs fine on real hardware.
This currently requires forked versions of folly and fsst.
The changes are small and the pull requests will hopefully be
merged upstream soon.
* Experimental support for 32-bit architectures.
While DwarFS should mostly "just work" on 32-bit when using
small images (a few hundred megabytes), the limited address
space is a problem for the extensive use of memory-mapped
files inside DwarFS. There will be changes to limit the use of
mmap in the future (mainly due to other issues), which should
help 32-bit compatibility as a side-effect.
gh#mhx/dwarfs#268.
* The category metadata for categorized blocks is now stored in
the metadata block by default. This allows re-compressing the
blocks with a metadata-dependent algorithm (e.g. FLAC) even if
they were previously compressed using a metadata-independent
algorithm.
This can be disabled using the --no-category-metadata option.
See the mkdwarfs man page for more details.
OBS-URL: https://build.opensuse.org/request/show/1302510
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/dwarfs?expand=0&rev=5
Bugfixes:
* Made section index discovery more robust.
gh#mhx/dwarfs#264
* A recent kernel change (https://lkml.org/lkml/2025/5/5/2868)
caused the tools_test to fail on Linux 6.14 and later.
This has been fixed by accepting both EPERM and ENOSYS as
valid error codes for link() calls.
Features:
* Support for big-endian architectures.
This is still experimental, even though all unit tests pass
with QEMU, and the benchmark suite runs fine on real hardware.
This currently requires forked versions of folly and fsst.
The changes are small and the pull requests will hopefully be
merged upstream soon.
* Experimental support for 32-bit architectures.
While DwarFS should mostly "just work" on 32-bit when using
small images (a few hundred megabytes), the limited address
space is a problem for the extensive use of memory-mapped
files inside DwarFS. There will be changes to limit the use of
mmap in the future (mainly due to other issues), which should
help 32-bit compatibility as a side-effect.
gh#mhx/dwarfs#268.
* The category metadata for categorized blocks is now stored in
the metadata block by default. This allows re-compressing the
blocks with a metadata-dependent algorithm (e.g. FLAC) even if
they were previously compressed using a metadata-independent
algorithm.
This can be disabled using the --no-category-metadata option.
See the mkdwarfs man page for more details.
OBS-URL: https://build.opensuse.org/package/show/filesystems/dwarfs?expand=0&rev=7
- Replace wrong BuildRequires pkgconfig(clzma) -> pkgconfig(liblzma);
build only succeeded previously by accident.
- Update to version 0.12.4:
Bugfixes
* Segfault on bad_compression_ratio_error. When recompressing a
filesystem where some blocks cannot be compressed using the
selected algorithm because of a bad_compression_ratio_error,
the resulting block was left empty.
* Add history unless --no-history is given when rewriting a file
system image.
* Allow dumping frozen_layout w/o frozen_analysis in dwarfsck.
* Logging timestamps should show local time.
Features
* More complete breakdown of metadata in dwarfsck.
* Add schema_raw_dump flag to dwarfsck --detail.
Build
* Update folly/fbthrift/fsst.
OBS-URL: https://build.opensuse.org/request/show/1301602
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/dwarfs?expand=0&rev=4
Bugfixes
* Segfault on bad_compression_ratio_error. When recompressing a
filesystem where some blocks cannot be compressed using the
selected algorithm because of a bad_compression_ratio_error,
the resulting block was left empty.
* Add history unless --no-history is given when rewriting a file
system image.
* Allow dumping frozen_layout w/o frozen_analysis in dwarfsck.
* Logging timestamps should show local time.
Features
* More complete breakdown of metadata in dwarfsck.
* Add schema_raw_dump flag to dwarfsck --detail.
Build
* Update folly/fbthrift/fsst.
OBS-URL: https://build.opensuse.org/package/show/filesystems/dwarfs?expand=0&rev=5
- Update to version 0.12.3:
Bugfixes
* Automatic image offset detection (for images using a custom
header) did not work correctly if the header contained a
string that would be identified as the start of a v1 section
header (these were only used before dwarfs-0.3.0).
If there was either "DWARFS\x02\x00" or "DWARFS\x02\x01" in
the header, offset detection would fail. The check has been
modified to peek further into the data and ensure this really
is a v1 section header, and also checking if the next section
header position can be derived from the length field.
It is still possible to construct a file system image where
offset detection will ultimately fail, but it is much less
likely with the change.
- Changes in version 0.12.2:
Bugfixes
*The dwarfs-0.12.0 release introduced a performance regression
where FLAC compression took more than twice as long as in the
previous releases. This has been fixed. FLAC decompression was
unaffected.
OBS-URL: https://build.opensuse.org/request/show/1271206
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/dwarfs?expand=0&rev=3
Bugfixes
* Automatic image offset detection (for images using a custom
header) did not work correctly if the header contained a
string that would be identified as the start of a v1 section
header (these were only used before dwarfs-0.3.0).
If there was either "DWARFS\x02\x00" or "DWARFS\x02\x01" in
the header, offset detection would fail. The check has been
modified to peek further into the data and ensure this really
is a v1 section header, and also checking if the next section
header position can be derived from the length field.
It is still possible to construct a file system image where
offset detection will ultimately fail, but it is much less
likely with the change.
- Changes in version 0.12.2:
Bugfixes
*The dwarfs-0.12.0 release introduced a performance regression
where FLAC compression took more than twice as long as in the
previous releases. This has been fixed. FLAC decompression was
unaffected.
OBS-URL: https://build.opensuse.org/package/show/filesystems/dwarfs?expand=0&rev=4
- Update to version 0.12.1:
Features
* Added --memory-limit=auto to mkdwarfs to use a more reasonable
(hopefully) default for the block queue. The old default of
1 GiB was quite arbitrary and definitely not suitable for
low-end systems. The new auto default will determine the limit
based on the number of workers (which in turn is based on the
number of CPUs), the block size, and the amount of physical
memory of the system.
* Replaced vector_byte_buffer with malloc_byte_buffer, which is
internally based around a simple buffer that doesn't incur the
cost of initializing each element like std::vector. Especially
for large blocks which are known to be overwritten immediately,
this can save a few CPU cycles.
- Changes in version 0.12.0:
* New Licensing Conditions: Instead of being all GPL-3.0 like all
the previous releases, this release changes the license of a
large fraction of the DwarFS code to MIT. All tools and
libraries that only read DwarFS images are now MIT-licensed.
Everything else (e.g. mkdwarfs) is still GPL-3.0 for the time
being.
Bugfixes
* Changes for compatibility with Boost.Process v2.
Features
* Re-licensed all libraries required for reading DwarFS images
under the MIT license. The source of all tools that just read
DwarFS images (i.e. everything except for mkdwarfs) are also
under the MIT license now. Everything else is still GPL-3.0.
gh#mhx/dwarfs#255
* New hotness categorizer in mkdwarfs that allows a list of "hot"
OBS-URL: https://build.opensuse.org/request/show/1269054
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/dwarfs?expand=0&rev=2
Features
* Added --memory-limit=auto to mkdwarfs to use a more reasonable
(hopefully) default for the block queue. The old default of
1 GiB was quite arbitrary and definitely not suitable for
low-end systems. The new auto default will determine the limit
based on the number of workers (which in turn is based on the
number of CPUs), the block size, and the amount of physical
memory of the system.
* Replaced vector_byte_buffer with malloc_byte_buffer, which is
internally based around a simple buffer that doesn't incur the
cost of initializing each element like std::vector. Especially
for large blocks which are known to be overwritten immediately,
this can save a few CPU cycles.
- Changes in version 0.12.0:
* New Licensing Conditions: Instead of being all GPL-3.0 like all
the previous releases, this release changes the license of a
large fraction of the DwarFS code to MIT. All tools and
libraries that only read DwarFS images are now MIT-licensed.
Everything else (e.g. mkdwarfs) is still GPL-3.0 for the time
being.
Bugfixes
* Changes for compatibility with Boost.Process v2.
Features
* Re-licensed all libraries required for reading DwarFS images
under the MIT license. The source of all tools that just read
DwarFS images (i.e. everything except for mkdwarfs) are also
under the MIT license now. Everything else is still GPL-3.0.
gh#mhx/dwarfs#255
* New hotness categorizer in mkdwarfs that allows a list of "hot"
OBS-URL: https://build.opensuse.org/package/show/filesystems/dwarfs?expand=0&rev=3
