pool/duperemove
SHA256
17
0
Fork 1
Code Issues Pull Requests Activity
75 Commits 4 Branches 0 Tags
factory
Commit Graph

75 Commits

This Branch
This Branch
All Branches
Author SHA256 Message Date
Ana Guerrero
e6e2ed9cdd Accepting request 1243434 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1243434
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=34
 2025-02-06 21:05:39 +00:00
Dirk Mueller
2b45b409e9 - Update to 0.15: 
* Files are no longer invalidated when renamed.
  * The scan phase now reports both files and bytes in its progress,
    which is especially useful when processing large files.
- Add duperemove-0.15-buildfail.patch:
  To fix the build. See #367.
- Add subpackage for zsh completions

OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=72
 2025-02-05 15:54:02 +00:00
Ana Guerrero
39e4e09698 Accepting request 1224241 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1224241
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=33
 2024-11-15 14:42:35 +00:00
David Sterba
1ed2548585 - force newer gcc version in sync (isc rq #353372) 
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=70
 2024-11-14 16:05:32 +00:00
Ana Guerrero
e90d0e7101 Accepting request 1129022 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1129022
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=32
 2023-11-27 21:43:24 +00:00
Dirk Mueller
2c4d3a704c Accepting request 1128968 from home:jubalh:branches:filesystems 
- Update to 0.14.1:
  * file_scan: satisfy clang CFI
  * file_scan: walk_dir: improve rootfs support
  * dbfile: get_duplicate_files: exclude inlined files before
    looking for duplicates
  * filerec: fiemap_scan_extent: fix typo: we must check the
    return from get_extent

OBS-URL: https://build.opensuse.org/request/show/1128968
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=68
 2023-11-27 06:36:16 +00:00
Ana Guerrero
9c6d4e28a8 Accepting request 1127570 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1127570
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=31
 2023-11-20 20:19:41 +00:00
Dirk Mueller
526ce195e2 Accepting request 1127541 from home:jubalh:branches:filesystems 
- Update to 0.14:
  * Batching has been reimplemented on top of the dedupe_seq.
  * The "scan" phase has been reimplemented (see 8264336 for details).
  * Filesystem locking has been implemented. See f3947e9 for details.

OBS-URL: https://build.opensuse.org/request/show/1127541
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=66
 2023-11-19 18:07:17 +00:00
Ana Guerrero
b2d8e26a80 Accepting request 1114344 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1114344
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=30
 2023-10-03 18:15:31 +00:00
David Sterba
550f46da46 Accepting request 1114307 from home:jubalh:branches:filesystems 
- Update to 0.13:
  * Add a new dedupe option: [no]rescan_files. It will increase
    performance in some use cases.
  * New behaviors from v0.12 has been consolidated. Extent-based
    lookup is always enabled, as is fiemap. The v2
    hashfile is no longer supported.
  * Hashfile are now updated after deduplication, to reflect the
    new physical offsets. This avoid (re)deduplicating extents
    in some cases.
  * Partial mode has been enhanced to support batching. The
    overall performance of this mode (which was previously known
    as "block-based mode") has been improved.
  * All files are now open in readonly mode.
  * Hashfile version has been increased to reflect the new
    database behaviors. Previous hashfiles are not compatible.
  * Always compute a hash for the entire file. This let us
    deduplicate same files easily, regardless of their
    extents mappings.
  * Deduplicating only parts of a file can be disabled using
    the [no]only_whole_files dedupe option.
  * Hashfiles with unsupported features or hash algorithm are
    now recreated transparently. Migration of the old content
    is not implemented.
  * Relative exclude patterns are no longer silently ingested.
    Such patterns are now rebuilt on top of the current
    working directory.
  * Batching is now set to 1024 by default.

OBS-URL: https://build.opensuse.org/request/show/1114307
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=64
 2023-09-29 12:59:48 +00:00
Dominique Leuenberger
7e215e7e8a Accepting request 1112021 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1112021
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=29
 2023-09-20 11:27:12 +00:00
David Sterba
d91e3b0cd3 Accepting request 1111318 from home:pgajdos 
- build for 15

OBS-URL: https://build.opensuse.org/request/show/1111318
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=62
 2023-09-18 15:20:47 +00:00
Dominique Leuenberger
965f6dc450 Accepting request 1099122 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/1099122
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=28
 2023-07-18 20:07:49 +00:00
David Sterba
da9b80fbb7 Accepting request 1099030 from home:jubalh:branches:filesystems 
- Update to 0.12:
  * Duplication lookup is now based on extents. This leads to a
    massive increase of the performances. Block-based lookup is
    still possible via --dedupe-options=partial.
  * Following that change, a new hashfile format has been introduced.
    Previous hashfile format is still supported when extents lookup
    are disabled, this is not recommended.
  * Batching has been implemented. When enabled with the -B <batchsize>
    option, duperemove will run the deduplication phase every <batchsize>
    scanned files. This is meant to help running duperemove on large
    dataset, with small blocksize, or on memory-constrained systems.
  * All hash algorithm has been removed and replaced by xxh128. This
    variant is as robust as murmur3 while being faster. Choosing a
    hash function via the --hash option has been removed. Hashfiles
    built with other algorithm must be removed.

OBS-URL: https://build.opensuse.org/request/show/1099030
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=60
 2023-07-17 16:36:57 +00:00
Dominique Leuenberger
14b6df6fee Accepting request 951385 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/951385
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=27
 2022-02-03 22:55:15 +00:00
Dirk Mueller
bd5f8f9365 Accepting request 951355 from home:jubalh:branches:filesystems 
- Update to 0.11.3:
  * Increase open file limit. (#269)
  * Create hash database file with 600 permission for improved
    security. (#262)
  * Read more data per pread, for v2 hashfile format this reduces
    the overall number of syscalls made which in turns results
    in better performance.
  * Fix truncated file handling, eliminating a an infinite
    loop case. (#255)

OBS-URL: https://build.opensuse.org/request/show/951355
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=59
 2022-02-03 21:01:17 +00:00
Dominique Leuenberger
2e2e1d2f0a Accepting request 846202 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/846202
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=26
 2020-11-05 20:55:38 +00:00
David Sterba
592fd6403a Accepting request 846138 from home:nikbor:branches:filesystems 
- Update to v0.11.2
  - Improved detection of hyperthreading
  - Multiple minor fixes and cleanups
  - Updated xxhash algorithm to latest release
  - Fixed extent collision handling
  - Fix overflow scanning in extents

OBS-URL: https://build.opensuse.org/request/show/846138
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=57
 2020-11-05 11:08:59 +00:00
Dominique Leuenberger
9b8665c95a Accepting request 819289 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/819289
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=25
 2020-07-08 17:15:27 +00:00
David Sterba
da158864ea Accepting request 814537 from home:pluskalm:branches:filesystems 
- Fix building with gcc10

OBS-URL: https://build.opensuse.org/request/show/814537
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=55
 2020-07-07 21:00:36 +00:00
Dominique Leuenberger
e059225fdc Accepting request 654118 from filesystems 
OBS-URL: https://build.opensuse.org/request/show/654118
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=24
 2018-12-05 08:46:17 +00:00
Mark Fasheh
9e08d2ffb2 Accepting request 654107 from home:mfasheh:branches:filesystems 
- Use source tarball from github

OBS-URL: https://build.opensuse.org/request/show/654107
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=53
 2018-12-04 22:04:12 +00:00
Mark Fasheh
35bcabda67 Accepting request 653159 from home:mfasheh:branches:filesystems 
- Update to v0.11.1
  - Add 'quiet' mode - duperemove will only print errors and a
    short summary of any dedupe.
  - Revert block dedupe default, it is causing excessive
    fragmentation on users systems.

OBS-URL: https://build.opensuse.org/request/show/653159
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=52
 2018-12-03 21:07:58 +00:00
Dirk Mueller
12617fe472 Accepting request 607705 from home:mfasheh:branches:filesystems 
- Update to v0.11
  - Fix a minor fd leak in hyperthreading detection code
  - Clean up some typos in documentation
  - Some build fixes
  - Removed patch: v0.11.beta4_to_git_f95e429.patch

OBS-URL: https://build.opensuse.org/request/show/607705
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=51
 2018-05-16 06:53:41 +00:00
Dominique Leuenberger
3180329c1a Accepting request 432126 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/432126
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=23
 2016-10-01 22:09:24 +00:00
Mark Fasheh
da623b2bae Accepting request 432087 from home:mfasheh:branches:filesystems 
- Update to v0.11.beta4 git commit f95e429
  * Use proper len of tail blocks during block dedupe
  * run_dedupe: avoid size_list corruption in push_blocks()
  - Added patch: v0.11.beta4_to_git_f95e429.patch

OBS-URL: https://build.opensuse.org/request/show/432087
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=49
 2016-10-01 04:52:10 +00:00
Dominique Leuenberger
079ea4be7f Accepting request 431135 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/431135
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=22
 2016-09-30 13:33:42 +00:00
Dirk Mueller
027db8aa5b Accepting request 430642 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.11.beta4
  - Many man page and FAQ updates
  - Much lower memory usage (changed to use block-dedupe by default)
  - removed unused sha256 module

OBS-URL: https://build.opensuse.org/request/show/430642
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=47
 2016-09-28 16:31:10 +00:00
Dominique Leuenberger
ff9ba08aeb Accepting request 428314 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/428314
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=21
 2016-09-17 12:41:00 +00:00
Mark Fasheh
fb71cfcf60 Accepting request 428313 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.11.beta3
  - Bugfixes
  - Large (2.5x) performance improvement to find dupes stage
  - Added '--cpu-threads' for user control of cpu bound tasks

OBS-URL: https://build.opensuse.org/request/show/428313
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=45
 2016-09-16 19:58:20 +00:00
Dominique Leuenberger
ef077e0fc1 Accepting request 426205 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/426205
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=20
 2016-09-10 22:55:05 +00:00
Mark Fasheh
75aeed8ea2 Accepting request 426198 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.11.beta2
  - Many bugfixes
  - Improved dedupe coverage with better extent selection algorithm
  - Improved (easier to read) output for csum stage
  - Progress counter added for dedupe stage
  - Removed patch v0.11.beta1_to_git_d4b9818.patch
  - Removed patch v0.11-d4b9818_to_82234f5.patch

OBS-URL: https://build.opensuse.org/request/show/426198
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=43
 2016-09-09 23:23:59 +00:00
Mark Fasheh
148f1a8c2f Accepting request 423277 from home:mfasheh:branches:filesystems 
- Update to v0.11.beta1 git commit 82234f5
  - Set file->dedupe_seq if we drop out in add_file_db()
  - add_file_db(): collapse all cases of ino/subvol/filename being
    different
  - Simplify the logic in add_file_db()
  - add_file_db(): check for same filename in filerec table
  - Store filerecs in a tree by filename

OBS-URL: https://build.opensuse.org/request/show/423277
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=42
 2016-08-26 22:19:57 +00:00
Mark Fasheh
c5fa21bf39 Accepting request 421850 from home:mfasheh:branches:filesystems 
- Update to git commit d4b9818:
  - Set scan flag on renamed file
  - Remove db record if a file was renamed
  - More vprints in add_file_db()
  - Fix confused error handling in dbfile_load_one_filerec()
  - Return 0 if user specified --help
  - Don't list '-?' as a valid option
  - add_file_db() Properly account for __add_file() skipping a file
  - Add hashstats to the man page

OBS-URL: https://build.opensuse.org/request/show/421850
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=41
 2016-08-24 19:46:51 +00:00
Dominique Leuenberger
756d038203 Accepting request 419114 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/419114
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=19
 2016-08-13 16:31:01 +00:00
Mark Fasheh
4262370af0 Accepting request 419113 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.11.beta1
  - Greatly increased performance during extent search
  - Hashfiles are reusable now, Duperemove will only scan files which have
    changed since the last time it was run.
  - Better selection of target extent during dedupe
    - reduces fragmentation of deduped space
  - Support for dedupe on xfs (experimental)

OBS-URL: https://build.opensuse.org/request/show/419113
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=39
 2016-08-12 23:32:36 +00:00
Dominique Leuenberger
72b1ad0b0f Accepting request 384429 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/384429
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=18
 2016-04-08 07:39:29 +00:00
Dirk Mueller
0b7d08b2bb Accepting request 383214 from home:pluskalm:branches:filesystems 
- Use pretty name for tarball
- Cleanup spec file with spec-cleaner
- Use pkgconfig style dependnencies

OBS-URL: https://build.opensuse.org/request/show/383214
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=37
 2016-04-05 08:54:46 +00:00
Dominique Leuenberger
2a6b6fa41d Accepting request 338310 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/338310
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=17
 2015-10-14 14:45:23 +00:00
Mark Fasheh
289d32af0b Accepting request 338308 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.10
  - Documentation updates for latest stable version
    - document --fdupes mode in manpage
    - update man page headers
    - update FAQ

OBS-URL: https://build.opensuse.org/request/show/338308
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=35
 2015-10-12 21:04:40 +00:00
Stephan Kulow
b976da28a8 Accepting request 317206 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/317206
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=16
 2015-07-19 09:45:46 +00:00
Mark Fasheh
076400119a Accepting request 317205 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.10.beta4
  - Better memory usage with hashstats utility

OBS-URL: https://build.opensuse.org/request/show/317205
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=33
 2015-07-16 22:05:12 +00:00
Stephan Kulow
cd21b83900 Accepting request 315979 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/315979
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=15
 2015-07-14 15:44:03 +00:00
Mark Fasheh
b5779bf30d Accepting request 315978 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.10.beta3
  - fixes many crashes in the dedupe stage
  - fixed an issue with our alignment function which would throw dedupe into
    a loop
  - improvements to output formatting
  - better logic when throwing out already-deduped extents won't give as
    many false positives

OBS-URL: https://build.opensuse.org/request/show/315978
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=31
 2015-07-10 20:05:07 +00:00
Mark Fasheh
fa628957a3 Accepting request 314546 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.10.beta2
  - multi-threaded dedupe stage (increased performance)
  - hashfile mode for drastically improved memory consumption
  - now uses murmur3 hash internally for improved cpu usage
  - can now take input from 'fdupes'

OBS-URL: https://build.opensuse.org/request/show/314546
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=30
 2015-06-30 17:05:51 +00:00
Dominique Leuenberger
1e184f173f Accepting request 294670 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/294670
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=14
 2015-04-07 07:30:10 +00:00
Mark Fasheh
3bc9c75563 Accepting request 294669 from home:mfasheh:branches:filesystems 
- Update with correct tarball (last one didn't have version number bump)

OBS-URL: https://build.opensuse.org/request/show/294669
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=28
 2015-04-06 19:47:44 +00:00
Mark Fasheh
83c2ffbc3f Accepting request 294667 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.09.3
  - Fix hang during file scan phase (bsc#926098)
- References: bsc#926098

OBS-URL: https://build.opensuse.org/request/show/294667
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=27
 2015-04-06 19:34:28 +00:00
Dominique Leuenberger
4e53d4b644 Accepting request 293348 from filesystems 
1

OBS-URL: https://build.opensuse.org/request/show/293348
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/duperemove?expand=0&rev=13
 2015-03-27 08:41:49 +00:00
Mark Fasheh
8642a0f974 Accepting request 293347 from home:mfasheh:branches:filesystems 
- Update to duperemove v0.09.2
  - Now compiles on SLE11 targets
  - Fallback to stat on filesystems that don't report file type from
    readdir
  - Don't assert on failed open during dedupe
- References: bsc#923277 bsc#924577

OBS-URL: https://build.opensuse.org/request/show/293347
OBS-URL: https://build.opensuse.org/package/show/filesystems/duperemove?expand=0&rev=25
 2015-03-26 23:13:54 +00:00