commit 6793951e8763531d5ad301a74cb2f0fd8815f94c42c63c726770800f00769950 Author: Dirk Mueller Date: Tue Jul 9 13:23:11 2024 +0000 - update to 272: * Move away from using DSA OpenSSH keys in tests; support has been removed in OpenSSH 9.8p1. * Move to assert_diff helper in test_openssh_pub_key.py * Update tests to support zipdetails version 4.004 shipped with Perl 5.40. * Drop apktool from Build-Depends; we can still test our APK code via autopkgtests. (Closes: #1071410) * Fix tests for 7zip version 24.05. * Add a versioned dependency for at least version 5.4.5 for the xz tests; they fail under (at least xz 5.2.8). * Relax Chris' versioned xz test dependency (5.4.5) to also allow version 5.4.1. * Include "xz --verbose --verbose" (ie. double --verbose) output, not just the single --verbose. (Closes: #1069329) * Only include "xz --list" output if the xz has no other differences. * Use "xz --list" to supplement the output when comparing .xz archives; essential when some underlying metadata differs. * Actually append the xz --list after the container differences, as it simplifies tests and the output. * Maintain an in-header boolean state to determine whether to drop from-file/to-file lines. This fixes an issue where HTML differences were being inadvertendly neglected. OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/diffoscope?expand=0&rev=96 diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..9b03811 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,23 @@ +## Default LFS +*.7z filter=lfs diff=lfs merge=lfs -text +*.bsp filter=lfs diff=lfs merge=lfs -text +*.bz2 filter=lfs diff=lfs merge=lfs -text +*.gem filter=lfs diff=lfs merge=lfs -text +*.gz filter=lfs diff=lfs merge=lfs -text +*.jar filter=lfs diff=lfs merge=lfs -text +*.lz filter=lfs diff=lfs merge=lfs -text +*.lzma filter=lfs diff=lfs merge=lfs -text +*.obscpio filter=lfs diff=lfs merge=lfs -text +*.oxt filter=lfs diff=lfs merge=lfs -text +*.pdf filter=lfs diff=lfs merge=lfs -text +*.png filter=lfs diff=lfs merge=lfs -text +*.rpm filter=lfs diff=lfs merge=lfs -text +*.tbz filter=lfs diff=lfs merge=lfs -text +*.tbz2 filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.ttf filter=lfs diff=lfs merge=lfs -text +*.txz filter=lfs diff=lfs merge=lfs -text +*.whl filter=lfs diff=lfs merge=lfs -text +*.xz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.zst filter=lfs diff=lfs merge=lfs -text diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..57affb6 --- /dev/null +++ b/.gitignore @@ -0,0 +1 @@ +.osc diff --git a/diffoscope-272.tar.bz2 b/diffoscope-272.tar.bz2 new file mode 100644 index 0000000..4db0d22 --- /dev/null +++ b/diffoscope-272.tar.bz2 @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:10574313c7f048ff9af2a67351bc18b6f96f6e61af7750aba19f64b3ccdb2a78 +size 3155265 diff --git a/diffoscope-272.tar.bz2.asc b/diffoscope-272.tar.bz2.asc new file mode 100644 index 0000000..51df2e8 --- /dev/null +++ b/diffoscope-272.tar.bz2.asc @@ -0,0 +1,16 @@ +-----BEGIN PGP SIGNATURE----- + +iQIzBAABCAAdFiEEwv5L0nHBObhsUz5GHpU+J9QxHlgFAmaIADgACgkQHpU+J9Qx +HljVSA//Ul9b0grmlcq4lrrCh7LEH7vvKunRd86po031QcWxZDUkzTM8XqHcOG2Y +48AJC+KdTGzzB7xPUDlRO7m7XBBG5AisOL3E9pfF3uzoaMD99INfyPdfJ7Fjo4K6 +Q/W7n2kiCzUls9FiwrwXgFlrbsYADPnJnSesjg/d52TA3+pzK1ACaDVvYDbUVSTd +lHO6tzqyyaqqSgepDpP+zzfPgmtf+EwrRDfLxA+IXjggeq7TgpFCPwvoeMo5a3Ee +oXmAHQC+YynDHcY83yuXQ0u2FWs64UP3Gigasa/4gClxx4h2lI4jVBWgKWagu9xT +dOehAl2RRLZT4rgicGiOKjvmnpqu9JMjLrBltPz10MzlgXZFvLZDPI+hB9FD0aBi +e+iGdmqxN75Wvo2utK41wKYKM5B7lZwtQGDBfcnScJezRKXbSz6P2ZIRI9rW1RS3 +UqZUrYjrs7RoPHbSy4RGRPINExpFhckDuEFyR10Q0BojGgAdZEqap9v7HwN4hjav +xMhP5OuhsuRfNKbBSvEe+e+PicgvH6Ywkb/ucOBBbGaBAwZ2V76+/5zf8+17eRSP +w5cClu2M8tlLt3MXl/++Qs7y6f2L5NME++7FijE6qr0mF7If+c1HyZPYUzLY4UJO +aiqR5H0365rz6f9fLW2WLrui8cpS+Db/OMPQ6CTw9cjWdjqLdS4= +=EHMm +-----END PGP SIGNATURE----- diff --git a/diffoscope.changes b/diffoscope.changes new file mode 100644 index 0000000..2bf3046 --- /dev/null +++ b/diffoscope.changes @@ -0,0 +1,2435 @@ +------------------------------------------------------------------- +Tue Jul 9 13:21:55 UTC 2024 - Dirk Müller + +- update to 272: + * Move away from using DSA OpenSSH keys in tests; support has + been removed in OpenSSH 9.8p1. + * Move to assert_diff helper in test_openssh_pub_key.py + * Update tests to support zipdetails version 4.004 shipped with + Perl 5.40. + * Drop apktool from Build-Depends; we can still test our APK + code via autopkgtests. (Closes: #1071410) + * Fix tests for 7zip version 24.05. + * Add a versioned dependency for at least version 5.4.5 for the + xz tests; they fail under (at least xz 5.2.8). + * Relax Chris' versioned xz test dependency (5.4.5) to also + allow version 5.4.1. + * Include "xz --verbose --verbose" (ie. double --verbose) + output, not just the single --verbose. (Closes: #1069329) + * Only include "xz --list" output if the xz has no other + differences. + * Use "xz --list" to supplement the output when comparing .xz + archives; essential when some underlying metadata differs. + * Actually append the xz --list after the container + differences, as it simplifies tests and the output. + * Maintain an in-header boolean state to determine whether to + drop from-file/to-file lines. This fixes an issue where HTML + differences were being inadvertendly neglected. + +------------------------------------------------------------------- +Mon Apr 22 21:12:38 UTC 2024 - Joshua Smith + +- Update to 265: + * Ensure that tests with ">=" version constraints actually print + the corresponding tool name. + (Closes: reproducible-builds/diffoscope#370) + * Prevent odt2txt tests from always being skipped due to an + impossibly new version requirement. + (Closes: reproducible-builds/diffoscope#369) + * Avoid nested parens-in-parens when printing "skipping…" + messages in the testsuite. + +- Update to 264: + * Don't crash on invalid zipfiles, even if we encounter 'badness' + halfway through the file. (Re: #1068705) + * Add note when there are duplicate entries in ZIP files. + (Closes: reproducible-builds/diffoscope!140) + * Add an external tool reference for GNU Guix for zipdetails. + +- Update to 263: + * Add support for the zipdetails(1) tool included in the Perl + distribution. + * Don't use parenthesis within test "skipping…" messages; PyTest + adds its own parenthesis, so we were ending up with double + nested parens. + * Fix the .epub tests after supporting zipdetails(1). + * Update copyright years and debian/tests/control. + * Fix MozillaZipContainer's monkeypatch after Python's zipfile + module changed to detect potentially insecure overlapping + entries within .zip files. + (Closes: reproducible-builds/diffoscope#362) + +- Update to 262: + * Factor out Python version checking in test_zip.py. (Re: #362) + * Also skip some zip tests under 3.10.14 as well; a potential + regression may have been backported to the 3.10.x series. The + underlying cause is still to be investigated. (Re: #362) + +------------------------------------------------------------------- +Sun Mar 24 17:59:20 UTC 2024 - Dirk Müller + +- update to 261 (bsc#1220157, CVE-2024-25711): + * Don't crash if we encounter an .rdb file without an + equivalent .rdx file. + * Actually test 7z support in the test_7z set of tests, not the + lz4 functionality. + * In addition, correctly check for the 7z binary being + available (and not lz4) when testing 7z. + * Prevent a traceback when comparing a contentful .pyc file + with an empty one. (Re: Debian:#1064973) + * Don't error-out with a traceback if we encounter + "struct.unpack"-related errors when parsing .pyc files. + * Fix compatibility with PyTest 8.0. + * Don't try and compare rdb_expected_diff on non-GNU systems as + %p formatting can vary. + * Use the 7zip package (over p7zip-full) after package + transition. + * Parse the header and hunksize of diffs strictly before + parsing the context below. + * Reformat code to comply with the latest version of Black + (24.1.1). + * Expand the previous changelog entry to include the CVE number + that was subsequently assigned. + * Bump the miniumum Black requirement to run the "Black clean" + test and make test_zip.py Black clean. + * CVE-2024-25711: Use a determistic name when extracting + content from GPG artifacts instead of trusting the value of + gpg's --use- embedded-filenames. + * Temporarily fix support for Python 3.11.8 re. a potential + regression with the handling of ZIP files. + * Add/fix compatibility for Python progressbar 2.5 & 3.0 etc. + * Add external tool on GNU Guix for 7z. + * Improve DOS/MBR extraction by adding support for 7z. + * Process objdump symbol comment filter inputs as the Python + "bytes" type (and not str). (Closes: reproducible- + builds/diffoscope#358) + * Add a missing RequiredToolNotFound import. + * Update copyright years. + * As UI/UX improvement, try and avoid printing an extended + traceback if diffoscope runs out of memory. This may not + always be possible to detect. + * Mark diffoscope as stable in setup.py (for PyPI.org). + Whatever diffoscope is, at least, not "alpha" anymore. + +------------------------------------------------------------------- +Wed Nov 1 17:44:17 UTC 2023 - Andrea Manzini + +- removed patch fix-file-5.45.patch as included in upstream + +- Update to version 251: + * If the equivalent of `file -i` returns text/plain, fallback to comparing + this file as a text file. This especially helps when file(1) miscategorises + text files as some esoteric type. + +- Update to version 250: + * Fix compatibility with file 5.45. + * Add external tool references for GNU Guix (for html2text and ttx). + +- Update to version 249: + * Add specialize_as() method, and use it to speed up .smali comparison in APKs. + * Add documentation for the new specialize_as, and expand the documentation + of `specialize` too. + * Correct typos in diffoscope/presenters/utils.py. + +- Update to version 246: + * Add support for LLVM 16. + +- Update to version 244: + * Address compatibility with python-libarchive-c version 5. + * Testsuite changes + +- Update to version 243: + * Improve the documentation on to produce that binary blob that in the arsc comparator. + +------------------------------------------------------------------- +Sat Aug 5 07:58:48 UTC 2023 - Sebastian Wagner + +- Add fix-file-5.45.patch to fix compatibility with file version 5.45 + https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/346 +- update to version 242: + - If the binwalk Python module is not available, ensure the user knows they + may be missing more differences in, for example, concatenated .cpio + archives. + - Factor out routine to generate a human-readable comments when + Python modules are missing. + +------------------------------------------------------------------- +Wed May 3 19:25:34 UTC 2023 - Dirk Müller + +- update to 241: + * Add a missing 'raise' statement + * document sending out an email upon release + * Drop the use of include_package_data=True in setup.py. + * Fix compatibility with pypdf 3.x, and correctly restore test + data. + * Rework PDF annotations processing into a separate method. + +------------------------------------------------------------------- +Fri Mar 3 08:53:06 UTC 2023 - Dirk Müller + +- update to 238: + * autopkgtest: fix tool name in the skippable list. + * autopkgtest: only install aapt and dexdump on architectures + where they are available. + * Drop backward compatibility assignment. + * Fix flake warnings, potentially reinstating PyPDF 1.x + support (untested). + * Update code to match latest version of Black. + * Require at least Black version 23.1.0 to run the internal + Black tests. + * Update copyright years. + * Fix compatibility with PyPDF2. + * Fix compatibility with ImageMagick 7.1. + * Update from PyPDF2 to pypdf. + * Add support for Android resources.arsc files. + * Add support for dexdump. + * Improve DexFile's FILE_TYPE_RE and add + FILE_TYPE_HEADER_PREFIX, and remove + "Dalvik dex file" from ApkFile's FILE_TYPE_RE as well. + * Update external tool for isoinfo on guix. + +------------------------------------------------------------------- +Sat Jan 21 11:18:35 UTC 2023 - Dirk Müller + +- update to 233: + * Split packaging metadata into an extras_require.json file instead of using + the pep517 and the pip modules directly. This was causing build failures if + not using a virtualenv and/or building without internet access. + * Add an external tool reference for GNU Guix (lzip). + * Drop an external tool reference for GNU Guix (pedump). + * Split inline Python code in shell script to generate test dependencies to a + separate Python script. + * No need for "from __future__ import print_function" import in setup.py + anymore. + * Comment and tidy the new extras_require.json handling. + +------------------------------------------------------------------- +Wed Jan 18 10:05:17 UTC 2023 - Daniel Garcia + +- Update to version 232: + * Allow ICC tests to (temporarily) fail. + * Update debian/tests/control after the addition of PyPDF 3 support. + * Update regular expression for Android .APK files. + * Support PyPDF version 3. +- 231: + * Improve "[X] may produce better output" messages. Based on a patch by + Helmut Grohne. (Closes: #1026982) +- 230: + * Fix compatibility with file(1) version 5.43; thanks, Christoph Biedl. + * Support Berkeley DB version 6. +- 229: + * Skip test_html.py::test_diff if html2text is not installed. + (Closes: #1026034) + * Bump standards version to 4.6.2, no changes needed. +- 228: + * As an optimisation, don't run apktool if no differences are detected before + the signing block. (Closes: reproducible-builds/diffoscope!105) + * Support both the python3-progressbar and python3-progressbar2 Debian + packages, two modules providing the "progressbar" Python module. + (Closes: reproducible-builds/diffoscope#323) + * Ensure we recommend apksigcopier. (Re: reproducible-builds/diffoscope!105) + * Make the code clearer around generating the Debian substvars and tidy + generation of os_list. + * Update copyright years. +- 227: + * Don't attempt to attach text-only differences notice if there are no + differences to begin with. (Closes: #1024171, #1024349) + * Don't run Python decompiling tests on Python bytecode that both file(1) + cannot yet detect and Python 3.11 cannot demarshall. (Closes: #1024335) +- 226: + * Add an lzip comparator with tests. + * Add support for comparing the "text" content of HTML files using html2text. + (Closes: #1022209, reproducible-builds/diffoscope#318) + * Misc/test improvements: + * Drop the ALLOWED_TEST_FILES test; it's mostly just annoying. + * Drop other copyright notices from lzip.py and test_lzip.py. + * Use assert_diff helper in test_lzip.py. + * Pylint tests/test_source.py. + * Add lzip to debian dependencies. +- 225: + * Add support for detecting ordering-only differences in XML files. + (Closes: #1022146) + * Fix an issue with detecting ordering differences. (Closes: #1022145) + * Add support for ttx(1) from fonttools. + (Re: reproducible-builds/diffoscope#315) + * Test improvements: + - Tidy up the JSON tests and use assert_diff over get_data and manual + assert in XML tests. + - Rename order1.diff to json_expected_ordering_diff for consistency. + - Temporarily allow the stable-po pipeline to fail in the CI. + * Use consistently capitalised "Ordering" everywhere we use the word in + diffoscope's output. +- 224: + * Fix rlib test failure with LLVM 15. Thanks to Gianfranco Costamagna + (locutusofborg) for the patch. +- 223: + * The cbfstools utility is now provided in Debian via the coreboot-utils + Debian package, so we can enable that functionality within Debian. + (Closes: #1020630) + * Also include coreboot-utils in Build-Depends and Test-Depends so it is + available for the tests. + * Add support for file 5.43. +- 222: + * Use pep517 and pip to load the requirements. (Closes: #1020091) + * Remove old Breaks/Replaces in debian/control that have been obsoleted since + bullseye + +------------------------------------------------------------------- +Thu Sep 22 17:01:08 UTC 2022 - Daniel Garcia + +- update to version 221: + - Don't crash if we can open a PDF file with PyPDF but cannot parse the + annotations within. (Closes: reproducible-builds/diffoscope#311) + - Depend on the dedicated xxd package, not vim-common. + - Update external_tools.py to reflect xxd/vim-common change. + +- Remove test_python.test_diff because it's failing, it's not recognizing the + pyc file type with the tests fixtures +https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/284#note_280561 + +------------------------------------------------------------------- +Wed Aug 17 07:16:08 UTC 2022 - Sebastian Wagner + +- update to version 220: + - Support Haskell 9.x series files and update the test files to match. Thanks + to Scott Talbert for the relevant info about the new format. + (Closes: reproducible-builds/diffoscope#309) + - Fix a regression introduced in diffoscope version 207 where diffoscope + would crash if one directory contained a directory that wasn't in the + other. Thanks to Alderico Gallo for the report and the testcase. + (Closes: reproducible-builds/diffoscope#310) +- update to version 219: + - Don't traceback if we encounter an invalid Unicode character in Haskell + versioning headers. (Closes: reproducible-builds/diffoscope#307) + - Update various copyright years. +- update to version 218: + - Improve output of Markdown and reStructuredText to use code blocks with + syntax highlighting. (Closes: reproducible-builds/diffoscope#306) +- update to version 217: + - Update test fixtures for GNU readelf 2.38 (now in Debian unstable). + - Be more specific about the minimum required version of readelf (ie. + binutils) as it appears that this "patch" level version change resulted in + a change of output, not the "minor" version. (Closes: #1013348) + - Don't leak the (likely-temporary) pathname when comparing PDF documents. + +------------------------------------------------------------------- +Tue Jun 21 09:09:17 UTC 2022 - Sebastian Wagner + +- update to version 216: + - Print profile output if we were called with --profile and we receive a + TERM signal. + - Emit a warning if/when we are handling a TERM signal. + - Clarify in the code in what situations the main "finally" block gets + called, especially in relation to handling TERM signals. + - Clarify and tidy some unconditional control flow in diffoscope.profiling. +- update to version 215: + - Bug fixes: + - Also catch IndexError (in addition to ValueError) when parsing .pyc + files. (Closes: #1012258) + - Strip "sticky" etc. from "x.deb: sticky Debian binary package […]". + Thanks to David Prévot for the report. (Closes: #1011635) + - Correctly package diffoscope's scripts/ directory, fixing the extraction + of vmlinuz kernel images. (Closes: reproducible-builds/diffoscope#305) + - Correct the logic for supporting different versions of argcomplete in + debian/rules. + - New features: + - Support both PyPDF 1.x and 2.x. + - Codebase improvements: + - Don't call re.compile and then call .sub on the result; just call + re.sub directly. + - Clarify the logic around the difference between --usage and --help. + - Testsuite improvements: + - Integrate test coverage with GitLab's concept of artifacts. + - Re-enable Gnumeric tests as its now available again. + - Test --help and --usage, and additionally test that --help includes + the programmatically-generated file format list as well. + - Bump Standards Version field in debian/control to 4.6.1. + +------------------------------------------------------------------- +Fri May 27 06:33:01 UTC 2022 - Sebastian Wagner + +- update to version 214: + - Support both python-argcomplete 1.x and 2.x. + - Add external tool on GNU Guix for xb-tool. + - Use assert_diff in test_zip over get_data and separate assert. + +------------------------------------------------------------------- +Sun May 22 10:16:02 UTC 2022 - Sebastian Wagner + +- update to version 213: + - Don't mask differences in .zip/.jar central directory extra fields. + - Don't show a binary comparison of .zip/.jar files if we have at least + one observed nested difference. + - Use assert_diff in test_zip over get_data and separate assert. +- update to version 212: + - Add support for extracting vmlinuz/vmlinux Linux kernel images. + (Closes: reproducible-builds/diffoscope#304) + - Some Python .pyc files report as "data", so support ".pyc" as a + fallback extension. + +------------------------------------------------------------------- +Sat May 7 19:59:00 UTC 2022 - Ferdinand Thiessen + +- Update to version 211: + * Drop mplayer from the Build-Depends, it's not needed. + +------------------------------------------------------------------- +Tue Apr 19 14:29:46 UTC 2022 - Sebastian Wagner + +- update to version 210: + - Make sure that PATH is properly mangled for all diffoscope actions, not + just when running comparators. + +------------------------------------------------------------------- +Fri Apr 1 19:16:51 UTC 2022 - Sebastian Wagner + +- update to version 209: + - Update R test fixture for R 4.2.x series. (Closes: #1008446) + - Update minimum version of Black to prevent test failure on Ubuntu jammy. + +------------------------------------------------------------------- +Fri Mar 25 21:21:18 UTC 2022 - Sebastian Wagner + +- update to version 208: + - Add graceful handling for UNIX sockets and named pipes. + (Closes: reproducible-builds/diffoscope#292) + - Remove a superfluous log message and reformatt comment lines. + - Reformat various files to satisfy current version of Black. + +------------------------------------------------------------------- +Fri Mar 4 18:59:10 UTC 2022 - Sebastian Wagner + +- update to version 207: + - Fix a gnarly regression when comparing directories against non-directories. + (Closes: reproducible-builds/diffoscope#292) + - Use our assert_diff utility where we can within test_directory.py + +------------------------------------------------------------------- +Fri Feb 25 20:08:22 UTC 2022 - Sebastian Wagner + +- update to version 206: + - Also allow "Unicode text, UTF-8 text" as well as "UTF-8 Unicode text" to + match for .buildinfo files too. + - Add a test for recent file(1) issue regarding .changes files. + (Re: reproducible-builds/diffoscope#291) + - Drop "_PATH" suffix from some module-level globals that are not paths. + +------------------------------------------------------------------- +Mon Feb 21 11:21:46 UTC 2022 - Dirk Müller + +- update to 205: + * Fix a file(1)-related regression where .changes files that contained + non-ASCII text were not identified as being .changes files, resulting in + seemingly arbitrary packages on tests.reproducible-builds.org and elswhere + not comparing the package at all. The non-ASCII parts could have been in + the Maintainer or in the upload changelog, so we were effectively + penalising anyone outside of the Anglosphere. + (Closes: reproducible-builds/diffoscope#291) + * Don't print a warning to the console regarding NT_GNU_BUILD_ID changes in + ELF binaries. + +------------------------------------------------------------------- +Fri Feb 11 19:14:39 UTC 2022 - Sebastian Wagner + +- update to version 204: + - Don't run the binwalk comparator tests as root (or fakeroot) as the + latest version of binwalk has some security protection against doing + precisely this. + - If we fail to scan a file using binwalk, return 'False' from + BinwalkFile.recognizes rather than raise a traceback. + - If we fail to import the Python "binwalk" module, don't accidentally report + that we are missing the "rpm" module instead. + - Use dependencies to ensure that "diffoscope" and "diffoscope-minimal" + packages always have the precise same version. + +------------------------------------------------------------------- +Fri Feb 4 16:51:28 UTC 2022 - Sebastian Wagner + +- update to version 203: + - Improve documentation for --timeout due to a few misconceptions. + Add an allowed-to-fail test regarding a regression in directory handling. + - Tidy control flow in Difference._reverse_self a little. + - Fix diffing CBFS names that contain spaces. + +------------------------------------------------------------------- +Fri Jan 28 16:24:38 UTC 2022 - Sebastian Wagner + +- update to version 202: + - Don't fail if comparing a nonexistent file with a .pyc file (and add test). + (Closes: #1004312) + - Drop a reference in the manual page which claims the ability to compare + non-existent files on the command-line. This has not been possible since + version 32 which was released in September 2015. (Closes: #1004182) + - Add experimental support for incremental output support with a timeout. + Passing, for example, --timeout=60 will mean that diffoscope will not + recurse into any sub-archives after 60 seconds total execution time has + elapsed and mark the diff as being incomplete. (Note that this is not a + fixed/strict timeout due to implementation issues.) + (Closes: reproducible-builds/diffoscope#301) + - Don't return with an exit code of 0 if we encounter device file such as + /dev/stdin with human-readable metadata that matches literal, non-device, + file contents. (Closes: #1004198) + - Correct a "recompile" typo. + - Fix/update whitespace for Black 21.12. + +------------------------------------------------------------------- +Fri Jan 21 16:48:00 UTC 2022 - Sebastian Wagner + +- update to version 201: + - If the debian.deb822 module raises any exception on import, re-raise it as + an ImportError instead. This should fix diffoscope on some Fedora systems. + Thanks to Mattia Rizzolo for suggesting this particular solution. + (Closes: reproducible-builds/diffoscope#300) + - Fix json detection with file-5.41-3.fc36.x86_64. + +------------------------------------------------------------------- +Fri Jan 14 09:22:48 UTC 2022 - Sebastian Wagner + +- update to version 200: + - Even if a Sphinx .inv inventory file is labelled "The remainder of this + file is compressed using zlib", it might not actually be. In this case, + don't traceback, and simply return the original content. + (Closes: reproducible-builds/diffoscope#299) + - Update "X has been modified after NT_GNU_BUILD_ID has been applied" message + to, for instance, not duplicating the full filename in the primary + diffoscope's output. + +------------------------------------------------------------------- +Fri Jan 7 09:00:17 UTC 2022 - Sebastian Wagner + +- update to version 199: + - Support both variants of "odt2txt", including the one provided by unoconv. + (Closes: reproducible-builds/diffoscope#298) + - Add external tool reference on Arch Linux for xb-tool. + +------------------------------------------------------------------- +Mon Jan 3 11:42:21 UTC 2022 - Sebastian Wagner + +- update to version 198: + - Support showing "Ordering differences only" within .dsc field values. + (Closes: #1002002, reproducible-builds/diffoscope#297) + - Support OCaml versions 4.11, 4.12 and 4.13. (Closes: #1002678) + - Add support for XMLb files. (Closes: reproducible-builds/diffoscope#295) + - Also add, for example, /usr/lib/x86_64-linux-gnu to our internal PATH. + - Also recognize "GnuCash file" files as XML. + +------------------------------------------------------------------- +Fri Dec 17 11:59:51 UTC 2021 - Sebastian Wagner + +- update to version 197: + - Drop unnecessary has_same_content_as logging calls. + - Ignore the new "binary-with-bad-dynamic-table" Lintian tag. + - Support pgpdump 0.34 in the tests. Thanks to Michael Weiss + for reporting and testing the fix. + +------------------------------------------------------------------- +Fri Dec 10 16:35:04 UTC 2021 - Sebastian Wagner + +- update to version 196: + - Add a comment/annotation when the GNU_BUILD_ID field has been modified. + - Fix the "Black" version detection. + - Replace "token" with anonymous variable "x" in order to remove extra lines. + +------------------------------------------------------------------- +Sun Dec 5 19:18:58 UTC 2021 - Sebastian Wagner + +- update to version 195: + - Don't use the runtime platform's native endianness when unpacking .pyc + files to fix test failures on big-endian machines. + +------------------------------------------------------------------- +Sun Nov 28 07:31:54 UTC 2021 - Sebastian Wagner + +- update to version 194: + - Don't traceback when comparing nested directories with non-directories. + (Closes: reproducible-builds/diffoscope#288) + +------------------------------------------------------------------- +Sat Nov 20 13:01:25 UTC 2021 - Sebastian Wagner + +- update to version 193: + - Don't duplicate file lists at each directory level. + (Closes: #989192, reproducible-builds/diffoscope#263) + - When pretty-printing JSON, mark the difference as such, additionally + avoiding including the full path. + (Closes: reproducible-builds/diffoscope#205) + - Codebase improvements: + - Update a bunch of %-style string interpolations into f-strings or + str.format. + - Import itertools top-level directly. + - Drop some unused imports. + - Use isinstance(...) over type(...) == + - Avoid aliasing variables if we aren't going to use them. + - Fix missing diff output on large diffs. + - Ignore a Python warning coming from a dependent library (triggered by + supporting Python 3.10) + - Document that support both Python 3.9 and 3.10. + +------------------------------------------------------------------- +Sun Nov 14 21:14:04 UTC 2021 - Sebastian Wagner + +- update to version 192: + - Update .epub test methodology after improving XML file parsing. +- update to version 191: + - Detect XML files as XML files if either file(1) claims if they are XML + files, or if they are named .xml. + (Closes: #999438, reproducible-builds/diffoscope#287) + - Don't reject Debian .changes files if they contain non-printable + characters. (Closes: reproducible-builds/diffoscope#286) + - Continue loading a .changes file even if the referenced files inside it do + not exist, but include a comment in the diff as a result. + - Log the reason if we cannot load a Debian .changes file. + - Fix inverted logic in the assert_diff_startswith() utility. +- update to version 190: + - Don't raise a traceback if we cannot de-marshal Python bytecode to support + Python 3.7 loading newer .pyc files. + (Closes: reproducible-builds/diffoscope#284) + - Fix Python tests under Python 3.7 with file 5.39+. + - Skip Python bytecode testing when "file" is older than 5.39. + - Detect whether the GNU_BUILD_ID field has been modified. + +------------------------------------------------------------------- +Tue Nov 2 19:29:18 UTC 2021 - Sebastian Wagner + +- update to version 189: + - Try some alternative suffixes (eg. ".py") to support distributions that + strip or retain them. (Closes: reproducible-builds/diffoscope#283) + - Skip Python bytecode testing where we do not have an expected diff. + (Closes: reproducible-builds/diffoscope#284) + - Refactor the find_executable utility into an explicit method. + - Split out a custom call to assert_diff to support a .startswith equivalent. + - Use skipif instead of manual conditionals in some tests. + - Add an external tool reference for Guix to support ppudump and dumppdf. + - Update uImage test output for file(1) version 5.41. + - Add Arch Linux as CI test target. + - Add external tools on Arch Linux for ffmpeg, openssl and ocalobjinfo. + +------------------------------------------------------------------- +Fri Oct 22 08:44:31 UTC 2021 - Sebastian Wagner + +- update to version 188: + - Add support for Python Sphinx inventory files, usually named objects.inv. + - Fix Python bytecode decompilation tests with Python 3.10+. + (Closes: reproducible-builds/diffoscope#278) + +------------------------------------------------------------------- +Sat Oct 9 10:24:44 UTC 2021 - Sebastian Wagner + +- update to version 187: + - Add support for comparing .pyc files. Thanks to Sergei Trofimovich. + (Closes: reproducible-builds/diffoscope#278) + +------------------------------------------------------------------- +Fri Oct 1 18:33:10 UTC 2021 - Sebastian Wagner + +- update to version 186: + - Don't call close_archive when garbage-collecting Archive instances unless + open_archive returned successfully. This prevents, amongst others, an + AttributeError traceback due to PGPContainer's cleanup routines assuming + that its temporary directory had been created. + (Closes: reproducible-builds/diffoscope#276) + - Ensure that the string "RPM archives" exists in the package description, + regardless of whether python3-rpm is installed or not at build time. + - Fix the LVM Macho comparator for non-x86-64 architectures. + +------------------------------------------------------------------- +Fri Sep 24 19:34:48 UTC 2021 - Sebastian Wagner + +- update to version 185: + - Fix the autopkgtest in order to fix testing migration: the androguard + Python module is not in the python3-androguard Debian package + - Ignore a warning in the tests from the h5py package that doesn't concern + diffoscope. + - Bump Standards-Version to 4.6.0. + +------------------------------------------------------------------- +Sat Sep 18 20:15:56 UTC 2021 - Sebastian Wagner + +- update to version 184: + - Fix the semantic comparison of R's .rdb files after a refactoring of + temporary directory handling in a previous version. + - Support a newer format version of R's .rds files. + - Update tests for OCaml 4.12. (Closes: reproducible-builds/diffoscope#274) + - Move diffoscope.versions to diffoscope.tests.utils.versions. + - Use assert_diff in tests/comparators/test_rdata.py. + - Reformat various modules with Black. + - Stop using the deprecated distutils module by adding a version + comparison class based on the RPM version rules. + - Update invocations of llvm-objdump for the latest version of LLVM. + - Adjust a test with one-byte text file for file(1) version 5.40. + - Improve the parsing of the version of OpenSSH. + - Add a --diff-context option to control the unified diff context size. + (reproducible-builds/diffoscope!88) + +------------------------------------------------------------------- +Sat Sep 4 17:53:01 UTC 2021 - Sebastian Wagner + +- update to version 183: + - Add support for extracting Android signing blocks. + (Closes: reproducible-builds/diffoscope#246) + - Format debug messages for elf sections using our + diffoscope.utils.format_class utility. + - Clarify a comment about the HUGE_TOOLS dict in diffoscope.external_tools. + - Clarify output around APK Signing Blocks and remove an accidental duplicate + "0x" prefix. + +------------------------------------------------------------------- +Sun Aug 29 09:41:54 UTC 2021 - Sebastian Wagner + +- update to version 182: + - Also ignore, for example, spurious "fwGCC: (Debian ... )" lines in output + from strings(1). + - Only use "java -jar /path/to/apksigner.jar" if we have a .jar as newer + versions of apksigner use a shell wrapper script which will obviously be + rejected by the JVM. Also mention in the diff if apksigner is missing. + - Pass "-f" to apktool to avoid creating a strangely-named subdirectory and + to simplify code. + - If we specify a suffix for temporary file or directory, ensure it starts + with a "_" to make the generated filenames more human-readable. + - Drop an unused File import. + - Update the minimum version of the Black source code formatter. + - Support parsing the return value of squashfs versions which discriminate + between fatal and non-fatal errors. + +------------------------------------------------------------------- +Fri Aug 20 19:49:53 UTC 2021 - Sebastian Wagner + +- update to version 181: + - New features and bug fixes: + - Don't require apksigner in order to compare .apk files using apktool. + - Add a special-case to squshfs image extraction to not fail if we aren't + root/superuser. (Closes: #991059) + - Reduce the maximum line length to avoid O(n^2) Wagner-Fischer algorithm, + which meant that diff generation took an inordinate amount of time. + (Closes: reproducible-builds/diffoscope#272) + - Include profiling information in --debug output if --profile is not set. + - Don't print an orphan newline when the Black source code formatter + self-test passes. + - Tests: + - Update test to check specific contents of squashfs listing, otherwise it + fails depending on the test systems uid-to-username mapping in passwd(5). + - Assign "seen" and "expected" values to local variables to improve + contextual information in/around failed tests. + - Misc changes: + - Print the size of generated HTML, text (etc.) reports. + - Profile calls to specialize and diffoscope.diff.linediff. + - Update various copyright years. + +------------------------------------------------------------------- +Sun Aug 8 18:02:08 UTC 2021 - Sebastian Wagner + +- update to version 180: + - Don't include specific ".debug"-like lines in the output, as it invariably + a duplicate of the debug ID that exists in a better form in the readelf(1) + differences for this file. + - Also ignore include short "GCC" lines that differs on a single prefix byte + too. These are distracting, not very useful and are simply the strings(1) + command's idea of the build ID, which, again, is displayed nearby in the + file's diff. + - Update the invocation arguments and tests for the latest version of + odt2txt. + +------------------------------------------------------------------- +Fri Jul 30 08:18:51 UTC 2021 - Sebastian Wagner + +- update to version 179: + - Ensure that various LLVM tools are installed, even when testing whether + a MacOS binary has zero differences when compared to itself. + (Closes: reproducible-builds/diffoscope#270) + +------------------------------------------------------------------- +Sun Jul 18 08:41:11 UTC 2021 - Sebastian Wagner + +- update to version 178: + - Don't traceback on an broken symlink in a directory. + (Closes: reproducible-builds/diffoscope#269) + - Rewrite the calculation of a file's "fuzzy hash" to make the control + flow cleaner. + - Support .deb package members compressed with the Zstandard algorithm. + (LP: #1923845) + - Overhaul the Mach-O executable file comparator. + - Implement tests for the Mach-O comparator. + - Switch to new argument format for the LLVM compiler. + - Fix test_libmix_differences in testsuite for the ELF format. + - Improve macOS compatibility for the Mach-O comparator. + - Add llvm-readobj and llvm-objdump to the internal EXTERNAL_TOOLS data + structure. + - Invoke gzip(1) with the short option variants to support Busybox's gzip. + +------------------------------------------------------------------- +Tue Jun 8 10:29:34 UTC 2021 - Ferdinand Thiessen + +- Update to version 177: + * Improve support for Apple "provisioning profiles". + +------------------------------------------------------------------- +Sat May 29 07:45:45 UTC 2021 - Sebastian Wagner + +- Update ffmpeg tests to work with ffmpeg 4.4. + (Closes: reproducible-builds/diffoscope#258) + +------------------------------------------------------------------- +Fri May 14 16:24:12 UTC 2021 - Sebastian Wagner + +- update to version 175: + * Use the actual filesystem path name (instead of diffoscope's concept of the + source name) to correct APK filename filtering when an APK file is in + another container -- we need to filter the auto-generated "1.apk" instead + of "original-filename.apk". (Closes: reproducible-builds/diffoscope#255) + * Don't call os.path.basename twice. + * Correct grammar in a fsimage.py debug message. + * Add a comment about stripping filenames. + +------------------------------------------------------------------- +Fri May 7 11:27:26 UTC 2021 - Sebastian Wagner + +- remove fix-file-5.40.patch, merged upstream +- update to version 174: + * Check that we are parsing an actual Debian .buildinfo file, not just + a file with that extension. + (Closes: deb#987994, reproducible-builds/diffoscope#254) + * Support signed .buildinfo files again -- file(1) reports them as + "PGP signed message". + * Make the testsuite pass with file(1) version 5.40. + * Embed some short test fixtures in the test code itself. + * Fix recognition of compressed .xz files with file(1) 5.40. + +------------------------------------------------------------------- +Sun May 2 08:38:43 UTC 2021 - Sebastian Wagner + +- remove fix-tests-libmix_differences-2.patch, merged upstream +- remove fix-tests-libmix_differences.patch, merged upstream +- added fix-file-5.40.patch +- update to version 173 + * Add support for showing annotations in PDF files. + (Closes: reproducible-builds/diffoscope#249) + * Move to assert_diff in test_pdf.py. + * Difference.__init__: Demote unified_diff argument to a Python "kwarg". +- update to version 172 + * If zipinfo(1) shows a difference but we cannot uncover a difference within + the underlying .zip or .apk file, add a comment and show the binary + comparison. (Closes: reproducible-builds/diffoscope#246) + * Make "error extracting X, falling back to binary comparison E" error + message nicer. +- update to version 171 + * Do not list as a "skipping reason" tools that do exist. + * Drop the "compose" tool from the list of required tools for these tests, + since it doesn't seem to be required. +- update to version 170 + * Avoid frequent long lines in RPM header outputs that cause very very slow + HTML outputs. (Closes: reproducible-builds/diffoscope#245) + * Fix test_libmix_differences on openSUSE Tumbleweed. + (Closes: reproducible-builds/diffoscope#244) + * Move test_rpm to use the assert_diff utility helper. + * Add a diffoscope.tools.get_tools() method to support programmatically + fetching Diffoscope's config. + * Become tolerant of malformed Debian .changes files. +- update to version 169 + * Optimisations: + - Use larger buffer/block sizes when extracting files from libarchive- + based archives. + - Use a much-shorter CSS class (instead of "diffponct") to dramatically + reduce uncompressed HTML output. + * Logging improvements: + - Don't emit "Unable to stat file" warning/debug messages; we have + entirely-artificial directory entries such as ELF sections which, of + course, never exist as filesystem files. + - Don't emit a "Returning a FooContainer" logging message - we already emit + "Instantiating a FooContainer" one and are unlikely to fail in the + middle. + - Format the report size logging messages when generating HTML reports. + - Add the target directory when logging which directory we are extracting + containers to. + * Miscellaneous: + - Ignore "--debug" and similar arguments when creating a (hopefully useful) + temporary directory. + - Ensure all internal temporary directories have useful names. + - Clarify a comment regarding diffoscope not extracting excluded files. + * Skip a DEX-related test if the "procyon" tool is unavailable. + +------------------------------------------------------------------- +Tue Mar 16 09:20:53 UTC 2021 - Sebastian Wagner + +- add fix-tests-libmix_differences-2.patch to fix a newline in the other upstream patch + +------------------------------------------------------------------- +Mon Mar 15 21:08:35 UTC 2021 - Sebastian Wagner + +- activate test_differences test, as it is fixed in openSUSE TW. +- use fix-tests-libmix_differences.patch based on upstream patch and activate test_limix_differences tests. + +------------------------------------------------------------------- +Fri Mar 12 08:03:39 UTC 2021 - Sebastian Wagner + +- Disable failing tests explicitly instead of ignoring the exit code. + +------------------------------------------------------------------- +Sat Feb 27 15:00:17 UTC 2021 - Sebastian Wagner + +- Update dependencies +- Update to version 168: + * Don't call difflib.Differ.compare with very large inputs; it is at least + O(n^2) and makes diffoscope appear to hang. + (Closes: reproducible-builds/diffoscope#240) + * Don't use "Inheriting PATH of X" in debug log message; use "PATH is X". + * Correct the capitalisation of jQuery. +- Update to version 167: + * Temporary directory handling: + - Ensure we cleanup our temporary directory by avoiding confusion between + the TemporaryDirectory instance and the underlying directory. + (Closes: deb#981123) + - Use a potentially-useful suffix to our temporary directory based on the + command-line passed to diffoscope. + - Fix some tempfile/weakref interaction in Python 3.7 (ie. Debian buster). + (Closes: reproducible-builds/diffoscope#239) + - If our temporary directory does not exist anymore (eg. it has been + cleaned up in tests, signal handling or reference counting), make sure + we recreate it. + * Bug fixes: + - Don't rely on magic.Magic(...) to have an identical API between file's + magic.py and PyPI's "python-magic" library. + (Closes: reproducible-builds/diffoscope#238) + - Don't rely on dumpimage returning an appropriate exit code; check that + the file actually exists after we call it. + * Codebase changes: + - Set a default Config.extended_filesystem_attributes. + - Drop unused Config.acl and Config.xattr attributes. + - Tidy imports in diffoscope/comparators/fit.py. + * Tests: + - Add u-boot-tools to test dependencies so that salsa.debian.org pipelines + actually test the new FIT comparator. + - Strip newlines when determining Black version to avoid "requires black + >= 20.8b1 (18.9b0\n detected)" in test output (NB. embedded newline). + - Gnumeric is back in testing so re-add to test dependencies. + - Use assert_diff (over get_data, etc.) in the FIT and APK comparators. + - Mark test_apk.py::test_android_manifest as being allowed to fail for now. + - Fix the FIT tests in buster and unstable. +- Update to version 166: + [ Chris Lamb ] + * New features and bugfixes: + - Explicitly remove our top-level temporary directory. + (Closes: deb#981123, reproducible-builds/diffoscope#234) + - Increase fuzzy matching threshold to 130 ensure that we show more + differences. (Closes: reproducible-builds/diffoscope#232) + - Save our sys.argv in our top-level temporary directory in case it + helps debug current/errant temporary directories. + - Prefer to use "magic.Magic" over the "magic.open" compatibility + interface. (Closes: reproducible-builds/diffoscope#236) + - Reduce fuzzy threshold to 110 to prevent some test failures. + (Closes: reproducible-builds/diffoscope#233) + * Output improvements: + - Show fuzzyness amount in percentage terms, not out of the + rather-arbitrary "400". + - Improve the logging of fuzzy matching. + - Print the free space in our temporary directory when we create it, not + from within diffoscope.main. + * Codebase improvements: + - Tidy the diffoscopecomparators.utils.fuzzy module. + - Update my copyright years. + - Clarify the grammar of a comment. + - Clarify in a comment that __del__ is not always called, so temporary + directories are not neccessary removed the *moment* they go out of scope. + [ Conrad Ratschan ] + * Fix U-Boot Flattened Image Tree ("FIT") image detection for larger image + files. (MR: reproducible-builds/diffoscope!75) +- Update to version 165: + [ Dimitrios Apostolou ] + * Introduce the --no-acl and --no-xattr arguments [later collapsed to + --extended-filesystem-attributes] to improve performance. + * Avoid calling the external stat command. + [ Chris Lamb ] + * Collapse --acl and --xattr into --extended-filesystem-attributes to cover + all of these extended attributes, defaulting the new option to false (ie. + to not check these very expensive external calls). + [ Mattia Rizzolo ] + * Override several lintian warnings regarding prebuilt binaries in the + * source. + * Add a pytest.ini file to explicitly use Junit's xunit2 format. + * Ignore the Python DeprecationWarning message regarding the `imp` module + deprecation as it comes from a third-party library. + * debian/rules: filter the content of the d/*.substvars files +- Update to version 164: + [ Chris Lamb ] + * Truncate jsondiff differences at 512 bytes lest they consume the entire page. + * Wrap our external call to cmp(1) with a profile (to match the internal + profiling). + * Add a note regarding the specific ordering of the new + all_tools_are_listed test. + [ Dimitrios Apostolou ] + * Performance improvements: + - Improve speed of has_same_content by spawning cmp(1) less frequently. + - Log whenever the external cmp(1) command is spawn.ed + - Avoid invoking external diff for identical, short outputs. + * Rework handling of temporary files: + - Clean up temporary directories as we go along, instead of at the end. + - Delete FIFO files when the FIFO feeder's context manager exits. + [ Mattia Rizzolo ] + * Fix a number of potential crashes in --list-debian-substvars, including + explicitly listing lipo and otool as external tools. + - Remove redundant code and let object destructors clean up after themselves. + [ Conrad Ratschan ] + * Add a comparator for Flattened Image Trees (FIT) files, a boot image format + used by U-Boot. +- Update to version 163: + [ Chris Lamb ] + * Bug fixes & new features: + - Normalise "ret" to "retq" in objdump output to support multiple versions + of obdump(1). (Closes: deb#976760, reproducible-builds/diffoscope#227) + - Don't show progress indicators when running zstd(1). + (Closes: reproducible-builds/diffoscope#226) + - Move the slightly-confusing behaviour of loading an "existing diff" if a + single file is passed to diffoscope to the new --load-existing-diff + command. + * Output improvements: + - Use pprint.pformat in the JSON comparator to serialise the differences + from jsondiff to make the output render better. + - Correct tense in --debug log output. + * Code quality: + * Don't use an old-style "super" call. + - Rewrite the filter routine for post-processing output from readelf(1). + - Update my copyright years. + - Remove unncessary PEP 263 encoding lines (replaced via PEP 3120). + - Use "minimal" instead of "basic" as a variable name to match the + underlying package name. + - Add comment regarding Java tests for diffoscope contributors who are not + using Debian. (Re: reproducible-builds/diffoscope!58) + * Debian packaging: + - Update debian/copyright to match copyright notices in source-tree. + (Closes: reproducible-builds/diffoscope#224) + - Ensure the new "diffoscope-minimal" package has a different short + description from the main "diffoscope" one. + [ Jean-Romain Garnier ] + * Add tests for OpenJDK 14. + [ Conrad Ratschan ] + * Add comparator for "legacy" uboot uImage files. + (MR: reproducible-builds/diffoscope!69) +- Update to version 162: + [ Chris Lamb ] + * Don't depends on radare2 in the Debian autopkgtests as it will not be in + bullseye due to security considerations (#950372). (Closes: deb#975313) + * Avoid "Command `s p a c e d o u t` failed" messages when creating an + artificial CalledProcessError instance in our generic from_operation + feeder creator. + * Overhaul long and short descriptions. + * Use the operation's full name so that "command failed" messages include + its arguments. + * Add a missing comma in a comment. + [ Jelmer Vernooij ] + * Add missing space to the error message when only one argument is passed to + diffoscope. + [ Holger Levsen ] + * Update Standards-Version to 4.5.1. + [ Mattia Rizzolo ] + * Split the diffoscope package into a diffoscope-minimal package that + excludes the larger packages from Recommends. (Closes: deb#975261) + * Drop support for Python 3.6. +- Update to version 161: + [ Chris Lamb ] + * Fix failing testsuite: (Closes: deb#972518) + - Update testsuite to support OCaml 4.11.1. (Closes: deb#972518) + - Reapply Black and bump minimum version to 20.8b1. + * Move the OCaml tests to the assert_diff helper. + [ Jean-Romain Garnier ] + * Add support for radare2 as a disassembler. + [ Paul Spooren ] + * Automatically deploy Docker images in the continuous integration pipeline. +- Update to version 160: + * Check that pgpdump is actually installed before attempting to run it. + Thanks to Gianfranco Costamagna (locutusofborg). (Closes: deb#969753) + * Add some documentation for the EXTERNAL_TOOLS dictionary. + * Ensure we check FALLBACK_FILE_EXTENSION_SUFFIX, otherwise we run pgpdump + against all files that are recognised by file(1) as "data". +- Update to version 159: + [ Chris Lamb ] + * Show "ordering differences only" in strings(1) output. + (Closes: reproducible-builds/diffoscope#216) + * Don't alias output from "os.path.splitext" to variables that we do not end + up using. + * Don't raise exceptions when cleaning up after a guestfs cleanup failure. + [ Jean-Romain Garnier ] + * Make "Command" subclass a new generic Operation class. +- Update to version 158: + * Improve PGP support: + - Support extracting of files within PGP signed data. + (Closes: reproducible-builds/diffoscope#214) + - pgpdump(1) can successfully parse some unrelated, non-PGP binary files, + so check that the parsed output contains something remotely sensible + before identifying it as a PGP file. + * Don't use Python's repr(...)-style output in "Calling external command" + logging output. + * Correct a typo of "output" in an internal comment. +- Update to version 157: + [ Chris Lamb ] + * Try obsensibly "data" files named .pgp against pgpdump to determine whether + they are PGP files. (Closes: reproducible-builds/diffoscope#211) + * Don't raise an exception when we encounter XML files with "" + declarations inside the DTD, or when a DTD or entity references an external + resource. (Closes: reproducible-builds/diffoscope#212) + * Temporarily drop gnumeric from Build-Depends as it has been removed from + testing due to Python 2.x deprecation. (Closes: deb#968742) + * Codebase changes: + - Add support for multiple file extension matching; we previously supported + only a single extension to match. + - Move generation of debian/tests/control.tmp to an external script. + - Move to our assert_diff helper entirely in the PGP tests. + - Drop some unnecessary control flow, unnecessary dictionary comprehensions + and some unused imports found via pylint. + * Include the filename in the "... not identified by any comparator" + logging message. +- Update to version 156: + [ Chris Lamb ] + * Update PPU tests for compatibility with Free Pascal versions 3.2.0 or + greater. (Closes: deb#968124) + * Emit a debug-level logging message when our ppudump(1) version does not + match file header. + * Add and use an assert_diff helper that loads and compares a fixture output + to avoid a bunch of test boilerplate. + [ Frazer Clews ] + * Apply some pylint suggestions to the codebase. +- Update to version 155: + [ Chris Lamb ] + * Bump Python requirement from 3.6 to 3.7 - most distributions are either + shipping3.5 or 3.7, so supporting 3.6 is not somewhat unnecessary and also + more difficult to test locally. + * Improvements to setup.py: + - Apply the Black source code reformatter. + - Add some URLs for the site of PyPI.org. + - Update "author" and author email. + * Explicitly support Python 3.8. + [ Frazer Clews ] + * Move away from the deprecated logger.warn method logger.warning. + [ Mattia Rizzolo ] + * Document ("classify") on PyPI that this project works with Python 3.8. +- Update to version 154: + [ Chris Lamb ] + * Add support for F2FS filesystems. + (Closes: reproducible-builds/diffoscope#207) + * Allow "--profile" as a synonym for "--profile=-". + * Add an add_comment helper method so don't mess with our _comments list + directly. + * Add missing bullet point in a previous changelog entry. + * Use "human-readable" over unhyphenated version. + * Add a bit more debugging around launching guestfs. + * Profile the launch of guestfs filesystems. + * Correct adding a comment when we cannot extract a filesystem due to missing + guestfs module. +- Update to version 153: + [ Chris Lamb ] + * Drop some legacy argument styles; --exclude-directory-metadata and + --no-exclude-directory-metadata have been replaced with + --exclude-directory-metadata={yes,no}. + * Code improvements: + - Make it easier to navigate the main.py entry point. + - Use a relative import for get_temporary_directory in diffoscope.diff. + - Rename bail_if_non_existing to exit_if_paths_do_not_exist. + - Rewrite exit_if_paths_do_not_exist to not check files multiple times. + * Documentation improvements: + - CONTRIBUTING.md: + - Add a quick note about adding/suggesting new options. + - Update and expand the release process documentation. + - Add a reminder to regenerate debian/tests/control. + - README.rst: + - Correct URL to build job on Jenkins. + - Clarify and correct contributing info to point to salsa.debian.org. +- Update to version 152: + [ Chris Lamb ] + * Bug fixes: + - Don't require zipnote(1) to determine differences in a .zip file as we + can use libarchive directly. + * Reporting improvements: + - Don't emit "javap not found in path" if it is available in the path but + it did not result in any actual difference. + - Fix "... not available in path" messages when looking for Java + decompilers; we were using the Python class name (eg. "") over the actual command we looked + for (eg. "javap"). + * Code improvements: + - Replace some simple usages of str.format with f-strings. + - Tidy inline imports in diffoscope.logging. + - In the RData comparator, always explicitly return a None value in the + failure cases as we return a non-None value in the "success" one. + [ Jean-Romain Garnier ] + * Improve output of side-by-side diffs, detecting added lines better. + (MR: reproducible-builds/diffoscope!64) + * Allow passing file with list of arguments to ArgumentParser (eg. + "diffoscope @args.txt"). (MR: reproducible-builds/diffoscope!62) +- Update to version 151: + [ Chris Lamb] + * Improvements and bug fixes: + - Pass the absolute path when extracting members from SquashFS images as we + run the command with our working directory set to the temporary + directory. (Closes: deb#964365, reproducible-builds/diffoscope#189) + - Increase the minimum length of the output from strings(1) to 8 characters + to avoid unnecessary diff noise. (Re. reproducible-builds/diffoscope#148) + * Logging improvements: + - Fix the compare_files message when the file does not have a literal name. + - Reduce potential log noise by truncating the has_some_content messages. + * Codebase changes: + - Clarify use of a "null" diff in order to remember an exit code. + - Don't alias a variable when don't end up it; use "_" instead. + - Use a "NullChanges" file to represent missing data in the Debian package + comparator. + - Update some miscellaneous terms. +- Update to version 150: + [ Chris Lamb ] + * Don't crash when listing entries in archives if they don't have a listed + size (such as hardlinks in .ISO files). + (Closes: reproducible-builds/diffoscope#188) + * Dump PE32+ executables (including EFI applications) using objdump. + (Closes: reproducible-builds/diffoscope#181) + * Tidy detection of JSON files due to missing call to File.recognizes that + checks against the output of file(1) which was also causing us to attempt + to parse almost every file using json.loads. (Whoops.) + * Drop accidentally-duplicated copy of the new --diff-mask tests. + * Logging improvements: + - Split out formatting of class names into a common method. + - Clarify that we are generating presenter formats in the opening logs. + [ Jean-Romain Garnier ] + * Remove objdjump(1) offsets before instructions to reduce diff noise. + (Closes: reproducible-builds/diffoscope!57) +- Update to version 149: + [ Chris Lamb ] + * Update tests for file 5.39. (Closes: reproducible-builds/diffoscope#179) + * Downgrade the tlsh warning message to an "info" level warning. + (Closes: deb#888237, reproducible-builds/diffoscope#29) + * Use the CSS "word-break" property over manually adding U+200B zero-width + spaces that make copy-pasting cumbersome. + (Closes: reproducible-builds/diffoscope!53) + * Codebase improvements: + - Drop some unused imports from the previous commit. + - Prevent an unnecessary .format() when rendering difference comments. + - Use a semantic "AbstractMissingType" type instead of remembering to check + for both "missing" files and missing containers. + [ Jean-Romain Garnier ] + * Allow user to mask/filter reader output via --diff-mask=REGEX. + (MR: reproducible-builds/diffoscope!51) + * Make --html-dir child pages open in new window to accommodate new web + browser content security policies. + * Fix the --new-file option when comparing directories by merging + DirectoryContainer.compare and Container.compare. + (Closes: reproducible-builds/diffoscope#180) + * Fix zsh completion for --max-page-diff-block-lines. + [ Mattia Rizzolo ] + * Do not warn about missing tlsh during tests. +- Update to version 148: + [ Daniel Fullmer ] + * Fix a regression in the CBFS comparator due to changes in our_check_output. + [ Chris Lamb ] + * Add a remark in the deb822 handling re. potential security issue in the + .changes, .dsc, .buildinfo comparator. +- Update to version 147: + [ Chris Lamb ] + * New features: + - Add output from strings(1) to ELF binaries. It is intended this will + expose expose build paths that are hidden somewhere within the objdump(1) + output. (Closes: reproducible-builds/diffoscope#148) + - Add basic zsh shell tab-completion support. + (Closes: reproducible-builds/diffoscope#158) + * Bug fixes: + - Prevent a traceback when comparing a PDF document that does not contain + any metadata, ie. it is missing a PDF "/Info" stanza. + (Closes: reproducible-builds/diffoscope#150) + - Fix compatibility with jsondiff 1.2.0 which was causing a traceback and + log the version of jsondiff we are using to aid debugging in the future. + (Closes: reproducible-builds/diffoscope#159 + - Fix an issue in GnuPG keybox handling that left filenames in the diff. + - Don't mask an existing test name; ie. ensure it is actually run. + * Reporting: + - Log all calls to subprocess.check_output by using our own wrapper utility. + (Closes: reproducible-builds/diffoscope#151) + * Code improvements: + - Replace references to "WF" with "Wagner-Fischer" for clarity. + - Drop a large number of unused imports (list_libarchive, + ContainerExtractionError, etc.) + - Don't assign exception to a variable that we do not use. + - Compare string values with the equality operator, not via "is" identity. + - Don't alias an open file to a variable when we don't use it. + - Don't alias "filter" builtin. + - Refactor many small parts of the HTML generation, dropping explicit + u"unicode" strings, tidying the generation of the "Offset X, Y lines + modified" messages, moving to PEP 498 f-strings where appropriate, etc. + - Inline a number of single-used utility methods. +- Update to version 146: + [ Chris Lamb ] + * Refactor .changes and .buildinfo handling to show all details (including + the GPG header and footer components), even when referenced files are not + present. (Closes: reproducible-builds/diffoscope#122) + * Normalise filesystem stat(2) "birth times" (ie. st_birthtime) in the same + way we do with stat(1)'s "Access:" and "Change:" times to fix a + nondetermistic build failure on GNU Guix. + (Closes: reproducible-builds/diffoscope#74) + * Drop the (default) subprocess.Popen(shell=False) keyword argument so that + the more unsafe shell=True is more obvious. + * Ignore lower vs. upper-case when ordering our file format descriptions. + * Don't skip string normalisation in Black. + [ Mattia Rizzolo ] + * Add a "py3dist" override for the rpm-python module (Closes: deb#949598) + * Bump the debhelper compat level to 13 and use the new + execute_after_*/execture_before_* style rules. + * Fix a spelling error in changelog. + [ Daniel Fullmer ] + * Mount GuestFS filesystem images readonly. + [ Jean-Romain Garnier ] + * Prevent an issue where (for example) LibarchiveMember's has_same_content + method is called regardless of the actual type of file. +- Update to version 145: + [ Chris Lamb ] + * Improvements: + - Add support for Apple Xcode mobile provisioning .mobilepovision files. + (Closes: reproducible-builds/diffoscope#113) + - Add support for printing the signatures via apksigner(1). + (Closes: reproducible-builds/diffoscope#121) + - Use SHA256 over MD5 when generating page names for the HTML directory + presenter, validate checksums for files referenced in .changes files + using SHA256 too, and move to using SHA256 in "Too much input for diff" + output too. (Closes: reproducible-builds/diffoscope#124) + - Don't leak the full path of the temporary directory in "Command [..] + exited with 1". (Closes: reproducible-builds/diffoscope#126) + - Identify "iOS App Zip archive data" files as .zip files. + (Closes: reproducible-builds/diffoscope#116) + * Bug fixes: + - Correct "differences" typo in the ApkFile handler. + (Closes: reproducible-builds/diffoscope#127) + * Reporting/output improvements: + - Never emit the same id="foo" TML anchor reference twice, otherwise + identically-named parts will not be able to linked to via "#foo". + (Closes: reproducible-builds/diffoscope#120) + - Never emit HTML with empty "id" anchor lements as it is not possible to + link to "#" (vs "#foo"). We use "#top" as a fallback value so it will + work for the top-level parent container. + - Clarify the message when we cannot find the "debian" Python module. + - Clarify "Command [..] failed with exit code" to remove duplicate "exited + with exit" but also to note that diffoscope is interpreting this as an + error. + - Add descriptions for the 'fallback' Debian module file types. + - Rename the --debugger command-line argument to --pdb. + * Testsuite improvements: + - Prevent CI (and runtime) apksigner test failures due to lack of + binfmt_misc on Salsa CI and elsewhere. + * Codebase improvements: + - Initially add a pair of comments to tidy up a slightly abstraction level + violating code in diffoscope.comparators.mising_file and the + .dsc/.buildinfo file handling, but replace this later by by inlining + MissingFile's special handling of deb822 to prevent leaking through + abstraction layers in the first place. + - Use a BuildinfoFile (etc.) regardless of whether the associated files + such as the orig.tar.gz and the .deb are present, but don't treat them as + actual containers. (Re: reproducible-builds/diffoscope#122) + - Rename the "Openssl" command class to "OpenSSLPKCS7" to accommodate other + commands with this prefix. + - Wrap a docstring across multiple lines, drop an inline pprint import and + comment the HTMLPrintContext class, etc. + [ Emanuel Bronshtein ] + * Avoid build-cache in building the released Docker image. + (Closes: reproducible-builds/diffoscope#123) + [ Holger Levsen ] + * Wrap long lines in older changelog entries. +- Update to version 144: + [ Chris Lamb ] + * Improvements: + - Print the amount of free space that we have available in our temporary + directory as a debugging message. + - Remove (broken) fuzzy matching of JSON files as file 5.35 (in buster, + released 2018-10-18) supports recognising JSON data. + (Closes: reproducible-builds/diffoscope#106) + - Don't pretty-print the JSON output by default as it will usually be so + complicated to be unreadable by the human eye and it can be easily + replaced by piping to "| jq". + - Don't print a traceback if we pass a single, missing argument to + diffoscope (eg. a JSON diff to re-load). + * Reporting/output improvements: + - Reduce the default number of maximum standard error lines printed from 50 + to 25; usually the error is obvious by that point. + - Clarify the message when we truncate the number of lines to standard + error. + - Clarify when an external command emits for both files, otherwise it can + look like diffoscope is repeating itself when it is actually being run + twice. + - Don't repeat "stderr from {}" if both commands emit the same thing. + * Dockerfile improvements: + - Use ARG instead of ENV for DEBIAN_FRONTEND so we do not set this + environment variable at runtime. + (Closes: reproducible-builds/diffoscope#103) + - Run diffoscope as a non-root user in the runtime container. + (Closes: reproducible-builds/diffoscope#102) + - Add a .dockerignore file to whitelist files we need in our container. + Thanks to Emanuel Bronshtein for the original idea. + (Closes: reproducible-builds/diffoscope#105) + - Install/remove the build-essential package during build so we can install + the recommended packages from Git. + * Testsuite improvements: + - Include the Black output in the assertion failure too. + - Update the Black self-test; we don't care about the length of the black + output, rather whether it has some or, preferably, not. + * Codebase improvements: + - Bump the officially required version of Python from 3.5 to 3.6. + (Closes: reproducible-builds/diffoscope#117) + - Drop an unused shlex import. + - Instruct linters to pass over a bare try-except when cleaning up + temporary files used to extract archives. + - Format diffoscope/comparators/utils/command.py according to Black + 19.10b0-3. + - Drop entries from Uploaders that have not uploaded in over three years + with esteemed thanks for their previous work. + - Drop .travis.yml; we are using Salsa now and likely would not give + support for running on Travis CI at this point. + [ Jelle van der Waa ] + * Update LLVM diff for LLVM version 10. + [ Vagrant Cascadian ] + * Add external tool reference on openssl for GNU Guix. +- Update to version 143: + * Add support for .p7c and .p7b certificates. + (Closes: reproducible-builds/diffoscope#94) + * Add "pdftotext" as a requirement to run the PDF test_metadata text. Special + thanks to Chocimier/Piotr for the debugging work. + (Closes: reproducible-builds/diffoscope#99) + * Improve the documentation of FALLBACK_FILE_TYPE_HEADER_PREFIX and + FILE_TYPE_HEADER_PREFIX to note that only the first 16 bytes are used. +- Update to version 142: + [ Chris Lamb ] + * Render multiline difference comments in a way to show indentation. + (Closes: reproducible-builds/diffoscope#101) + * Correct parsing of "./setup.py test --pytest-args ...". + * Capitalise "Ordering differences only" in text comparison comments. + * Don't include the JSON similarity percentage if it is 0.0%. + [ Ben Hutchings ] + * Document how --exclude arguments are matched against filenames. + [ Vagrant Cascadian ] + * Add external tool reference for h5dump on Guix. +- Update to version 141: + [ Chris Lamb ] + * Dalvik .dex files can also serve as APK containers. Restrict the narrower + identification of .dex files to files ending with this extension, and widen + the identification of APK files to when file(1) discovers a Dalvik file. + (Closes: deb#884095, reproducible-builds/diffoscope#28) + * Explicitly list python3-h5py in debian/tests/control.in to ensure that we + have this module installed during an autopkgtest run to generate the test + fixture & regenerate debian/tests/control from debian/tests/control.in + to match. + * Drop unnecessary and unused assignment to "diff" variable. + * Strip paths from the output of zipinfo(1) warnings. + (re. reproducible-builds/diffoscope#97) + [ Michael Osipov ] + * Revert to using zipinfo(1) directly instead of piping input via /dev/stdin + for BSD portability. (Closes: reproducible-builds/diffoscope#97) + [ Jelle van der Waa ] + * Add an external tool for h5dump on Arch. +- Update to version 140: + * apktool 2.5.0 changed the handling of output of XML schemas, so update and + restrict the corresponding test to match. + (Closes: reproducible-builds/diffoscope#96) + * Add support for Hierarchical Data Format (HD5) files. + (Closes: reproducible-builds/diffoscope#95) +- Update to version 139: + [ Mattia Rizzolo ] + * Fix Build-Depends on python3-pdfminer. (Closes: deb#955645) +- Update to version 138: + * New features & bug fixes: + - Use dumppdf from python3-pdfminer if we do not see any other differences + from pdftext, etc. (reproducible-builds/diffoscope#92) + - Prevent a traceback when comparing two .rdx files directly as get_member + will return a file even if the file is missing and not raise a KeyError + exception; we therefore explicitly test for the existence of the file. + * Don't allow errors within "Rscript" deserialisation to cause the entire + operation to fail (for example if an external library cannot be loaded). + (reproducible-builds/diffoscope#91) + * Reporting improvements: + - Inject the supported file formats into the package long description and + remove any duplicate comparator descriptions when formatting. + (#reproducible-builds/diffoscope#90) + - Print a potentially-helpful message if the PyPDF2 module is not installed + (#reproducible-builds/diffoscope#92) + - Weaken "Install the X package to get a better output" claim to "... may + produce a better output" as this is not guaranteed. + * Code improvements: + - Inline the RequiredToolNotFound.get_package method; only used once. + - Drop deprecated "py36 = [..]" argument in pyproject.toml. + - Factor out generation of comparator descriptions. + - Add an upstream metadata file. +- Update to version 137: + * Also extract classes2.dex, classes3.dex etc. from .apk files. + (Closes: reproducible-builds/diffoscope#88) + * Add generalised support for "ignoring" returncodes and move special-casing + of returncodes in the zip comparator to this. + * Accommodate sng returning with a UNIX exit code of 1 even if there minor + errors in the file (discovered via #950806). +- Update to version 136: + [ Chris Lamb ] + * Improvements: + - Support external build tools. (Closes: reproducible-builds/diffoscope#87) + - Fallback to the regular .zip container format for .apk files if apktool + is not available. + - Clarify that "install X" in difference comment messages actually refer to + a system/distribution package. + - Drop the --max-report-size-child and --max-diff-block-lines-parent + options; both deprecated and scheduled for removal in January 2018. + * Bug fixes: + - No longer raise a KeyError exception if we request an invalid member from + a directory container. + * Logging improvements: + - Log a debug-level message if we cannot open a file as container due to a + missing tool in order to assist diagnosing issues. + - If we don't know the HTML output name, don't emit an enigmatic "html + output for" message. + - Add the current PATH environment variable to the "Normalising locale..." + debug-level message. + - Print the "Starting diffoscope $VERSION" line as the first line. + - Correct a debug-level message for compare_meta calls to quote the + arguments correctly. + * Refactoring: + - Add support for alternative container types for a file, allowing for + runtime (vs import time) control of fallbacks such as adding comments. + and append a comment to a difference if we fallback to an inferior + container format due to missing a tool. + - Factor-out the generation of "foo not available in path" difference + comment messages as a helper method in the exception that represents + them. + * Code improvements: + - Tidy diffoscope.main's configure method, factoring out the set of the + Config() global out of the run_diffoscope method and inlining the + functionality of maybe_set_limit, etc. + - Rename diffoscope.locale module to diffoscope.environ as we are modifying + things beyond just the locale (eg. calling tzset(), etc.) + - Drop unused "Difference" import from the APK comparator. + - Drop an assertion that is guaranteed by parallel "if" conditional. + - Add a "noqa" line to avoid a false-positive flake8 "unused import" + warning. + - Turn down the "volume" for a recommendation in a comment. + * Release/source-code management: + - Add a .git-blame-ignore-revs file to improve the output of git-blame(1) by + ignoring large changes when introducing the Black source code reformatter + and update the CONTRIBUTING.md guide on how to optionally use it locally. + - Convert CONTRIBUTING.rst to CONTRIBUTING.md and include it in the + PyPI.org release. + * Test improvements + - Refresh and update the fixtures for the .ico tests to match the latest + version of Imagemagick in Debian unstable. + [ Holger Levsen ] + * Bump Standards Version to 4.5.0, no changes needed. + [ Marc Herbert ] + * Search for expected keywords in the output of cbfstool tests and not a + specific output. (Closes: reproducible-builds/diffoscope!42) +- Update to version 135: + [ Chris Lamb ] + * Extract resources.arsc files (as well as classes.dex) from Android .apk + files to ensure that we at least show differences there. (Closes: deb#916359) + * Always pass a filename with a ".zip" extension to zipnote otherwise it + will return with an exit code of 9 and revert to a binary diff for the + entire file. + * Don't exhaustively output the entire HTML report when testing the + regression for #875281; parsing the JSON and pruning the tree should be + enough. + * Factor out running all of our zipinfo variants into a new method and use + this routine when comparing .apk files, thus now also displaying any + differences exposed by bsdtar. + * Testsuite improvements: + - Always allow fixtures called "output*". + - Actually test the output of the test_html_regression_875281 test. + * Add a note to the "Contributing" page to suggest enable concurrency when + running the tests locally. + [ Marc Herbert ] + * Fix a misplaced debug "Loading diff from stdin" logging message. + * Add new "Testing" section to CONTRIBUTING.rst. +- Update to version 134: + [ Chris Lamb ] + * Ensure that autopkgtests are run with our pyproject.toml present for the + correct "black" source code formatter settings. (Closes: deb#945993) + * Tidy some unnecessary boolean logic in the ISO9660 tests. + * Rename the "text_option_with_stdiout" to "text_option_with_stdout". + * Include the libarchive file listing for ISO images to ensure that + timestamps (not just dates) are visible in any difference. + (Closes: reproducible-builds/diffoscope#81) + [ Eli Schwartz ] + * Fix an exception in the progressbar handler. + [ Vagrant Cascadian ] + * Add an external tool reference for zstd on GNU Guix. +- Update to version 133: + * Correct the substitution/filtering of paths in ELF output to avoid + unnecessary differences depending on the path name provided on the + commandline. (Closes: deb#945572) + * Silence/correct a SyntaxWarning message due to incorrectly comparing an + integer by identity (is) over equality (==). (Closes: deb#945531) +- Update to version 132: + * Allow all possible .zip variations to return with non-zero exit codes, not + just known types we can explicitly identify (eg. Java .jmod and .jar + files). (Closes: reproducible-builds/diffoscope#78) + * Also permit UTF-8 encoded .dsc and .changes files. + (Re: reproducible-builds/diffoscope#78) + * Rework a long string of "or" statements into a loop with a "break". +- Update to version 131: + * Clarify in the HTML and text outputs that the limits are per output format, + not global. (Closes: deb#944882, reproducible-builds/diffoscope#76) + - Bump the previous "max_page_size" limit from 400 kB to 4 MB. + * Update an Android manifest test to reflect that parsed XML attributes are + returned in a sorted or otherwise novel manner under Python 3.8. + * Update code to reflect version 19.10b0 of the black source code + reformatter and don't run our self-test for versions earlier than this as + it will generate different results and thus fail. + * Limit .dsc and .buildinfo matching; don't attempt to compare them as + ELF sections or similar. (Closes: reproducible-builds/diffoscope#77) + * Add a comment that the text_ascii{1,2} test fixture files are used in + multiple places so are not trivial to generate on the fly. +- Update to version 130: + [ Chris Lamb ] + * debian/tests/basic-command-line: + Move from deprecated ADTTMP to AUTOPKGTEST_TMP. + * Correct the matching of R .rds files by also detecting newer versions + of this file format. + * Drop unused BASE_DIR global in the tests. + * Try and ensure that new test data files are generated dynamically, ie. + at least no new ones are added without "good" reasons. + * Truncate the tcpdump expected diff to 8KB (from ~600KB). + * Refresh OCaml test fixtures to support OCaml >= 4.08.1. Closes: deb#944709 + * Correct reference to the ".rdx" extension in a comment. + * Update XML test for Python 3.8+. + * Don't use line-base dbuffering when communucating with subprocesses + in binary mode. (Closes: reproducible-builds/diffoscope#75) + [ Jelle van der Waa ] + * Add support for comparing .zst files are created by zstd. + (Closes: reproducible-builds/diffoscope!34) +- Update to version 129: + * Call R's "deparse" function to ensure that we do not error out and revert + to a binary diff when processing .rdb files with internal "vector" types as + they do not automatically coerce to strings. + * Add the ability to pass Python bytestrings to external commands and + pass our long script to parse R .rdb files using this new method over a + long command-line argument + * Use Rscript's --vanilla option over --no-environ as this also enables + --no-save, --no-restore, --no-site-file and --no-init-file. + * Improve command-line error messages: + - Split out formatting into a separate utility function. + - Truncate very long lines when displaying them as an external source + of data. + - When printing an error from a command, format the command for the user. + * Use "exit code" over "return code" when referring to UNIX error codes in + displayed differences. +- Update to version 128: + * Query the container for the full path of the parallel R .rdx file for a + .rdb file as well as looking in the same directory. This ensures that + comparing two .deb/.changes files shows a varying path introduced in + version 127. +- Update to version 127: + [ Chris Lamb ] + * Move build-dependency on python-argcomplete to the Python 3.x version to + facilitate Python 2.x removal. (Closes: deb#942967) + * Overhaul the handling of GNU R .rdb files: + - Rework and refactor the handling of .rdb files specifically with respect + to locating the parallel .rdx file prior to inspecting the file to ensure + that we do not add files to the user's filesystem in the case of directly + comparing two .rdb files or, worse, overwriting a file in is place. + - Use a ("""-formatted) docstring for our internal R script to dump + variables. + - Mask/hide standard errors; R will often produce noisy output that is not + useful to us. + - Don't include a useless and misleading "NULL". + - Include all R object names are displayed, including ones beginning with a + fullstop ("."). + - Sort package fields when dumping data for output stability. + - Format package contents as "foo = bar" rather than using ugly and + misleading brackets. + - Include the object's type when dumping package contents. + - Never read the site or user's R location environment configuration to + ensure output stability. + - Expose absolute paths in the semantic/human-readable output, preventing + falling back to a useless hexdump. + * Improve the formatting of command lines: + - Ensure newlines and other metacharacters appear escaped as "\n", etc. + - Use the newline (etc.) escaped version of the commandline being executed + in logging/debug output. + - When displaying standard error, ensure use the escaped version too. + * Add support for easily masking the standard error of commands and use this + in the ffprobe comparator. + * To match the libarchive container, raise a KeyError exception if we request + an invalid member from a directory container. + * Correct string representation output in the traceback when we cannot locate + a specific item in a container. + [ Mattia Rizzolo ] + * Run Debian autopkgtests against all Python versions. +- Update to version 126: + [ Chris Lamb ] + * Track and report on missing Python modules. + (Closes: reproducible-builds/diffoscope#72) + * Drop some unnecessary control flow. + * Drop explicit inheriting from 'object' class; unnecessary in Python 3. + * Bump Standards-Version to 4.4.1. + [ Mattia Rizzolo ] + * Exit with 2 instead of 1 in case of no disk space. +- Update to version 125: + * The test_libmix_differences ELF test requires xxd. (Closes: deb#940645) +- Update to version 124: + [ Chris Lamb ] + * Also conditionally skip the identification and "no differences" tests as we + require the Ocaml compiler to be present when building the test files + themselves. (Closes: deb#940471) + [ Mattia Rizzolo ] + * Permit all sorts of version suffixes when checking the Debian package + version matches setup.py, not just for backports. (Closes: deb#939387) + [ Vagrant Cascadian ] + * Add external tools on GNU Guix for odt2txt, sng and pgpdump. + [ Marc Herbert ] + * Remove StaticLibFile in the ELF comparator -- ArFile.compare_details() is + superior. (Closes: reproducible-builds/diffoscope#64) +- Update to version 123: + [ Chris Lamb ] + * Build OCaml test input files on-demand rather than shipping them with the + package in order to prevent test failures with OCaml 4.08. + (Closes: deb#939386) + * Update test skipping messages: + - When skipping tests due to the lack of installed tool, print the package + that may provide it. + - Update "requires foo module" messages to clarify that they are regarding + Python modules, not packages. + * Rebuild the test squashfs images to exclude the character device as they + requires root or fakeroot to extract. (reproducible-builds/diffoscope#65) + * Remove accidentally-committed test fixture generation code from tests. + * Set long_description_content_type to 'text/x-rst' in setup.py to appease + the PyPI.org linter. + [ Mattia Rizzolo ] + * Don't crash when the progress is requested but the Python module is + missing. (Closes: deb#939085) + [ Vagrant Cascadian ] + * Add external tools for gifbuild, javap and kbxutil in GNU Guix. + [ Marc Herbert ] + * In the cbfs tests, add a "Comp" column test to support Coreboot + utils > 4.6. +- Update to version 122: + [ Chris Lamb ] + * Apply patch from László Böszörményi to update the squashfs test output and + bump the required version for the test itself. (Closes: deb#935684) + * Skip calls to unsquashfs when we are not root or fakeroot. + (Re: reproducible-builds/diffoscope#63) + * Include either standard error or standard output (and not just the latter) + if/when an external command fails. + * Fix a few unicode/bytes issues: + - Avoid a possible traceback caused by a str/bytes confusion when handling + the output of failing external commands. + - Ensure that all of our artificially-created subprocess.CalledProcessError + instances have `output` instances that are bytes objects, not str. + * Improve debugging output: + * Add the containing module name to the (eg.) "Using StaticLibFile for ..." + * Improve and condense output when creating our Comparator object types. + * Correct a reference to `parser.diff` as `diff` in this context is a Python + function in the module, not the actual output returned from diff(1). + * Add the "wabt" Debian package to the test dependencies so that we run the + wasm tests. + [ Mattia Rizzolo ] + * Now that we test-require wabt, expect that its tools to be available during + autopkgtests. +- Update to version 121: + * Don't fallback to a (useless) raw hexdump when readelf(1) reports an minor + issue in a section in an ELF binary. For example, when the "frames" section + is of the "NOBITS" type, its contents are apparently "unreliable" and thus + readelf(1) exits with a returncode of 1. (Closes: deb#849407, #931962) + * Add support to Difference.from_command_exc and friends to optionally ignore + specified returncodes from the called program and treat them as "no" + difference. + * Simplify the parsing of the optional "command_args" argument to the + from_command and from_command_exc methods in the Difference class. +- Update to version 120: + * No-change sourceful after accidentally uploading binaries in order to + migration to testing. +- Update to version 119: + [ Chris Lamb ] + * If a command fails to execute but does not print anything to standard + error, try and include the first line of standard output in the message we + include in the diff. This was motivated by readelf(1) returning its error + messages on stdout. (Closes: deb#931963) + * Add support for Java ".jmod" modules. Not all versions of file(1) support + detection of Jmod files yet, so we perform a manual comparison instead. + (Closes: deb#933308) + * Re-add "return code" noun to "Command `foo` exited with X" error messages. + * Factor out the ability to ignore the exit codes of "zipinfo" and + "zipinfo -v" in the presence of non-standard headers. + * Only override the exit code from our special-cased calls to zipinfo(1) if + they are 1 or 2 to avoid potentially masking real errors. + * Also add missing textual description entries for ZipFile and + MozillaZipFile file types. + * Skip extra newline in "Output:\n". + * Cease ignoring test failures in stable-backports. + * Bump debhelper compat level to 12. + [ Marc Herbert ] + * Catch failures to disassemble and rescue all other differences. + (Closes: reproducible-builds/diffoscope!29) +- Update to version 118: + * Don't fail in autopkgtests when, for example, we do not have sufficiently + newer or older version of file. (Closes: deb#931881) + * Also include python3-tlsh in test dependencies. + * Tidy handling of DIFFOSCOPE_FAIL_TESTS_ON_MISSING_TOOLS: + - Merge two previously-overlapping environment variables. + - Use repr(..)-style output when printing test status. + - Add some explicit return values to appease pylint. +- Update to version 117: + [ Chris Lamb ] + * Add support for file 5.37. Thanks again to Christoph Biedl for the heads-up + in advance. (Closes: reproducible-builds/diffoscope/#57) + * Apply patch from Gianfranco Costamagna to fix autopkgtest failures in + Debian. (Closes: deb#931709) + [ Holger Levsen ] + * debian/control: Bump Standards-Version to 4.4.0. +- Update to version 116: + [ Chris Lamb ] + * Upload to unstable after the release of Debian "buster". + * README.rst & manual page generation: + - Strip out manpage-only parts of the README rather than using "only" + reStructuredText directives in order to support the demands of the latest + PyPI website. + - Use "real" reStructuredText comments instead of using the "raw" directive. + * Dockerfile: + - Build the Docker image from the current Git checkout, not the Debian + archive. (reproducible-builds/diffoscope#56) + - Use the ENTRYPOINT directive with the JSON syntax instead so we pass all + arguments to the underlying diffoscope executable. + * Document that run_diffoscope should not be considered a stable API. + [ Mattia Rizzolo ] + * Rename a test function to prevent shadowing a previous one with the + same name. + * Add ffmpeg to the list of Debian build-dependencies for the testsuite. + [ Vagrant Cascadian ] + * Add support for known external tools in GNU Guix. +- Update to version 115): + [ Chris Lamb ] + * Fix execution of symbolic links that point to the "bin/diffoscope" entry + point by fully resolving the location. + * Add a Dockerfile. + * Update contributing messages with instructions regarding the Docker image. + [ Mattia Rizzolo ] + * tests: + + Allow specifying which tools are known missing using + DIFFOSCOPE_TESTS_MISSING_TOOLS, to override a _FAIL_ON_MISSING_TOOLS. + + With DIFFOSCOPE_TESTS_FAIL_ON_MISSING_TOOLS=1, actually fail only tests + that are missing the required tools. + * debian: + + Add black to the test dependencies. + + Enforce the "fail on missing tools" only when testing with all the + recommended packages. + + Ack some missing tools during autopkgtest. + + Reinstate oggvideotools and procyon-decompiler test dependencies, + now that are not buggy anymore. +- Update to version 114): + [ Chris Lamb ] + * Add support for GnuPG "keybox" files. Thanks to Daniel Kahn Gillmor for the + suggestion. (Closes: deb#871244, reproducible-builds/diffoscope#23) + * Always warn if tlsh module is not available (not just if a specific + fuzziness threshold is specified) to match the epilog of the --help output. + This prevents missing support for file rename detection. + (Closes: deb#888237, reproducible-builds/diffoscope#29) + * Treat missing tools on Debian autopkgtests as individual test failures by + checking whether a new DIFFOSCOPE_TESTS_FAIL_ON_MISSING_TOOLS environment + variable is exported. (Closes: deb#905885, reproducible-builds/diffoscope#35) + * Require that "-" is explicitly specified to read a single diff from + standard input to avoid non-intuitive behaviour when diffoscope is called + without any arguments. (Closes: reproducible-builds/diffoscope#54) + * Make --use-dbgsym a ternary operator to make it easier to totally disable. + Thanks to Mattia Rizzolo for the suggestion. + * Consolidate on "e" as the aliased exception name. + [ Milena Boselli Rosa ] + * Various fixes to the HTML markup to prevent validation warnings/errors: + - Prevent empty values for the "name" attribute name on HTML anchor tags, + and add an "id" to its parent "div" container. + - Fix "table column x established by element 'col' has no cells beginning + in it" warnings. + - Fix "Text run is not in Unicode Normalization Form C". + - Remove the "type" HTML attribute from