draigBrady.com> -Subject: Re: coreutils-8.10 released [stable] -Newsgroups: gmane.comp.gnu.coreutils.general -Date: 2011-02-07 12:39:27 GMT (2 days, 22 hours and 55 minutes ago) - -On 05/02/11 13:59, Jim Meyering wrote: -> Pádraig Brady wrote: ->> Yep, just did that, and got a couple of failures for sparse-fiemap ->> on and ext3 and loopback ext4 file systems. ->> On a very quick glance, I think cp is OK and that the filefrag ->> matching is a bit brittle. ->> Attached are filefrag outputs. -> -> I saw similar differences, but I think they were due to the fact that -> the kernel had not yet forced cp's metadata update to disk when filefrag -> does its FIEMAP ioctl -> -> Uncommenting the "sync" just after the "cp" in the sparse-fiemap test -> solved the problem (for me it arose only on rawhide's btrfs) but made -> the test take a lot more time, as mentioned in the comment. - -It seems that the sync is needed for ext4 loopback also, on my system. -However for ext3, the extents between src and dst still don't match up. -My systems uses a 4K block size and is 2.6.35.10-72.fc14.i686 -For now, I've disabled (indirectly) using ext3 for fiemap-perf and -sparse-fiemap in the attached. - -> Instead of that sync, using filefrag's -s option may -> have the same result without the unwanted overhead. - -That doesn't work actually which baffles me. -I got the e2fsprogs source rpm to verify that -FIEMAP_FLAG_SYNC was set, so I guess this is a kernel issue -(on ext4 loop back at least). -Though I was able to get a a working sync restricted to the file -by using dd in the attached patch. -Even with that, the test only takes about 10s on my old laptop, so -I didn't bother tagging the test as EXPENSIVE. - -cheers, -Pádraig. - -From 1da62d67ce4d4c78b98bc947c9367a10f1bdba9d Mon Sep 17 00:00:00 2001 -From: =?UTF-8?q?P=C3=A1draig=20Brady?=
draigBrady.com> -Date: Fri, 4 Feb 2011 22:05:20 +0000 -Subject: [PATCH] test: improve the fiemap_capable_ check - -* tests/cp/fiemap-2: Enable the fiemap check for files, which -will enable the test on ext3. -* tests/cp/fiemap-perf: Comment why we're not enabling for ext3. -* tests/cp/sparse-fiemap: Ditto. -* tests/fiemap-capable: A new python script to determine -if a specified path supports fiemap. -* tests/init.cfg (fiemap_capable_): Use the new python script. -* tests/Makefile.am (EXTRA_DIST): Include the new python script. ---- - tests/Makefile.am | 1 + - tests/cp/fiemap-2 | 3 ++- - tests/cp/fiemap-perf | 2 ++ - tests/cp/sparse-fiemap | 12 ++++++++---- - tests/fiemap-capable | 16 ++++++++++++++++ - tests/init.cfg | 9 ++++----- - 6 files changed, 33 insertions(+), 10 deletions(-) - create mode 100644 tests/fiemap-capable - -diff --git a/tests/Makefile.am b/tests/Makefile.am -index 751b327..8aa56cd 100644 ---- a/tests/Makefile.am -+++ b/tests/Makefile.am -@@ -11,6 +11,7 @@ EXTRA_DIST = \ - check.mk \ - envvar-check \ - filefrag-extent-compare \ -+ fiemap-capable \ - init.cfg \ - init.sh \ - lang-default \ -diff --git a/tests/cp/fiemap-2 b/tests/cp/fiemap-2 -index a17076c..691ead2 100755 ---- a/tests/cp/fiemap-2 -+++ b/tests/cp/fiemap-2 -@@ -20,7 +20,8 @@ - print_ver_ cp - - # Require a fiemap-enabled FS. --fiemap_capable_ . \ -+touch fiemap_chk # check a file rather than current dir for best coverage -+fiemap_capable_ fiemap_chk \ - || skip_ "this file system lacks FIEMAP support" - - # Exercise the code that handles a file ending in a hole. -diff --git a/tests/cp/fiemap-perf b/tests/cp/fiemap-perf -index 7369a7d..dbb2a81 100755 ---- a/tests/cp/fiemap-perf -+++ b/tests/cp/fiemap-perf -@@ -20,6 +20,8 @@ - print_ver_ cp - - # Require a fiemap-enabled FS. -+# Note we don't check a file here as that could enable -+# the test on ext3 where emulated extent scanning can be slow. - fiemap_capable_ . \ - || skip_ "this file system lacks FIEMAP support" - -diff --git a/tests/cp/sparse-fiemap b/tests/cp/sparse-fiemap -index f224b5b..fc27869 100755 ---- a/tests/cp/sparse-fiemap -+++ b/tests/cp/sparse-fiemap -@@ -19,6 +19,8 @@ - . "${srcdir=.}/init.sh"; path_prepend_ ../src - print_ver_ cp - -+# Note we don't check a file here as that could enable -+# the test on ext3 where this test is seen to fail. - if fiemap_capable_ . ; then - : # Current dir is on a partition with working extents. Good! - else -@@ -66,11 +68,13 @@ for i in $(seq 1 2 21); do - $PERL -e 'BEGIN { $n = '$i' * 1024; *F = *STDOUT }' \ - -e 'for (1..'$j') { sysseek (*F, $n, 1)' \ - -e '&& syswrite (*F, chr($_)x$n) or die "$!"}' > j1 || fail=1 -- # sync -+ -+ # Note the explicit fdatasync is used here as -+ # it was seen that `filefrag -s` (FIEMAP_FLAG_SYNC) was -+ # ineffective on ext4 loopback on Linux 2.6.35.10-72.fc14.i686 -+ dd if=/dev/null of=j1 conv=notrunc,fdatasync - cp --sparse=always j1 j2 || fail=1 -- # sync -- # Technically we may need the 'sync' uses above, but -- # uncommenting them makes this test take much longer. -+ dd if=/dev/null of=j2 conv=notrunc,fdatasync - - cmp j1 j2 || fail=1 - filefrag -v j1 | grep extent \ -diff --git a/tests/fiemap-capable b/tests/fiemap-capable -new file mode 100644 -index 0000000..05c6926 ---- /dev/null -+++ b/tests/fiemap-capable -@@ -0,0 +1,16 @@ -+import struct, fcntl, sys, os -+ -+def sizeof(t): return struct.calcsize(t) -+IOCPARM_MASK = 0x7f -+IOC_OUT = 0x40000000 -+IOC_IN = 0x80000000 -+IOC_INOUT = (IOC_IN|IOC_OUT) -+def _IOWR(x,y,t): return (IOC_INOUT|((sizeof(t)&IOCPARM_MASK)<<16)|((x)<<8)|y) -+ -+try: -+ fd = os.open (len (sys.argv) == 2 and sys.argv[1] or '.', os.O_RDONLY) -+ struct_fiemap = '=qqllll' -+ FS_IOC_FIEMAP = _IOWR (ord ('f'), 11, struct_fiemap) -+ fcntl.ioctl (fd, FS_IOC_FIEMAP, struct.pack(struct_fiemap, 0,~0,0,0,0,0)) -+except: -+ sys.exit (1) -diff --git a/tests/init.cfg b/tests/init.cfg -index b2d1bab..f49c5cf 100644 ---- a/tests/init.cfg -+++ b/tests/init.cfg -@@ -295,13 +295,12 @@ require_proc_pid_status_() - kill $pid - } - --# Return nonzero if the specified directory is on a file system for --# which FIEMAP support exists, and the file system type is new enough --# (unlike ext2 and ext3) that it is hard to find an instance *without* --# FIEMAP support. -+# Return nonzero if the specified path is on a file system for -+# which FIEMAP support exists. Note some file systems (like ext3) -+# only support FIEMAP for files, not directories. - fiemap_capable_() - { -- df -T -t btrfs -t xfs -t ext4 -t ocfs2 -t gfs2 "$@" -+ python $abs_srcdir/fiemap-capable "$@" - } - - # Does the current (working-dir) file system support sparse files? --- -1.7.3.4 - diff --git a/coreutils-gl_printf_safe.patch b/coreutils-gl_printf_safe.patch index 9e8d299..30c7cee 100644 --- a/coreutils-gl_printf_safe.patch +++ b/coreutils-gl_printf_safe.patch @@ -1,20 +1,20 @@ Index: configure =================================================================== ---- configure.orig 2011-01-04 12:23:03.000000000 +0100 -+++ configure 2011-01-05 14:27:40.804235553 +0100 -@@ -3423,7 +3423,6 @@ as_fn_append ac_func_list " alarm" +--- configure.orig 2011-04-13 21:22:30.000000000 +0200 ++++ configure 2011-04-14 14:39:11.027077753 +0200 +@@ -3513,7 +3513,6 @@ as_fn_append ac_func_list " alarm" as_fn_append ac_header_list " sys/statvfs.h" as_fn_append ac_header_list " sys/select.h" as_fn_append ac_func_list " nl_langinfo" -gl_printf_safe=yes as_fn_append ac_header_list " priv.h" as_fn_append ac_header_list " pthread.h" - as_fn_append ac_func_list " readlinkat" + as_fn_append ac_header_list " utmp.h" Index: m4/gnulib-comp.m4 =================================================================== ---- m4/gnulib-comp.m4.orig 2011-01-01 22:23:14.000000000 +0100 -+++ m4/gnulib-comp.m4 2011-01-05 14:27:40.806235609 +0100 -@@ -1190,7 +1190,6 @@ AC_DEFUN([gl_INIT], +--- m4/gnulib-comp.m4.orig 2011-04-09 09:51:23.000000000 +0200 ++++ m4/gnulib-comp.m4 2011-04-14 14:38:04.363003296 +0200 +@@ -1217,7 +1217,6 @@ AC_DEFUN([gl_INIT], # Code from module printf-frexpl: gl_FUNC_PRINTF_FREXPL # Code from module printf-safe: diff --git a/coreutils-testsuite.spec b/coreutils-testsuite.spec index bb64b52..9758d32 100644 --- a/coreutils-testsuite.spec +++ b/coreutils-testsuite.spec @@ -23,7 +23,7 @@ BuildRequires: help2man libacl-devel libcap-devel libselinux-devel pam-devel xz Url: http://www.gnu.org/software/coreutils/ License: GFDLv1.2 ; GPLv2+ ; GPLv3+ Group: System/Base -Version: 8.10 +Version: 8.11 Release: 0 Provides: fileutils = %{version}, sh-utils = %{version}, stat = %version}, textutils = %{version}, mktemp = %{version} Obsoletes: fileutils < %{version}, sh-utils < %{version}, stat < %version}, textutils < %{version}, mktemp < %{version} @@ -55,9 +55,6 @@ Patch30: coreutils-8.6-compile-su-with-fpie.diff Patch31: coreutils-getaddrinfo.patch Patch32: coreutils-ptr_int_casts.patch Patch33: coreutils-8.9-singlethreaded-sort.patch -#Next two patches added after 8.10 release so should be obsolete with next release -Patch34: coreutils-compute_range_endpoint.patch -Patch35: coreutils-fix_sparse_fiemap_test.patch BuildRoot: %{_tmppath}/%{name}-%{version}-build # this will create a cycle, broken up randomly - coreutils is just too core to have other @@ -99,8 +96,6 @@ uname unexpand uniq unlink uptime users vdir wc who whoami yes %patch31 %patch32 %patch33 -%patch34 -p1 -%patch35 -p1 %build AUTOPOINT=true autoreconf -fi diff --git a/coreutils.changes b/coreutils.changes index 7be1c11..65b7d24 100644 --- a/coreutils.changes +++ b/coreutils.changes @@ -1,3 +1,62 @@ +------------------------------------------------------------------- +Thu Apr 14 14:46:41 CEST 2011 - pth@suse.de + +- Update to 8.11: + + * Bug fixes + + cp -a --link would not create a hardlink to a symlink, instead + copying the symlink and then not preserving its timestamp. + [bug introduced in coreutils-8.0] + + cp now avoids FIEMAP issues with BTRFS before Linux 2.6.38, + which could result in corrupt copies of sparse files. + [bug introduced in coreutils-8.10] + + cut could segfault when invoked with a user-specified output + delimiter and an unbounded range like "-f1234567890-". + [bug introduced in coreutils-5.3.0] + + du would infloop when given --files0-from=DIR + [bug introduced in coreutils-7.1] + + sort no longer spawns 7 worker threads to sort 16 lines + [bug introduced in coreutils-8.6] + + touch built on Solaris 9 would segfault when run on Solaris 10 + [bug introduced in coreutils-8.8] + + wc would dereference a NULL pointer upon an early out-of-memory error + [bug introduced in coreutils-7.1] + + ** New features + + dd now accepts the 'nocache' flag to the iflag and oflag options, + which will discard any cache associated with the files, or + processed portion thereof. + + dd now warns that 'iflag=fullblock' should be used, + in various cases where partial reads can cause issues. + + ** Changes in behavior + + cp now avoids syncing files when possible, when doing a FIEMAP copy. + The sync is only needed on Linux kernels before 2.6.39. + [The sync was introduced in coreutils-8.10] + + cp now copies empty extents efficiently, when doing a FIEMAP copy. + It no longer reads the zero bytes from the input, and also can + efficiently create a hole in the output file when --sparse=always + is specified. + + df now aligns columns consistently, and no longer wraps entries + with longer device identifiers, over two lines. + + install now rejects its long-deprecated --preserve_context option. + Use --preserve-context instead. + + test now accepts "==" as a synonym for "=" + ------------------------------------------------------------------- Tue Apr 5 15:13:42 CEST 2011 - pth@suse.de diff --git a/coreutils.spec b/coreutils.spec index 838085f..34fb814 100644 --- a/coreutils.spec +++ b/coreutils.spec @@ -23,7 +23,7 @@ BuildRequires: help2man libacl-devel libcap-devel libselinux-devel pam-devel xz Url: http://www.gnu.org/software/coreutils/ License: GFDLv1.2 ; GPLv2+ ; GPLv3+ Group: System/Base -Version: 8.10 +Version: 8.11 Release: 2 Provides: fileutils = %{version}, sh-utils = %{version}, stat = %version}, textutils = %{version}, mktemp = %{version} Obsoletes: fileutils < %{version}, sh-utils < %{version}, stat < %version}, textutils < %{version}, mktemp < %{version} @@ -55,9 +55,6 @@ Patch30: coreutils-8.6-compile-su-with-fpie.diff Patch31: coreutils-getaddrinfo.patch Patch32: coreutils-ptr_int_casts.patch Patch33: coreutils-8.9-singlethreaded-sort.patch -#Next three patches added after 8.10 release so should be obsolete with next release -Patch34: coreutils-compute_range_endpoint.patch -Patch35: coreutils-fix_sparse_fiemap_test.patch BuildRoot: %{_tmppath}/%{name}-%{version}-build # this will create a cycle, broken up randomly - coreutils is just too core to have other # prerequires @@ -98,8 +95,6 @@ uname unexpand uniq unlink uptime users vdir wc who whoami yes %patch31 %patch32 %patch33 -%patch34 -p1 -%patch35 -p1 %build AUTOPOINT=true autoreconf -fi