Upstream patch - to diagnose readdir() failures in fts-based utilities, - fixes boo#984910, - to be removed with coreutils > v8.25, - consists of the following parts: 1. the fix in thee gnulib submodule 2. the NEWS entry 3. the test added to verify the fix. 4. Upstream coreutils commit to fix the test on 32-bit ---------------------------------------------------------------------- 1. Upstream gnulib submodule commit: http://git.sv.gnu.org/cgit/gnulib.git/commit/?id=6835fc458f30 Taken without the ChangeLog change: From 6835fc458f30b94f15d69c35a79cbc2dfabe2d06 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= Date: Wed, 22 Jun 2016 13:49:53 +0100 Subject: [PATCH] fts: handle readdir() errors * lib/fts.c (fts_build): readdir(3) returns NULL when finished, but also upon error when it will also set errno. Therefore flag the error case from readdir(). We treat the case where no items are read the same as if the dir can't be accessed, i.e. by setting fts_errno to FTS_DNR. The bug was initially reported by Peter Benie http://bugzilla.opensuse.org/show_bug.cgi?id=984910 where it was mentioned that readdir() may fail when an NFS server has a poor readdir cookie implementation. ---------------------------------------------------------------------- 2. Upstream coreutils commit: http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=ef9650170f79 Take the NEWS change only, as the fix in fts.c was already pulled in with the above gnulib commit. From ef9650170f795be41223c8887258a1c596ecc162 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= Date: Sun, 26 Jun 2016 20:58:41 +0100 Subject: [PATCH] all: update gnulib submodule and tests/init.sh to latest * NEWS: Specifically mention the fts readdir() fix. ---------------------------------------------------------------------- 3. Upstream coreutils commit to add a test: http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=26616776c0c6 From 26616776c0c620ce72b3b69aa5ed63f495552a9e Mon Sep 17 00:00:00 2001 From: Peter Benie Date: Sun, 26 Jun 2016 19:07:45 +0100 Subject: [PATCH] tests: verify that fts diagnoses readdir() failures * tests/rm/rm-readdir-fail.sh: A new test to simulate readdir() failing immediately or after returning a few entries, and verifying that rm does the appropriate thing. This was initially reported at: http://bugzilla.opensuse.org/show_bug.cgi?id=984910 where it was mentioned that readdir() may fail when an NFS server has a poor readdir cookie implementation. ---------------------------------------------------------------------- 4. Upstream coreutils commit to fix the test on 32-bit http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=54c1397510cb From 54c1397510cb08433680b5b7da46a8201770e9ee Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?P=C3=A1draig=20Brady?= Date: Wed, 6 Jul 2016 18:08:32 +0100 Subject: [PATCH] tests: avoid false failure on 32 bit in readdir() test * tests/rm/rm-readdir-fail.sh: Since we use the returned dirent from the readdir wrapper it must be the correct type and not just cast. Therefore setup so that we only have to define a wrapper for readdir(), which works appropriately on 32 and 64 bit. Issue reported by Bernhard Voelker, where rm was seen to invoke rmdir() on invalid file names. ---------------------------------------------------------------------- --- NEWS | 9 +++ lib/fts.c | 14 ++++- tests/local.mk | 1 tests/rm/rm-readdir-fail.sh | 107 ++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 129 insertions(+), 2 deletions(-) Index: NEWS =================================================================== --- NEWS.orig +++ NEWS @@ -1,5 +1,14 @@ GNU coreutils NEWS -*- outline -*- +* Noteworthy downstream / openSUSE changes + + chcon, chgrp, chmod, chown, du, and rm, or specifically utilities + using the FTS interface, now diagnose failures returned by readdir(). + [this bug was inherent in the use of fts: thus, for rm the bug was + introduced in coreutils-8.0. du, chmod, chgrp and chown started using + fts in 6.0. chcon was added in coreutils-6.9.91 with fts support. ] + + * Noteworthy changes in release 8.25 (2016-01-20) [stable] ** Bug fixes Index: lib/fts.c =================================================================== --- lib/fts.c.orig +++ lib/fts.c @@ -1461,9 +1461,18 @@ fts_build (register FTS *sp, int type) while (cur->fts_dirp) { bool is_dir; size_t d_namelen; + __set_errno (0); struct dirent *dp = readdir(cur->fts_dirp); - if (dp == NULL) + if (dp == NULL) { + if (errno) { + cur->fts_errno = errno; + /* If we've not read any items yet, treat + the error as if we can't access the dir. */ + cur->fts_info = (continue_readdir || nitems) + ? FTS_ERR : FTS_DNR; + } break; + } if (!ISSET(FTS_SEEDOT) && ISDOT(dp->d_name)) continue; @@ -1622,7 +1631,8 @@ mem1: saved_er /* If didn't find anything, return NULL. */ if (!nitems) { - if (type == BREAD) + if (type == BREAD + && cur->fts_info != FTS_DNR && cur->fts_info != FTS_ERR) cur->fts_info = FTS_DP; fts_lfree(head); return (NULL); Index: tests/local.mk =================================================================== --- tests/local.mk.orig +++ tests/local.mk @@ -223,6 +223,7 @@ all_tests = \ tests/rm/unreadable.pl \ tests/rm/v-slash.sh \ tests/rm/many-dir-entries-vs-OOM.sh \ + tests/rm/rm-readdir-fail.sh \ tests/chgrp/default-no-deref.sh \ tests/chgrp/deref.sh \ tests/chgrp/no-x.sh \ Index: tests/rm/rm-readdir-fail.sh =================================================================== --- /dev/null +++ tests/rm/rm-readdir-fail.sh @@ -0,0 +1,107 @@ +#!/bin/sh +# Test rm's behaviour when the directory cannot be read. +# This test is skipped on systems that lack LD_PRELOAD support. + +# Copyright (C) 2016 Free Software Foundation, Inc. + +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +. "${srcdir=.}/tests/init.sh"; path_prepend_ ./src +print_ver_ rm +require_gcc_shared_ + +mkdir -p dir/notempty || framework_failure_ + +# Simulate "readdir" failure. +cat > k.c <<\EOF || framework_failure_ +#define _GNU_SOURCE + +/* Setup so we don't have to worry about readdir64. */ +#ifndef __LP64__ +# define _FILE_OFFSET_BITS 64 +#endif + +#include +#include +#include +#include +#include + +struct dirent *readdir (DIR *dirp) +{ + static struct dirent *(*real_readdir)(DIR *dirp); + if (! real_readdir && ! (real_readdir = dlsym (RTLD_NEXT, "readdir"))) + { + fprintf (stderr, "Failed to find readdir()\n"); + errno = ESRCH; + return NULL; + } + static struct dirent* d; + if (! d && ! ( d = real_readdir (dirp))) + { + fprintf (stderr, "Failed to get dirent\n"); + errno = ENOENT; + return NULL; + } + + /* Flag that LD_PRELOAD and above functions work. */ + static int count = 1; + if (count == 1) + fclose (fopen ("preloaded", "w")); + + /* Return some entries to trigger partial read failure, + ensuring we don't return ignored '.' or '..' */ + char const *readdir_partial = getenv ("READDIR_PARTIAL"); + if (readdir_partial && *readdir_partial && count <= 3) + { + count++; + d->d_name[0]='0'+count; d->d_name[1]='\0'; +#ifdef _DIRENT_HAVE_D_NAMLEN + _D_EXACT_NAMELEN(d)=2; +#endif + errno = 0; + return d; + }; + + /* Fail. */ + errno = ENOENT; + return NULL; +} +EOF + +# Then compile/link it: +gcc_shared_ k.c k.so \ + || framework_failure_ 'failed to build shared library' + +# Test if LD_PRELOAD works: +export READDIR_PARTIAL +for READDIR_PARTIAL in '' '1'; do + rm -f preloaded + (LD_PRELOAD=$LD_PRELOAD:./k.so returns_ 1 rm -Rf dir 2>>err) || fail=1 + test -f preloaded || + skip_ "internal test failure: maybe LD_PRELOAD doesn't work?" +done + +# First case is failure to read any items from dir, then assume empty. +# Generally that will be diagnosed when rm tries to rmdir(). +# Second case is more general error where we fail immediately +# (with ENOENT in this case but it could be anything). +cat < exp +rm: cannot remove 'dir': Directory not empty +rm: traversal failed: dir: No such file or directory +EOF + +compare exp err || fail=1 + +Exit $fail