testing/coreutils
SHA256
3
0
Fork 0
forked from pool/coreutils
Code Pull Requests Activity

Accepting request 416999 from Base:System

Browse Source 
- coreutils-maint-fix-dependency-of-man-arch.1.patch: Add Upstream
  patch to fix the build dependency between src/arch -> man/arch.1
  which lead to spurious build failures.
- coreutils-df-hash-in-filter.patch: Refresh with -p0.
- Add coreutils-df-hash-in-filter.patch that speeds up df.

OBS-URL: https://build.opensuse.org/request/show/416999
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/coreutils?expand=0&rev=119
This commit is contained in:
Dominique Leuenberger 2016-08-05 16:11:47 +00:00 committed by Git OBS Bridge
commit 68d2f243f6
6 changed files with 420 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

304
coreutils-df-hash-in-filter.patch Normal file
View File

@ -0,0 +1,304 @@
# Upstream patch to improve df performance with many mount points;
# to be removed with coreutils > v8.25.
Upstream patch:
http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=1c17f61ef9
Downstream addition:
* THANKS: Propagate the addition of Josef Cejka from THANKS.in
to this file (usually done via "make dist").
______________________________________________________________________
From 1c17f61ef993a5ee5fb0d3bc47b7b25782ae386c Mon Sep 17 00:00:00 2001
From: Philipp Thomas <pth@suse.de>
Date: Sun, 31 Jul 2016 21:24:18 +0200
Subject: [PATCH] df: improve performance with many mount points
Use hash table for seaching in filter_mount_list() and get_dev()
This improves performance for 20K mount entries from:
real 0m1.731s
user 0m0.532s
sys 0m1.188s
to:
real 0m1.066s
user 0m0.028s
sys 0m1.032s
* src/df.c (devlist_table): Define hash table.
(devlist_hash): Add hash function.
(devlist_compare): Add hash comparison function.
(devlist_for_dev): Add lookup function.
(devlist_free): Add cleanup function.
(filter_mount_list): Use the above hash table.
While at it, rename the variable 'devlist' to 'seen_dev' for
better readability.
(me_for_dev): Use the above lookup function.
NEWS: Mention the improvement.
THANKS.in: Remove the committer; add original submitter Josef Cejka.
---
NEWS | 3 +
THANKS | 1
THANKS.in | 2 -
src/df.c | 110 +++++++++++++++++++++++++++++++++++++++++++-------------------
4 files changed, 82 insertions(+), 34 deletions(-)
Index: NEWS
===================================================================
--- NEWS.orig
+++ NEWS
@@ -8,6 +8,9 @@ GNU coreutils NEWS
introduced in coreutils-8.0. du, chmod, chgrp and chown started using
fts in 6.0. chcon was added in coreutils-6.9.91 with fts support. ]
+ df now filters the system mount list more efficiently, with 20000
+ mount entries now being processed in about 1.1s compared to 1.7s.
+
* Noteworthy changes in release 8.25 (2016-01-20) [stable]
Index: THANKS.in
===================================================================
--- THANKS.in.orig
+++ THANKS.in
@@ -322,6 +322,7 @@ Jon Peatfield J.S.
Joost van Baal joostvb@xs4all.nl
Jordi Pujol jordipujolp@gmail.com
Jorge Stolfi stolfi@ic.unicamp.br
+Josef Cejka jcejka@suse.com
Joseph D. Wagner joe@josephdwagner.info
Joseph S. Myers jsm28@cam.ac.uk
Josh Triplett josh@freedesktop.org
@@ -514,7 +515,6 @@ Petter Reinholdtsen pere
Phelippe Neveu pneveu@pcigeomatics.com
Phil Richards phil.richards@vf.vodafone.co.uk
Philipp Gortan gortan@gmail.com
-Philipp Thomas pth@suse.de
Philippe De Muyter phdm@macqel.be
Philippe Schnoebelen Philippe.Schnoebelen@imag.fr
Phillip Jones mouse@datastacks.com
Index: src/df.c
===================================================================
--- src/df.c.orig
+++ src/df.c
@@ -34,6 +34,7 @@
#include "mountlist.h"
#include "quote.h"
#include "find-mount-point.h"
+#include "hash.h"
/* The official name of this program (e.g., no 'g' prefix). */
#define PROGRAM_NAME "df"
@@ -43,14 +44,16 @@
proper_name ("David MacKenzie"), \
proper_name ("Paul Eggert")
-/* Filled with device numbers of examined file systems to avoid
- duplicates in output. */
-static struct devlist
+struct devlist
{
dev_t dev_num;
struct mount_entry *me;
struct devlist *next;
-} *device_list;
+};
+
+/* Filled with device numbers of examined file systems to avoid
+ duplicates in output. */
+static Hash_table *devlist_table;
/* If true, show even file systems with zero size or
uninteresting types. */
@@ -603,23 +606,67 @@ excluded_fstype (const char *fstype)
return false;
}
+static size_t
+devlist_hash (void const *x, size_t table_size)
+{
+ struct devlist const *p = x;
+ return (uintmax_t) p->dev_num % table_size;
+}
+
+static bool
+devlist_compare (void const *x, void const *y)
+{
+ struct devlist const *a = x;
+ struct devlist const *b = y;
+ return a->dev_num == b->dev_num;
+}
+
+static struct devlist *
+devlist_for_dev (dev_t dev)
+{
+ if (devlist_table == NULL)
+ return NULL;
+ struct devlist dev_entry;
+ dev_entry.dev_num = dev;
+ return hash_lookup (devlist_table, &dev_entry);
+}
+
+static void
+devlist_free (void *p)
+{
+ free (p);
+}
+
/* Filter mount list by skipping duplicate entries.
In the case of duplicates - based on the device number - the mount entry
with a '/' in its me_devname (i.e., not pseudo name like tmpfs) wins.
If both have a real devname (e.g. bind mounts), then that with the shorter
me_mountdir wins. With DEVICES_ONLY == true (set with df -a), only update
- the global device_list, rather than filtering the global mount_list. */
+ the global devlist_table, rather than filtering the global mount_list. */
static void
filter_mount_list (bool devices_only)
{
struct mount_entry *me;
+ /* Temporary list to keep entries ordered. */
+ struct devlist *device_list = NULL;
+ int mount_list_size = 0;
+
+ for (me = mount_list; me; me = me->me_next)
+ mount_list_size++;
+
+ devlist_table = hash_initialize (mount_list_size, NULL,
+ devlist_hash,
+ devlist_compare,
+ devlist_free);
+ if (devlist_table == NULL)
+ xalloc_die ();
+
/* Sort all 'wanted' entries into the list device_list. */
for (me = mount_list; me;)
{
struct stat buf;
- struct devlist *devlist;
struct mount_entry *discard_me = NULL;
/* Avoid stating remote file systems as that may hang.
@@ -635,21 +682,20 @@ filter_mount_list (bool devices_only)
else
{
/* If we've already seen this device... */
- for (devlist = device_list; devlist; devlist = devlist->next)
- if (devlist->dev_num == buf.st_dev)
- break;
+ struct devlist *seen_dev = devlist_for_dev (buf.st_dev);
- if (devlist)
+ if (seen_dev)
{
- bool target_nearer_root = strlen (devlist->me->me_mountdir)
+ bool target_nearer_root = strlen (seen_dev->me->me_mountdir)
> strlen (me->me_mountdir);
/* With bind mounts, prefer items nearer the root of the source */
- bool source_below_root = devlist->me->me_mntroot != NULL
+ bool source_below_root = seen_dev->me->me_mntroot != NULL
&& me->me_mntroot != NULL
- && (strlen (devlist->me->me_mntroot)
+ && (strlen (seen_dev->me->me_mntroot)
< strlen (me->me_mntroot));
- if (! print_grand_total && me->me_remote && devlist->me->me_remote
- && ! STREQ (devlist->me->me_devname, me->me_devname))
+ if (! print_grand_total
+ && me->me_remote && seen_dev->me->me_remote
+ && ! STREQ (seen_dev->me->me_devname, me->me_devname))
{
/* Don't discard remote entries with different locations,
as these are more likely to be explicitly mounted.
@@ -658,21 +704,21 @@ filter_mount_list (bool devices_only)
}
else if ((strchr (me->me_devname, '/')
/* let "real" devices with '/' in the name win. */
- && ! strchr (devlist->me->me_devname, '/'))
+ && ! strchr (seen_dev->me->me_devname, '/'))
/* let points towards the root of the device win. */
|| (target_nearer_root && ! source_below_root)
/* let an entry overmounted on a new device win... */
- || (! STREQ (devlist->me->me_devname, me->me_devname)
+ || (! STREQ (seen_dev->me->me_devname, me->me_devname)
/* ... but only when matching an existing mnt point,
to avoid problematic replacement when given
inaccurate mount lists, seen with some chroot
environments for example. */
&& STREQ (me->me_mountdir,
- devlist->me->me_mountdir)))
+ seen_dev->me->me_mountdir)))
{
/* Discard mount entry for existing device. */
- discard_me = devlist->me;
- devlist->me = me;
+ discard_me = seen_dev->me;
+ seen_dev->me = me;
}
else
{
@@ -691,12 +737,14 @@ filter_mount_list (bool devices_only)
}
else
{
- /* Add the device number to the global list devlist. */
- devlist = xmalloc (sizeof *devlist);
+ /* Add the device number to the device_table. */
+ struct devlist *devlist = xmalloc (sizeof *devlist);
devlist->me = me;
devlist->dev_num = buf.st_dev;
devlist->next = device_list;
device_list = devlist;
+ if (hash_insert (devlist_table, devlist) == NULL)
+ xalloc_die ();
me = me->me_next;
}
@@ -711,28 +759,24 @@ filter_mount_list (bool devices_only)
me = device_list->me;
me->me_next = mount_list;
mount_list = me;
- /* Free devlist entry and advance. */
- struct devlist *devlist = device_list->next;
- free (device_list);
- device_list = devlist;
+ device_list = device_list->next;
}
+
+ hash_free (devlist_table);
+ devlist_table = NULL;
}
}
+
/* Search a mount entry list for device id DEV.
Return the corresponding mount entry if found or NULL if not. */
static struct mount_entry const * _GL_ATTRIBUTE_PURE
me_for_dev (dev_t dev)
{
- struct devlist *dl = device_list;
-
- while (dl)
- {
- if (dl->dev_num == dev)
+ struct devlist *dl = devlist_for_dev (dev);
+ if (dl)
return dl->me;
- dl = dl->next;
- }
return NULL;
}
Index: THANKS
===================================================================
--- THANKS.orig
+++ THANKS
@@ -406,6 +406,7 @@ Jon Ringuette jonr
Joost van Baal joostvb@xs4all.nl
Jordi Pujol jordipujolp@gmail.com
Jorge Stolfi stolfi@ic.unicamp.br
+Josef Cejka jcejka@suse.com
Joseph D. Wagner joe@josephdwagner.info
Joseph S. Myers jsm28@cam.ac.uk
Josh Triplett josh@freedesktop.org

70
coreutils-maint-fix-dependency-of-man-arch.1.patch Normal file
View File

@ -0,0 +1,70 @@
Upstream patch to fix the build dependency between src/arch -> man/arch.1;
to be removed with coreutils > v8.25.
Downstream addition:
* Propagate the change from local.mk to Makefile.in, which is done
upstreams by autoconf.
Original patch:
http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=a69e54cfdf7e
From a69e54cfdf7e5d8c2c3fe315ded649272d7e8711 Mon Sep 17 00:00:00 2001
From: Bernhard Voelker <mail@bernhard-voelker.de>
Date: Tue, 26 Jul 2016 20:58:08 +0200
Subject: [PATCH] maint: fix dependency of man/arch.1
The following is a reproducer for the wrong dependency:
$ ./configure --enable-install-program=arch
$ make
$ rm -f src/arch man/arch.1
$ make man/arch.1
GEN man/arch.1
help2man: can't get `--help' info from man/arch.td/arch
Try `--no-discard-stderr' if option outputs to stderr
Makefile:14378: recipe for target 'man/arch.1' failed
make: *** [man/arch.1] Error 127
* man/local.mk (man/arch.1): Change to depend on src/arch rather than
src/uname: while the arch binary depends on uname.c and uname-arch.c,
its man page depends on the arch binary.
Reported downstream by Rodrigues Goldwyn <rgoldwyn@suse.com> in
https://build.opensuse.org/request/show/415172
---
Makefile.in | 2 +-
man/local.mk | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
Index: man/local.mk
===================================================================
--- man/local.mk.orig
+++ man/local.mk
@@ -56,10 +56,10 @@ if SINGLE_BINARY
mandeps += src/coreutils$(EXEEXT)
else
# Most prog.1 man pages depend on src/prog. List the exceptions:
-man/arch.1: src/uname$(EXEEXT)
man/install.1: src/ginstall$(EXEEXT)
man/test.1: src/[$(EXEEXT)
+man/arch.1: src/arch$(EXEEXT)
man/base32.1: src/base32$(EXEEXT)
man/base64.1: src/base64$(EXEEXT)
man/basename.1: src/basename$(EXEEXT)
Index: Makefile.in
===================================================================
--- Makefile.in.orig
+++ Makefile.in
@@ -14276,10 +14276,10 @@ distclean-local:
$(ALL_MANS): $(mandeps)
# Most prog.1 man pages depend on src/prog. List the exceptions:
-@SINGLE_BINARY_FALSE@man/arch.1: src/uname$(EXEEXT)
@SINGLE_BINARY_FALSE@man/install.1: src/ginstall$(EXEEXT)
@SINGLE_BINARY_FALSE@man/test.1: src/[$(EXEEXT)
+@SINGLE_BINARY_FALSE@man/arch.1: src/arch$(EXEEXT)
@SINGLE_BINARY_FALSE@man/base32.1: src/base32$(EXEEXT)
@SINGLE_BINARY_FALSE@man/base64.1: src/base64$(EXEEXT)
@SINGLE_BINARY_FALSE@man/basename.1: src/basename$(EXEEXT)

13
coreutils-testsuite.changes
View File

@ -1,3 +1,16 @@
-------------------------------------------------------------------
Thu Jul 28 17:02:53 UTC 2016 - mail@bernhard-voelker.de
- coreutils-maint-fix-dependency-of-man-arch.1.patch: Add Upstream
patch to fix the build dependency between src/arch -> man/arch.1
which lead to spurious build failures.
- coreutils-df-hash-in-filter.patch: Refresh with -p0.
-------------------------------------------------------------------
Fri Jul 22 10:48:50 CEST 2016 - pth@suse.de
- Add coreutils-df-hash-in-filter.patch that speeds up df.
-------------------------------------------------------------------
Wed Jul 6 06:54:54 UTC 2016 - mail@bernhard-voelker.de

10
coreutils-testsuite.spec
View File

@ -132,6 +132,14 @@ Patch501: coreutils-test_without_valgrind.patch
# to be removed with coreutils > v8.25.
Patch700: coreutils-diagnose-fts-readdir-failure.patch
# Upstream patch to improve df performance with many mount points;
# to be removed with coreutils > v8.25.
Patch705: coreutils-df-hash-in-filter.patch
# Upstream patch to fix the build dependency between src/arch -> man/arch.1;
# to be removed with coreutils > v8.25.
Patch710: coreutils-maint-fix-dependency-of-man-arch.1.patch
# ================================================
%description
These are the GNU core utilities. This package is the union of
@ -175,6 +183,8 @@ the GNU fileutils, sh-utils, and textutils packages.
%patch501
%patch700
%patch705
%patch710
#???## We need to statically link to gmp, otherwise we have a build loop
#???#sed -i s,'$(LIB_GMP)',%%{_libdir}/libgmp.a,g Makefile.in

13
coreutils.changes
View File

@ -1,3 +1,16 @@
-------------------------------------------------------------------
Thu Jul 28 17:02:53 UTC 2016 - mail@bernhard-voelker.de
- coreutils-maint-fix-dependency-of-man-arch.1.patch: Add Upstream
patch to fix the build dependency between src/arch -> man/arch.1
which lead to spurious build failures.
- coreutils-df-hash-in-filter.patch: Refresh with -p0.
-------------------------------------------------------------------
Fri Jul 22 10:48:50 CEST 2016 - pth@suse.de
- Add coreutils-df-hash-in-filter.patch that speeds up df.
-------------------------------------------------------------------
Wed Jul 6 06:54:54 UTC 2016 - mail@bernhard-voelker.de

10
coreutils.spec
View File

@ -132,6 +132,14 @@ Patch501: coreutils-test_without_valgrind.patch
# to be removed with coreutils > v8.25.
Patch700: coreutils-diagnose-fts-readdir-failure.patch
# Upstream patch to improve df performance with many mount points;
# to be removed with coreutils > v8.25.
Patch705: coreutils-df-hash-in-filter.patch
# Upstream patch to fix the build dependency between src/arch -> man/arch.1;
# to be removed with coreutils > v8.25.
Patch710: coreutils-maint-fix-dependency-of-man-arch.1.patch
# ================================================
%description
These are the GNU core utilities. This package is the union of
@ -175,6 +183,8 @@ the GNU fileutils, sh-utils, and textutils packages.
%patch501
%patch700
%patch705
%patch710
#???## We need to statically link to gmp, otherwise we have a build loop
#???#sed -i s,'$(LIB_GMP)',%%{_libdir}/libgmp.a,g Makefile.in