rpm/coreutils
SHA256
1
0
Fork 0
forked from pool/coreutils
Code Pull Requests Activity

- coreutils-df-hash-in-filter.patch: Refresh with now-upstream version.

Browse Source 
- coreutils.spec, coreutils-testsuite.spec: Change number of the above
patch to match the order of applying.

OBS-URL: https://build.opensuse.org/package/show/Base:System/coreutils?expand=0&rev=276
This commit is contained in:
Bernhard Voelker 2016-08-04 19:18:42 +00:00 committed by Git OBS Bridge
parent 7790f3a24f
commit 838c7971b1
3 changed files with 174 additions and 30 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

186
coreutils-df-hash-in-filter.patch
View File

@ -1,17 +1,82 @@
# Upstream patch to improve df performance with many mount points;
# to be removed with coreutils > v8.25.
Upstream patch:
http://git.sv.gnu.org/cgit/coreutils.git/commit/?id=1c17f61ef9
Downstream addition:
* THANKS: Propagate the addition of Josef Cejka from THANKS.in
to this file (usually done via "make dist").
______________________________________________________________________
From 1c17f61ef993a5ee5fb0d3bc47b7b25782ae386c Mon Sep 17 00:00:00 2001
From: Philipp Thomas <pth@suse.de>
Date: 2016-07-22 10:34:59+02:00
Subject: speed up df
References:
Upstream:
Date: Sun, 31 Jul 2016 21:24:18 +0200
Subject: [PATCH] df: improve performance with many mount points
Use hash table for seaching in filter_mount_list() and get_dev()
Use hash table for seaching in filter_mount_list() and get_dev().
Patch was done by Josef Cejka <jcejka@suse.com>.
This improves performance for 20K mount entries from:
real 0m1.731s
user 0m0.532s
sys 0m1.188s
to:
real 0m1.066s
user 0m0.028s
sys 0m1.032s
* src/df.c (devlist_table): Define hash table.
(devlist_hash): Add hash function.
(devlist_compare): Add hash comparison function.
(devlist_for_dev): Add lookup function.
(devlist_free): Add cleanup function.
(filter_mount_list): Use the above hash table.
While at it, rename the variable 'devlist' to 'seen_dev' for
better readability.
(me_for_dev): Use the above lookup function.
NEWS: Mention the improvement.
THANKS.in: Remove the committer; add original submitter Josef Cejka.
---
src/df.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++---------------
1 file changed, 62 insertions(+), 18 deletions(-)
NEWS | 3 +
THANKS | 1
THANKS.in | 2 -
src/df.c | 110 +++++++++++++++++++++++++++++++++++++++++++-------------------
4 files changed, 82 insertions(+), 34 deletions(-)
Index: NEWS
===================================================================
--- NEWS.orig
+++ NEWS
@@ -8,6 +8,9 @@ GNU coreutils NEWS
introduced in coreutils-8.0. du, chmod, chgrp and chown started using
fts in 6.0. chcon was added in coreutils-6.9.91 with fts support. ]
+ df now filters the system mount list more efficiently, with 20000
+ mount entries now being processed in about 1.1s compared to 1.7s.
+
* Noteworthy changes in release 8.25 (2016-01-20) [stable]
Index: THANKS.in
===================================================================
--- THANKS.in.orig
+++ THANKS.in
@@ -322,6 +322,7 @@ Jon Peatfield J.S.
Joost van Baal joostvb@xs4all.nl
Jordi Pujol jordipujolp@gmail.com
Jorge Stolfi stolfi@ic.unicamp.br
+Josef Cejka jcejka@suse.com
Joseph D. Wagner joe@josephdwagner.info
Joseph S. Myers jsm28@cam.ac.uk
Josh Triplett josh@freedesktop.org
@@ -514,7 +515,6 @@ Petter Reinholdtsen pere
Phelippe Neveu pneveu@pcigeomatics.com
Phil Richards phil.richards@vf.vodafone.co.uk
Philipp Gortan gortan@gmail.com
-Philipp Thomas pth@suse.de
Philippe De Muyter phdm@macqel.be
Philippe Schnoebelen Philippe.Schnoebelen@imag.fr
Phillip Jones mouse@datastacks.com
Index: src/df.c
===================================================================
--- src/df.c.orig
@ -45,7 +110,7 @@ Index: src/df.c
/* If true, show even file systems with zero size or
uninteresting types. */
@@ -603,6 +606,37 @@ excluded_fstype (const char *fstype)
@@ -603,23 +606,67 @@ excluded_fstype (const char *fstype)
return false;
}
@ -67,27 +132,33 @@ Index: src/df.c
+static struct devlist *
+devlist_for_dev (dev_t dev)
+{
+ struct devlist dev_entry;
+ dev_entry.dev_num = dev;
+ if (devlist_table == NULL)
+ return NULL;
+ return hash_lookup(devlist_table, &dev_entry);
+ struct devlist dev_entry;
+ dev_entry.dev_num = dev;
+ return hash_lookup (devlist_table, &dev_entry);
+}
+
+static void
+devlist_free(void *p)
+devlist_free (void *p)
+{
+ free(p);
+ free (p);
+}
+
/* Filter mount list by skipping duplicate entries.
In the case of duplicates - based on the device number - the mount entry
with a '/' in its me_devname (i.e., not pseudo name like tmpfs) wins.
@@ -615,6 +649,20 @@ filter_mount_list (bool devices_only)
If both have a real devname (e.g. bind mounts), then that with the shorter
me_mountdir wins. With DEVICES_ONLY == true (set with df -a), only update
- the global device_list, rather than filtering the global mount_list. */
+ the global devlist_table, rather than filtering the global mount_list. */
static void
filter_mount_list (bool devices_only)
{
struct mount_entry *me;
+ /* temp list to keep entries ordered */
+ /* Temporary list to keep entries ordered. */
+ struct devlist *device_list = NULL;
+ int mount_list_size = 0;
+
@ -104,18 +175,77 @@ Index: src/df.c
/* Sort all 'wanted' entries into the list device_list. */
for (me = mount_list; me;)
{
@@ -635,9 +683,7 @@ filter_mount_list (bool devices_only)
struct stat buf;
- struct devlist *devlist;
struct mount_entry *discard_me = NULL;
/* Avoid stating remote file systems as that may hang.
@@ -635,21 +682,20 @@ filter_mount_list (bool devices_only)
else
{
/* If we've already seen this device... */
- for (devlist = device_list; devlist; devlist = devlist->next)
- if (devlist->dev_num == buf.st_dev)
- break;
+ devlist = devlist_for_dev(buf.st_dev);
+ struct devlist *seen_dev = devlist_for_dev (buf.st_dev);
if (devlist)
- if (devlist)
+ if (seen_dev)
{
@@ -697,6 +743,8 @@ filter_mount_list (bool devices_only)
- bool target_nearer_root = strlen (devlist->me->me_mountdir)
+ bool target_nearer_root = strlen (seen_dev->me->me_mountdir)
> strlen (me->me_mountdir);
/* With bind mounts, prefer items nearer the root of the source */
- bool source_below_root = devlist->me->me_mntroot != NULL
+ bool source_below_root = seen_dev->me->me_mntroot != NULL
&& me->me_mntroot != NULL
- && (strlen (devlist->me->me_mntroot)
+ && (strlen (seen_dev->me->me_mntroot)
< strlen (me->me_mntroot));
- if (! print_grand_total && me->me_remote && devlist->me->me_remote
- && ! STREQ (devlist->me->me_devname, me->me_devname))
+ if (! print_grand_total
+ && me->me_remote && seen_dev->me->me_remote
+ && ! STREQ (seen_dev->me->me_devname, me->me_devname))
{
/* Don't discard remote entries with different locations,
as these are more likely to be explicitly mounted.
@@ -658,21 +704,21 @@ filter_mount_list (bool devices_only)
}
else if ((strchr (me->me_devname, '/')
/* let "real" devices with '/' in the name win. */
- && ! strchr (devlist->me->me_devname, '/'))
+ && ! strchr (seen_dev->me->me_devname, '/'))
/* let points towards the root of the device win. */
|| (target_nearer_root && ! source_below_root)
/* let an entry overmounted on a new device win... */
- || (! STREQ (devlist->me->me_devname, me->me_devname)
+ || (! STREQ (seen_dev->me->me_devname, me->me_devname)
/* ... but only when matching an existing mnt point,
to avoid problematic replacement when given
inaccurate mount lists, seen with some chroot
environments for example. */
&& STREQ (me->me_mountdir,
- devlist->me->me_mountdir)))
+ seen_dev->me->me_mountdir)))
{
/* Discard mount entry for existing device. */
- discard_me = devlist->me;
- devlist->me = me;
+ discard_me = seen_dev->me;
+ seen_dev->me = me;
}
else
{
@@ -691,12 +737,14 @@ filter_mount_list (bool devices_only)
}
else
{
- /* Add the device number to the global list devlist. */
- devlist = xmalloc (sizeof *devlist);
+ /* Add the device number to the device_table. */
+ struct devlist *devlist = xmalloc (sizeof *devlist);
devlist->me = me;
devlist->dev_num = buf.st_dev;
devlist->next = device_list;
device_list = devlist;
@ -135,7 +265,7 @@ Index: src/df.c
+ device_list = device_list->next;
}
+
+ hash_free(devlist_table);
+ hash_free (devlist_table);
+ devlist_table = NULL;
}
}
@ -152,7 +282,7 @@ Index: src/df.c
- while (dl)
- {
- if (dl->dev_num == dev)
+ struct devlist *dl = devlist_for_dev(dev);
+ struct devlist *dl = devlist_for_dev (dev);
+ if (dl)
return dl->me;
- dl = dl->next;
@ -160,3 +290,15 @@ Index: src/df.c
return NULL;
}
Index: THANKS
===================================================================
--- THANKS.orig
+++ THANKS
@@ -406,6 +406,7 @@ Jon Ringuette jonr
Joost van Baal joostvb@xs4all.nl
Jordi Pujol jordipujolp@gmail.com
Jorge Stolfi stolfi@ic.unicamp.br
+Josef Cejka jcejka@suse.com
Joseph D. Wagner joe@josephdwagner.info
Joseph S. Myers jsm28@cam.ac.uk
Josh Triplett josh@freedesktop.org

9
coreutils-testsuite.spec
View File

@ -114,9 +114,6 @@ Patch112: coreutils-getaddrinfo.patch
# Assorted fixes
Patch113: coreutils-misc.patch
#use hash table for seaching in filter_mount_list() and get_dev()
Patch114: coreutils-df-hash-in-filter.patch
# Skip 2 valgrind'ed sort tests on ppc/ppc64 which would fail due to
# a glibc issue in mkstemp.
Patch300: coreutils-skip-some-sort-tests-on-ppc.patch
@ -135,6 +132,10 @@ Patch501: coreutils-test_without_valgrind.patch
# to be removed with coreutils > v8.25.
Patch700: coreutils-diagnose-fts-readdir-failure.patch
# Upstream patch to improve df performance with many mount points;
# to be removed with coreutils > v8.25.
Patch705: coreutils-df-hash-in-filter.patch
# Upstream patch to fix the build dependency between src/arch -> man/arch.1;
# to be removed with coreutils > v8.25.
Patch710: coreutils-maint-fix-dependency-of-man-arch.1.patch
@ -182,7 +183,7 @@ the GNU fileutils, sh-utils, and textutils packages.
%patch501
%patch700
%patch114
%patch705
%patch710
#???## We need to statically link to gmp, otherwise we have a build loop

9
coreutils.spec
View File

@ -114,9 +114,6 @@ Patch112: coreutils-getaddrinfo.patch
# Assorted fixes
Patch113: coreutils-misc.patch
#use hash table for seaching in filter_mount_list() and get_dev()
Patch114: coreutils-df-hash-in-filter.patch
# Skip 2 valgrind'ed sort tests on ppc/ppc64 which would fail due to
# a glibc issue in mkstemp.
Patch300: coreutils-skip-some-sort-tests-on-ppc.patch
@ -135,6 +132,10 @@ Patch501: coreutils-test_without_valgrind.patch
# to be removed with coreutils > v8.25.
Patch700: coreutils-diagnose-fts-readdir-failure.patch
# Upstream patch to improve df performance with many mount points;
# to be removed with coreutils > v8.25.
Patch705: coreutils-df-hash-in-filter.patch
# Upstream patch to fix the build dependency between src/arch -> man/arch.1;
# to be removed with coreutils > v8.25.
Patch710: coreutils-maint-fix-dependency-of-man-arch.1.patch
@ -182,7 +183,7 @@ the GNU fileutils, sh-utils, and textutils packages.
%patch501
%patch700
%patch114
%patch705
%patch710
#???## We need to statically link to gmp, otherwise we have a build loop