3faff776a8
* Increase sftp file reading speed up to 20 times OBS-URL: https://build.opensuse.org/package/show/KDE:Applications/kio-extras5?expand=0&rev=85
61 lines
2.6 KiB
Diff
61 lines
2.6 KiB
Diff
From 40d962d80b6f2e9d04428c5d1944834de1208fac Mon Sep 17 00:00:00 2001
|
|
From: Harald Sitter <sitter@kde.org>
|
|
Date: Wed, 12 Sep 2018 16:58:04 +0200
|
|
Subject: [PATCH] [sftp] bump pending request count from 1 to 128
|
|
|
|
Summary:
|
|
with the previous value we basically did sync reading which means that
|
|
network and cyrptographic overhead head a huge impact on throughput.
|
|
meanwhile the perfect way to use asyncness is to schedule a whole bunch of
|
|
requests before starting to read.
|
|
previously this was documented as auto-adjusting, which it never was,
|
|
there's also little to be gained from adjusting this value on the fly.
|
|
more requests in most scenarios will simply mean a larger RAM footprint as
|
|
more data potentially sits in libssh waiting to be read. with 128 requests
|
|
that'd be ~8mb (assuming the file transferred is that large)
|
|
|
|
this improves read performance with libssh 0.8 by up to 20 times for large
|
|
files. read performance with libssh 0.6 is 2 to 3 times better.
|
|
the faster the connection the higher the gain of course.
|
|
|
|
128 gives somewhat competitive performance results compared to openssh's
|
|
ssh implementations while not having too large a footprint.
|
|
|
|
in raw numbers: a local link read was averaging around 10mb/s on both
|
|
libssh versions. with 128 requests this goes up to 200. competitive is
|
|
between 200 and 300 it seems (obviously all specific to my system)
|
|
|
|
CHANGELOG: sftp file reading is now up to 20 times faster
|
|
|
|
Reviewers: broulik
|
|
|
|
Reviewed By: broulik
|
|
|
|
Differential Revision: https://phabricator.kde.org/D15452
|
|
---
|
|
sftp/kio_sftp.h | 7 ++++---
|
|
1 file changed, 4 insertions(+), 3 deletions(-)
|
|
|
|
diff --git a/sftp/kio_sftp.h b/sftp/kio_sftp.h
|
|
index cc6b9e09..e5639970 100644
|
|
--- a/sftp/kio_sftp.h
|
|
+++ b/sftp/kio_sftp.h
|
|
@@ -141,10 +141,11 @@ private: // Private variables
|
|
* @param file the sftp_file object which should be transferred.
|
|
* @param sb the attributes of that sftp_file object.
|
|
* @param maxPendingRequests the maximum number of parallel requests to start with.
|
|
- * The number will be adjusted automatically depending
|
|
- * on the connection speed.
|
|
+ * The more are pending the higher the potential memory
|
|
+ * foot print, however if the connection allows it
|
|
+ * we'll get better throughput.
|
|
*/
|
|
- GetRequest(sftp_file file, sftp_attributes sb, ushort maxPendingRequests = 1);
|
|
+ GetRequest(sftp_file file, sftp_attributes sb, ushort maxPendingRequests = 128);
|
|
/**
|
|
* Removes all pending requests and closes the SFTP channel and attributes
|
|
* in order to avoid memory leaks.
|
|
--
|
|
2.18.0
|
|
|