From 40d962d80b6f2e9d04428c5d1944834de1208fac Mon Sep 17 00:00:00 2001 From: Harald Sitter Date: Wed, 12 Sep 2018 16:58:04 +0200 Subject: [PATCH] [sftp] bump pending request count from 1 to 128 Summary: with the previous value we basically did sync reading which means that network and cyrptographic overhead head a huge impact on throughput. meanwhile the perfect way to use asyncness is to schedule a whole bunch of requests before starting to read. previously this was documented as auto-adjusting, which it never was, there's also little to be gained from adjusting this value on the fly. more requests in most scenarios will simply mean a larger RAM footprint as more data potentially sits in libssh waiting to be read. with 128 requests that'd be ~8mb (assuming the file transferred is that large) this improves read performance with libssh 0.8 by up to 20 times for large files. read performance with libssh 0.6 is 2 to 3 times better. the faster the connection the higher the gain of course. 128 gives somewhat competitive performance results compared to openssh's ssh implementations while not having too large a footprint. in raw numbers: a local link read was averaging around 10mb/s on both libssh versions. with 128 requests this goes up to 200. competitive is between 200 and 300 it seems (obviously all specific to my system) CHANGELOG: sftp file reading is now up to 20 times faster Reviewers: broulik Reviewed By: broulik Differential Revision: https://phabricator.kde.org/D15452 --- sftp/kio_sftp.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sftp/kio_sftp.h b/sftp/kio_sftp.h index cc6b9e09..e5639970 100644 --- a/sftp/kio_sftp.h +++ b/sftp/kio_sftp.h @@ -141,10 +141,11 @@ private: // Private variables * @param file the sftp_file object which should be transferred. * @param sb the attributes of that sftp_file object. * @param maxPendingRequests the maximum number of parallel requests to start with. - * The number will be adjusted automatically depending - * on the connection speed. + * The more are pending the higher the potential memory + * foot print, however if the connection allows it + * we'll get better throughput. */ - GetRequest(sftp_file file, sftp_attributes sb, ushort maxPendingRequests = 1); + GetRequest(sftp_file file, sftp_attributes sb, ushort maxPendingRequests = 128); /** * Removes all pending requests and closes the SFTP channel and attributes * in order to avoid memory leaks. -- 2.18.0