pool/ndpi
SHA256
4
0
Fork 1
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Accepting request 914423 from server:monitoring

Browse Source 
- Add conflicts for ndpi-common package, as version 3 did not follow
  packaging guidelines fully
- Create -common subpackage
- Update to version 4.0
  New Features
  * Add API for computing RSI (Relative Strenght Index)
  * Add GeoIP support
  * Add fragments management
  * Add API for jitter calculation
  * Add single exponential smoothing API
  * Add timeseries forecasting support implementing Holt-Winters
    with confidence interval
  * Add support for MAC to radi tree and expose the full API to
    applications
  * Add JA3+, with ALPN and elliptic curve
  * Add double exponential smoothing implementation
  * Extended API for managing flow risks
  * Add flow risk score
  * New flow risks:
    + Desktop or File Sharing Session
    + HTTP suspicious content (useful for tracking trickbot)
    + Malicious JA3
    + Malicious SHA1
    + Risky domain
    + Risky AS
    + TLS Certificate Validity Too Long
    + TLS Suspicious Extension
  New Supported Protocols and Services
  * New protocols:
    + AmongUs
    + AVAST SecureDNS
    + CPHA (CheckPoint High Availability Protocol)
    + DisneyPlus
    + DTLS
    + Genshin Impact
    + HP Virtual Machine Group Management (hpvirtgrp)
    + Mongodb
    + Pinterest
    + Reddit
    + Snapchat VoIP calls
    + Tumblr
    + Virtual Asssitant (Alexa, Siri)
    + Z39.50
  * Add protocols to HTTP as subprotocols
  * Add detection of TLS browser type
  * Add connectionless DCE/RPC detection
  Improvements
  * 2.5x speed bump. Example ndpiReader with a long mixed pcap
    v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec
    v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec
  * Improve detection/dissection of:
    + AnyDesk
    + DNS
    + Hulu
    + DCE/RPC (avoid false positives)
    + dnscrypt
    + Facebook (add new networks)
    + Fortigate
    + FTP Control
    + HTTP
      - Fix user-agent parsing
      - Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined
    + IEC104
    + IEC60870
    + IRC
    + Netbios
    + Netflix
    + Ookla speedtest (detection over IPv6)
    + openspeedtest.com
    + Outlook / MicrosoftMail
    + QUIC
      - update to draft-33
      - improve handling of SNI
      - support for fragmented Client Hello
      - support for DNS-over-QUIC
    + RTSP
    + RTSP via HTTP
    + SNMP (reimplemented)
    + Skype
    + SSH
    + Steam (Steam Datagram Relay - SDR)
    + STUN (avoid false positives, improved Skype detection)
    + TeamViewer (add new hosts)
    + TOR (update hosts)
    + TLS
      - Certificate Subject matching
      - Check for common ALPNs
      - Reworked fingerprint calculation
      - Fix extraction for TLS signature algorithms
      - Fix ClientHello parsing
    + UPnP
    + wireguard
    + Improve DGA detection
    + Improve JA3
    + Improve Mining detection
    + Improve string matching algorithm
    + Improve ndpi_pref_enable_tls_block_dissection
    + Optimize speed and memory size
    + Update ahocorasick library
    + Improve subprotocols detection
  Fixes
  * Fix partial application matching
  * Fix multiple segfault and leaks
  * Fix uninitialized memory use
  * Fix release of patterns allocated in ndpi_add_string_to_automa
  * Fix return value of ndpi_match_string_subprotocol
  * Fix setting of flow risks on 32 bit machines
  * Fix TLS certificate threshold
  * Fix a memory error in TLS JA3 code
  * Fix false positives in Z39.50
  * Fix off-by-one memory error for TLS-JA3
  * Fix bug in ndpi_lru_find_cache
  * Fix invalid xbox and playstation port guesses
  * Fix CAPWAP tunnel decoding
  * Fix parsing of DLT_PPP datalink type
  * Fix dissection of QUIC initial packets coalesced with 0-RTT one
  * Fix parsing of GTP headers
  * Add bitmap boundary checks
  Misc
  * Update download category name
  * Update category labels
  * Renamed Skype in Skype_Teams (the protocol is now shared across
    these apps)
  * Add IEC analysis wireshark plugin
  * Flow risk visualization in Wireshark
  * ndpiReader
    + add statistics about nDPI performance
    + fix memory leak
    + fix collecting of risks statistics
  * Move installed libraries from /usr/local to /usr
  * Improve NDPI_API_VERSION generation
  * Update ndpi_ptree_match_addr prototype
- Add patches (for compatibility with ntopng 5.0):
  * 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch
  * 0002-Report-whether-a-protocol-is-encrypted.patch
  * 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch

OBS-URL: https://build.opensuse.org/request/show/914423
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/ndpi?expand=0&rev=10
This commit is contained in:
Dominique Leuenberger 2021-08-26 21:14:57 +00:00 committed by Git OBS Bridge
commit 3580238835
7 changed files with 1596 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1343
0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch Normal file
View File

File diff suppressed because it is too large Load Diff

26
0002-Report-whether-a-protocol-is-encrypted.patch Normal file
View File

@ -0,0 +1,26 @@
From 5f286a17c1ecb88b06ee069650fa73f7565165dc Mon Sep 17 00:00:00 2001
From: Luca Deri <deri@ntop.org>
Date: Sat, 7 Aug 2021 17:35:34 +0200
Subject: [PATCH 2/3] Report whether a protocol is encrypted
---
example/ndpiReader.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/example/ndpiReader.c b/example/ndpiReader.c
index b4434650..053dc2ec 100644
--- a/example/ndpiReader.c
+++ b/example/ndpiReader.c
@@ -1368,6 +1368,9 @@ static void printFlow(u_int32_t id, struct ndpi_flow_info *flow, u_int16_t threa
ndpi_protocol2name(ndpi_thread_info[thread_id].workflow->ndpi_struct,
flow->detected_protocol, buf1, sizeof(buf1)));
+ fprintf(out, "[%s]",
+ ndpi_is_encrypted_proto(ndpi_thread_info[thread_id].workflow->ndpi_struct, flow->detected_protocol) ? "Encrypted" : "ClearText");
+
if(flow->detected_protocol.category != 0)
fprintf(out, "[cat: %s/%u]",
ndpi_category_get_name(ndpi_thread_info[thread_id].workflow->ndpi_struct,
--
2.32.0

43
0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch Normal file
View File

@ -0,0 +1,43 @@
From 817c00b65ab4d0bf78927e494227ac5308417f91 Mon Sep 17 00:00:00 2001
From: Luca Deri <deri@ntop.org>
Date: Wed, 18 Aug 2021 11:33:33 +0200
Subject: [PATCH 3/3] Firs crash on ARM during steam protocol dissection
---
src/lib/protocols/steam.c | 19 ++++++++++++-------
1 file changed, 12 insertions(+), 7 deletions(-)
diff --git a/src/lib/protocols/steam.c b/src/lib/protocols/steam.c
index 8cd3ec41..53bbfc6a 100644
--- a/src/lib/protocols/steam.c
+++ b/src/lib/protocols/steam.c
@@ -114,14 +114,19 @@ static void ndpi_check_steam_udp1(struct ndpi_detection_module_struct *ndpi_stru
}
/* Check for Steam Datagram Relay (SDR) packets. */
- if (payload_len > 8 &&
- ndpi_ntohll(get_u_int64_t(packet->payload, 0)) == 0x0101736470696e67 /* "\x01\x01sdping" */)
- {
- NDPI_LOG_INFO(ndpi_struct, "found STEAM (Steam Datagram Relay)\n");
- ndpi_int_steam_add_connection(ndpi_struct, flow);
- return;
- }
+ if (payload_len > 8) {
+ u_int64_t n;
+ /* Necessary as simple cast crashes on ARM */
+ memcpy(&n, packet->payload, sizeof(u_int64_t));
+
+ if(ndpi_ntohll(n) == 0x0101736470696e67 /* "\x01\x01sdping" */) {
+ NDPI_LOG_INFO(ndpi_struct, "found STEAM (Steam Datagram Relay)\n");
+ ndpi_int_steam_add_connection(ndpi_struct, flow);
+ return;
+ }
+ }
+
/* Check if we so far detected the protocol in the request or not. */
if (flow->steam_stage1 == 0) {
NDPI_LOG_DBG2(ndpi_struct, "STEAM stage 0: \n");
--
2.32.0

3
ndpi-3.4.tar.gz
View File

@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dc9b291c7fde94edb45fb0f222e0d93c93f8d6d37f4efba20ebd9c655bfcedf9
size 37976087

3
ndpi-4.0.tar.gz Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:99e0aba6396fd633c3840f30e4942f6591a08066d037f560b65ba64e7310f4d6
size 120053617

148
ndpi.changes
View File

@ -1,3 +1,151 @@
-------------------------------------------------------------------
Thu Aug 26 16:30:53 UTC 2021 - Dirk Stoecker <opensuse@dstoecker.de>
- Add conflicts for ndpi-common package, as version 3 did not follow
packaging guidelines fully
-------------------------------------------------------------------
Thu Aug 26 09:15:54 UTC 2021 - Martin Hauke <mardnh@gmx.de>
- Create -common subpackage
-------------------------------------------------------------------
Sun Aug 22 12:48:59 UTC 2021 - Martin Hauke <mardnh@gmx.de>
- Update to version 4.0
New Features
* Add API for computing RSI (Relative Strenght Index)
* Add GeoIP support
* Add fragments management
* Add API for jitter calculation
* Add single exponential smoothing API
* Add timeseries forecasting support implementing Holt-Winters
with confidence interval
* Add support for MAC to radi tree and expose the full API to
applications
* Add JA3+, with ALPN and elliptic curve
* Add double exponential smoothing implementation
* Extended API for managing flow risks
* Add flow risk score
* New flow risks:
+ Desktop or File Sharing Session
+ HTTP suspicious content (useful for tracking trickbot)
+ Malicious JA3
+ Malicious SHA1
+ Risky domain
+ Risky AS
+ TLS Certificate Validity Too Long
+ TLS Suspicious Extension
New Supported Protocols and Services
* New protocols:
+ AmongUs
+ AVAST SecureDNS
+ CPHA (CheckPoint High Availability Protocol)
+ DisneyPlus
+ DTLS
+ Genshin Impact
+ HP Virtual Machine Group Management (hpvirtgrp)
+ Mongodb
+ Pinterest
+ Reddit
+ Snapchat VoIP calls
+ Tumblr
+ Virtual Asssitant (Alexa, Siri)
+ Z39.50
* Add protocols to HTTP as subprotocols
* Add detection of TLS browser type
* Add connectionless DCE/RPC detection
Improvements
* 2.5x speed bump. Example ndpiReader with a long mixed pcap
v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec
v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec
* Improve detection/dissection of:
+ AnyDesk
+ DNS
+ Hulu
+ DCE/RPC (avoid false positives)
+ dnscrypt
+ Facebook (add new networks)
+ Fortigate
+ FTP Control
+ HTTP
- Fix user-agent parsing
- Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined
+ IEC104
+ IEC60870
+ IRC
+ Netbios
+ Netflix
+ Ookla speedtest (detection over IPv6)
+ openspeedtest.com
+ Outlook / MicrosoftMail
+ QUIC
- update to draft-33
- improve handling of SNI
- support for fragmented Client Hello
- support for DNS-over-QUIC
+ RTSP
+ RTSP via HTTP
+ SNMP (reimplemented)
+ Skype
+ SSH
+ Steam (Steam Datagram Relay - SDR)
+ STUN (avoid false positives, improved Skype detection)
+ TeamViewer (add new hosts)
+ TOR (update hosts)
+ TLS
- Certificate Subject matching
- Check for common ALPNs
- Reworked fingerprint calculation
- Fix extraction for TLS signature algorithms
- Fix ClientHello parsing
+ UPnP
+ wireguard
+ Improve DGA detection
+ Improve JA3
+ Improve Mining detection
+ Improve string matching algorithm
+ Improve ndpi_pref_enable_tls_block_dissection
+ Optimize speed and memory size
+ Update ahocorasick library
+ Improve subprotocols detection
Fixes
* Fix partial application matching
* Fix multiple segfault and leaks
* Fix uninitialized memory use
* Fix release of patterns allocated in ndpi_add_string_to_automa
* Fix return value of ndpi_match_string_subprotocol
* Fix setting of flow risks on 32 bit machines
* Fix TLS certificate threshold
* Fix a memory error in TLS JA3 code
* Fix false positives in Z39.50
* Fix off-by-one memory error for TLS-JA3
* Fix bug in ndpi_lru_find_cache
* Fix invalid xbox and playstation port guesses
* Fix CAPWAP tunnel decoding
* Fix parsing of DLT_PPP datalink type
* Fix dissection of QUIC initial packets coalesced with 0-RTT one
* Fix parsing of GTP headers
* Add bitmap boundary checks
Misc
* Update download category name
* Update category labels
* Renamed Skype in Skype_Teams (the protocol is now shared across
these apps)
* Add IEC analysis wireshark plugin
* Flow risk visualization in Wireshark
* ndpiReader
+ add statistics about nDPI performance
+ fix memory leak
+ fix collecting of risks statistics
* Move installed libraries from /usr/local to /usr
* Improve NDPI_API_VERSION generation
* Update ndpi_ptree_match_addr prototype
- Add patches (for compatibility with ntopng 5.0):
* 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch
* 0002-Report-whether-a-protocol-is-encrypted.patch
* 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch
-------------------------------------------------------------------
Fri Apr 23 14:57:05 UTC 2021 - Mathias Homann <Mathias.Homann@opensuse.org>

41
ndpi.spec
View File

@ -2,7 +2,7 @@
# spec file for package ndpi
#
# Copyright (c) 2021 SUSE LLC
# Copyright (c) 2017, Martin Hauke <mardnh@gmx.de>
# Copyright (c) 2017-2021, Martin Hauke <mardnh@gmx.de>
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@ -21,9 +21,9 @@
%bcond_without hyperscan
%endif
%define sover 3
%define sover 4
Name: ndpi
Version: 3.4
Version: 4.0
Release: 0
Summary: Extensible deep packet inspection library
# wireshark/ndpi.lua is GPL-3.0-or-later
@ -31,6 +31,12 @@ License: LGPL-3.0-only
Group: Development/Libraries/C and C++
URL: https://github.com/ntop/nDPI
Source: https://github.com/ntop/nDPI/archive/%{version}.tar.gz#/%{name}-%{version}.tar.gz
# PATCH-FIX-UPSTREAM 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch # ntopng 5.0 needs this from the ndpi 4.0-stable branch
Patch0: 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch
# PATCH-FIX-UPSTREAM 0002-Report-whether-a-protocol-is-encrypted.patch # ntopng 5.0 needs this from the ndpi 4.0-stable branch
Patch1: 0002-Report-whether-a-protocol-is-encrypted.patch
# PATCH-FIX-UPSTREAM 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch
Patch2: 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch
BuildRequires: autoconf
BuildRequires: automake
BuildRequires: gcc-c++
@ -51,6 +57,7 @@ available only on the paid version of OpenDPI.
%package -n libndpi%{sover}
Summary: Extensible deep packet inspection library
Group: System/Libraries
Requires: ndpi-common
%description -n libndpi%{sover}
nDPI is a ntop-maintained superset of the OpenDPI library. It extends
@ -86,8 +93,24 @@ available only on the paid version of OpenDPI.
This package contains the ndpiReader binary.
%package -n ndpi-common
Summary: Common files used by nDPI
Group: Development/Libraries/C and C++
# version 3 rpm did not yet follow rules correctly
Conflicts: libndpi3
%description -n ndpi-common
nDPI is a ntop-maintained superset of the OpenDPI library. It extends
the original library by adding new protocols that are otherwise
available only on the paid version of OpenDPI.
This package contains common files used by nDPI.
%prep
%setup -q -n nDPI-%{version}
%patch0 -p1
%patch1 -p1
%patch2 -p1
%build
sh autogen.sh
@ -101,16 +124,12 @@ make %{?_smp_mflags}
%install
%make_install PREFIX=%{_prefix} prefix=%{_prefix} libdir=%{_libdir}
rm -f %{buildroot}/%{_libdir}/libndpi.a
rm -rf %{buildroot}/%{_sbindir}/ndpi
rm -f %{buildroot}/%{_sbindir}/ndpi
%post -n libndpi%{sover} -p /sbin/ldconfig
%postun -n libndpi%{sover} -p /sbin/ldconfig
%files -n libndpi%{sover}
%license COPYING
%doc CHANGELOG.md README.md README.nDPI README.protocols
%doc doc/nDPI_QuickStartGuide.pdf
%{_datadir}/%{name}
%{_libdir}/libndpi.so.%{sover}*
%files -n libndpi-devel
@ -122,4 +141,10 @@ rm -rf %{buildroot}/%{_sbindir}/ndpi
%{_bindir}/ndpiReader
%doc wireshark
%files -n ndpi-common
%license COPYING
%doc CHANGELOG.md README.md README.nDPI README.protocols
%doc doc/nDPI_QuickStartGuide.pdf
%{_datadir}/%{name}
%changelog