- Update to 1.15.0
- UCP
- Added 2-stage pipeline protocol in the new protocol infrastructure
- Added reset and abort functionality of rendezvous protocols in the
new infrastructure
- Added zero-copy rendezvous data send protocol in the new infrastructure
- Added support for user memory handle in the new protocol infrastructure
- Added option to force ODP registration for certain memory types
- Enabled lock free memory region deregistration
- Updated allow/deny transport list feature to control auxiliary transport selection
- Multiple performance improvements of the new protocol infrastructure
- Multiple improvements in error and debug messages
- Fixed assertion when sending from non-contiguous GPU buffer to managed buffer
- Fixed the race condition on endpoint configurations
- Fixed endpoint reconfiguration issues due to asymmetrical selection
- Fixed endpoint reconfiguration error due to wrong locality detection
- Fixed crash during connection manager cleanup
- Fixed rkey index calculation for rendezvous protocol
- Fixed rcache dump function
- Removed logging from rkey unpack in release mode
- Fixed dobule free of rkey in rendezvous protocol
- Fixed rendezvous pipeline protocol error flow
- Fixed error handling in rendezvous get zcopy protocol
- Replay pending requests of wireup EP CM during connection establishment
to prevent potential ordering issues and wrong configuration
- Pass user-provided memory type to the function that checks whether the buffer
can be sent inline or not
- Avoid memory registration during UCP context initialization
- Fixed CPU/device atomics selection in the new protocol infrastructure
- Multiple fixes in the new protocol infrastructure information output
OBS-URL: https://build.opensuse.org/request/show/1115979
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=63
- Update to v1.14.1
- Fixed ROCm to prevent the locking of host pinned memory
- Added CUDA 12 based UCX builds to the release flow
- Increased the maximal number of endpoint configurations
- Fixed filter for a slow-lanes in selection logic
- Fixed TCP transport bandwidth calculation
- Fixed device detection for ROCM
- Fixed compatibility with CUDA 12
- Fixed rendezvous threshold for multi-path configurations
- Fixed error message in case of static link
- Fixed BlueField-3 detection
- Multiple fixes for Azure CI pipeline
OBS-URL: https://build.opensuse.org/request/show/1100640
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=61
- Update to v1.14.0
- UCP
- Added API for querying transport and device names on endpoint
- Added API for querying datatype object
- Added API for exporting and importing memory keys (no implementation yet)
- Added support for non-persistent active message header
- Added infrastructure to print protocols v2 performance
- Multiple performance improvements for protocols v2
- Added support for non-contiguous datatypes for rendezvous protocols v2
- Added support for reset and abort request in protocols v2
- Added support for user memory handles in RMA API
- Added multi-rail support for RMA API in protocols v2
- Added support for up to 16 different lanes per endpoint
- Added support for dmabuf memory registration in protocols v2
- Added strong fence mode for ucp_worker_fence() API
- UCT
- Added new uct_md_mem_attach() API to support exported memory handles
- Added remote completion mode for endpoint flush (via new flag)
- Added support for dmabuf registration
- Added new uct_ep_connect_to_ep_v2() API
- Added new uct_mem_reg_v2() API
- Added new uct_md_query_v2() API
- Added support for IPv6 loopback address in TCP transport
- RDMA CORE (IB, ROCE, etc.)
- Added ECE (enhanced connection establishment) support for RC and DC transports
- Added support for hardware DCS in DC transport
- Added UD interface and endpoint resource information to VFS
- Added CQ creation via DEVX API
- Removed support for accelerated IB transports over legacy experimental verbs
- UCS
- Added support for auto-correction of user environment variables
- UCM
- Implemented CUDA bistro hooks for aarch64 (to enable memory cache on this platform)
- Added support for CUDA virtual/stream-ordered memory with cudaMallocAsync
- Documentation
- Added FAQ for using pkg-config tool to build applications with UCX
- Tools
- Added runtime library version to the 'ucx_info -v' output
- Added support for memory types in ucx_info
- Many bugfixes. See NEWS.
- Drop patch merged upstream:
- UCS-DEBUG-replace-PTR-with-void.patch
- gcc13-fix.patch
- Refresh openucx-s390x-support.patch
OBS-URL: https://build.opensuse.org/request/show/1075600
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/openucx?expand=0&rev=26
- Update to v1.14.0
- UCP
- Added API for querying transport and device names on endpoint
- Added API for querying datatype object
- Added API for exporting and importing memory keys (no implementation yet)
- Added support for non-persistent active message header
- Added infrastructure to print protocols v2 performance
- Multiple performance improvements for protocols v2
- Added support for non-contiguous datatypes for rendezvous protocols v2
- Added support for reset and abort request in protocols v2
- Added support for user memory handles in RMA API
- Added multi-rail support for RMA API in protocols v2
- Added support for up to 16 different lanes per endpoint
- Added support for dmabuf memory registration in protocols v2
- Added strong fence mode for ucp_worker_fence() API
- UCT
- Added new uct_md_mem_attach() API to support exported memory handles
- Added remote completion mode for endpoint flush (via new flag)
- Added support for dmabuf registration
- Added new uct_ep_connect_to_ep_v2() API
- Added new uct_mem_reg_v2() API
- Added new uct_md_query_v2() API
- Added support for IPv6 loopback address in TCP transport
- RDMA CORE (IB, ROCE, etc.)
- Added ECE (enhanced connection establishment) support for RC and DC transports
- Added support for hardware DCS in DC transport
- Added UD interface and endpoint resource information to VFS
- Added CQ creation via DEVX API
- Removed support for accelerated IB transports over legacy experimental verbs
- UCS
- Added support for auto-correction of user environment variables
- UCM
- Implemented CUDA bistro hooks for aarch64 (to enable memory cache on this platform)
- Added support for CUDA virtual/stream-ordered memory with cudaMallocAsync
- Documentation
- Added FAQ for using pkg-config tool to build applications with UCX
- Tools
- Added runtime library version to the 'ucx_info -v' output
- Added support for memory types in ucx_info
- Many bugfixes. See NEWS.
- Drop patch merged upstream:
- UCS-DEBUG-replace-PTR-with-void.patch
- gcc13-fix.patch
- Refresh openucx-s390x-support.patch
OBS-URL: https://build.opensuse.org/request/show/1075167
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=57
- Update to v1.13.1 (jsc#PED-912)
- Core
- Added new objects to VFS: local and remote address of endpoint,
statistics of ucp_ep_create success/failure, failed/destroyed endpoints
- Added support for UCX static libraries
- Added profiling for rkey management routines
- PCIe relaxed order enabled by default for AMD CPUs
- Fixed not deallocating memory from ucp_mem_unmap if no rcache
- Fixed versioning infrastructure
- Multiple code improvements: refactoring, debug prints and assertions, etc.
- Multiple improvements in build, test and docs infrastructure
- Added new objects to VFS (md, component, log_level, etc.)
- Added configuration variable to specify which loadable modules are allowed
- Added build-time configuration to disable sigaction overriding
- UCP
- Added API to pass pre-registered memory handle to UCP operations
- Added implementation of AM rendezvous protocol
- Added 2-stage pipeline rendezvous protocol for GPU
- Added support for fragment mem_type for v1 pipeline proto, disabled by default
- Added active message support for proto v2
- Added UCP memory registration cache
- Improved adaptive progress - deactivate iface when all p2p lanes are destroyed
- Added support for user memh in proto_v1
- Added support for selecting local address when creating a client endpoint
- Added option to limit GPUDirectRDMA size in rendezvous protocol, UCX_RNDV_MEMTYPE_DIRECT_SIZE
- Deprecated UCX_SOCKADDR_AUX_TLS configuration parameter
- Resolving remote EP ID when creating local EP disabled by default
- Added client_id to ucp_worker_create() and ucp_conn_request_query() APIs
- Added ucp_worker_address_query() API
- Updated ucp_ep_query() API for getting local and remote addresses
- Added address versioning to correctly preserve wire compatibility starting from version 1.11.0
- Added new client/server connection establishment packet header format
- Enabled rendezvous and tag sync protocols when error handling is enabled on the endpoint
- Added iov zcopy support to RMA operations
- Reduced memory usage of unexpected messages by fitting receive buffer size to packet size
- Added support for modifying UCT and UCS configs by ucp_config_modify() API
- Optimized unpacked rkeys memory consumption
- Added request flag to influence latency vs. bandwidth protocol
- Reduced memory management overhead with new protocols
- Improved performance calculations for new protocols
- Added AMO support with GPU memory target using new protocols
- Added put_zcopy, get_zcopy and pipeline based rendezvous in new protocols
- Added support for user-defined alignment in Active Messages
- Added support for offload tag sync in new protocols
- Updated ucp_atomic_post() to use NBX flow
- UCT
- Introduced API uct_md_mkey_pack_v2
- Introduced UCT iface features API
- Introduced max_inflight_eps parameter in perf_attr API
- Introduced UCT_SEND_FLAG_PEER_CHECK flag that forces checking connectivity to a peer
- Introduced UCX_RCACHE_PURGE_ON_FORK to enable/disable cleaning regions when application is forking
- Disabled PEER_FAILURE capability for XPMEM
- Added API - uct_iface_is_reachable_v2()
- Added IPv6 address support in TCP
- Added latency estimation to uct_iface_estimate_perf()
- Adjusted knem and cma overhead cost
- Increased built-in TCP keep-alive interval to 2 seconds
- RDMA CORE (IB, ROCE, etc.)
- Introduced NDR autorecognition
- Introduced CQE zipping support
- Set the default MAX_RD_ATOMIC to maximum value supported by the hardware
- Disabled mlx5 ifaces on verbs MD
- Added detection of IB NDR devices
- Added check for CQ overrun in assert mode
- Added bitmap usage for releasing detached DCIs
- Added configuration for requests ack frequency with DevX
- Added remote QP info to tx error CQE traces
- ROCM
- Increased maximum number of HSA agents
- UCS
- Added topo module infrastructure
- Added memtrack and rcache information to VFS
- Added API for a per-process aggregate-sum statistics report
- Added memory pool set data structure
- Added new ptr_array API for bulk allocation
- Added ucs_string_buffer_append_flags() for string buffer
- Added ucs_ffs32()
- Added ucs_vsnprintf_safe() which always adds '\0'
- Added thread-safe put to ptr_map
- Improved accuracy of the topology distance estimation
- Added prints of leaked callbacks from the callback queue
- Removed a diagnostic message when fuse thread is stopped
- Added configurable limit for the memory consumed by rcache
- Added configuration for VFS(FUSE) thread affinity
- Added memory limit support to memtrack
- Packaging
- Added cmake config files for better integration with external cmake based projects
- Tools
- Added loop-back transport support in ucx_perftest
- Split ucx_perftest into separate modules
- Added process placement option for ucx_info
- Extended parameters correctness check in ucx_perftest
- Backported UCS-DEBUG-replace-PTR-with-void.patch
from upstream to fix compilation
OBS-URL: https://build.opensuse.org/request/show/1006486
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=48
- Update to v1.11.1 (jsc#SLE-19260)
- Core:
- Added support for UCX monitoring using virtual file system (VFS)/FUSE
- Added support for applications with static CUDA runtime linking
- Added support for a configuration file
- Updated clang format configuration
- UCP
- Added rendezvous API for active messages
- Added user-defined name to context, worker, and endpoint objects
- Added flag to silence request leak check
- Added API for endpoint performance evaluation
- Added API - ucp_request_query
- Added API - ucp_lib_query
- Added bandwidth optimizations for new protocols multi-lane
- Added support for multi-rail over lanes with BW ratio >= 1/4
- Added support for tracking outstanding requests and aborting those in case of connection failure
- Refactored keep-alive protocol
- Added device id to wireup protocol
- Added support up to 128 transport layer resources in UCP context
- Added support CUDA memory allocations with ucp_mem_map
- Increased UCP_WORKER_MAX_EP_CONFIG to 64
- Adjusted memory type zcopy threshold when UCX_ZCOPY_THRESH set
- Refactored wireup protocols, rendezvous, get, zcopy protocols
- Added put zcopy multi-rail
- Improved logging for new protocols
- Added system topology information
- Added new protocols for eager offload protocols
- UCT
- Extended connection establishment API
OBS-URL: https://build.opensuse.org/request/show/921702
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=46
- Update to v1.9.0 (jsc#SLE-15163)
- Features:
- Added a new class of communication APIs '*_nbx' that enable API extendability while
- preserving ABI backward compatibility
- Added asynchronous event support to UCT/IB/DEVX
- Added support for latest CUDA library version
- Added NAK-based reliability protocol for UCT/IB/UD to optimize resends
- Added new tests for ROCm
- Added new configuration parameters for protocol selection
- Added performance optimization for Fujitsu A64FX with InfiniBand
- Added performance optimization for clear cache code aarch64
- Added support for relaxed-order PCIe access in IB RDMA transports
- Added new TCP connection manager
- Added support for UCT/IB PKey with partial membership in IB transports
- Added support for RoCE LAG
- Added support for ROCm 3.7 and above
- Added flow control for RDMA read operations
- Improved endpoint flush implementation for UCT/IB
- Improved UD timer to avoid interrupting the main thread when not in use
- Improved latency estimation for network path with CUDA
- Improved error reporting messages
- Improved performance in active message flow (removed malloc call)
- Improved performance in ptr_array flow
- Improved performance in UCT/SM progress engine flow
- Improved I/O demo code
- Improved rendezvous protocol for CUDA
- Updated examples code
- Bugfixes:
- Fixes for most resent versions of GCC, CLANG, ARMCLANG, PGI
- Fixes in UCT/IB for strict order keys
OBS-URL: https://build.opensuse.org/request/show/840386
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=42
- Update to v1.8.0
- Features:
- Improved detection for DEVX support
- Improved TCP scalability
- Added support for ROCM to perftest
- Added support for different source and target memory types to perftest
- Added optimized memcpy for ROCM devices
- Added hardware tag-matching for CUDA buffers
- Added support for CUDA and ROCM managed memories
- Added support for client/server disconnect protocol over rdma connection manager
- Added support for striding receive queue for hardware tag-matching
- Added XPMEM-based rendezvous protocol for shared memory
- Added support shared memory communication between containers on same machine
- Added support for multi-threaded RDMA memory registration for large regions
- Added new test cases to Azure CI
- Added support for multiple listening transports
- Added UCT socket-based connection manager transport
- Updated API for UCT component management
- Added API to retrieve the listening port
- Added UCP active message API
- Removed deprecated API for querying UCT memory domains
- Refactored server/client examples
- Added support for dlopen interception in UCM
- Added support for PCIe atomics
- Updated Java API: added support for most of UCP layer operations
- Updated support for Mellanox DevX API
- Added multiple UCT/TCP transport performance optimizations
- Optimized memcpy() for Intel platforms
- Added protection from non-UCX socket based app connections
- Improved search time for PKEY object
- Enabled gtest over IPv6 interfaces
- Updated Mellanox and Bull device IDs
- Added support for CUDA_VISIBLE_DEVICES
- Increased limits for CUDA IPC registration
- Bugfixes:
- Multiple fixes in JUCX
- Fixes in UCP thread safety
- Fixes for most recent versions GCC, PGI, and ICC
- Fixes for CPU affinity on Azure instances
- Fixes in XPMEM support on PPC64
- Performance fixes in CUDA IPC
- Fixes in RDMA CM flows
- Multiple fixes in TCP transport
- Multiple fixes in documentation
- Fixes in transport lane selection logic
- Fixes in Java jar build
- Fixes in socket connection manager for Nvidia DGX-2 platform
- Multiple fixes in UCP, UCT, UCM libraries
- Multiple fixes for BSD and Mac OS systems
- Fixes for Clang compiler
- Fix CPU optimization configuration options
- Fix JUCX build on GPU nodes
- Fix in Azure release pipeline flow
- Fix in CUDA memory hooks management
- Fix in GPU memory peer direct gtest
- Fix in TCP connection establishment flow
- Fix in GPU IPC check
- Fix in CUDA Jenkins test flow
- Multiple fixes in CUDA IPC flow
- Fix adding missing header files
- Fix to prevent failures in presence of VPN enabled Ethernet interfaces
- Refresh openucx-s390x-support.patch against new version
OBS-URL: https://build.opensuse.org/request/show/811684
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=37
- Update to v1.6.0
- Features:
- Modular architecture for UCT transports
- ROCm transport re-design: support for managed memory, direct copy, ROCm GDR
- Random scheduling policy for DC transport
- Optimized out-of-box settings for multi-rail
- Added support for OmniPath (using Verbs)
- Support for PCI atomics with IB transports
- Reduced UCP address size for homogeneous environments
- Bugfixes:
- Multiple stability and performance improvements in TCP transport
- Multiple stability fixed in Verbs and MLX5 transports
- Multiple stability fixes in UCM memory hooks
- Multiple stability fixes in UGNI transport
- RPM Spec file cleanup
- Fixing compilation issues with most recent clang and gcc compilers
- Fixing the wrong name of aliases
- Fix data race in UCP wireup
- Fix segfault when libuct.so is reloaded - issue #3558
- Include Java sources in distribution
- Handle EADDRNOTAVAIL in rdma_cm connection manager
- Disable ibcm on RHEL7+ by default
- Fix data race in UCP proxy endpoint
- Static checker fixes
- Fallback to ibv_create_cq() if ibv_create_cq_ex() returns ENOSYS
- Fix malloc hooks test
- Fix checking return status in ucp_client_server example
- Fix gdrcopy libdir config value
- Fix printing atomic capabilities in ucx_info
- Fix perftest warmup iterations to be non-zero
OBS-URL: https://build.opensuse.org/request/show/733589
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=33
- Update openucx-s390x-support.patch to fix support of 1.5.0 on s390x
- Add baselibs.conf for ppc
- Update to v1.5.0 (bsc#1121267)
* Features:
* New emulation mode enabling full UCX functionality (Atomic, Put, Get)
* over TCP and RDMA-CORE interconnects which don't implement full RDMA semantics
* Non-blocking API for all one-sided operations. All blocking communication APIs marked
* as deprecated
* New client/server connection establishment API, which allows connected handover between workers
* Support for rdma-core direct-verbs (DEVX) and DC with mlx5 transports
* GPU - Support for stream API and receive side pipelining
* Malloc hooks using binary instrumentation instead of symbol override
* Statistics for UCT tag API
* GPU-to-Infiniband HCA affinity support based on locality/distance (PCIe)
* Bugfixes:
* Fix overflow in RC/DC flush operations
* Update description in SPEC file and README
* Fix RoCE source port for dc_mlx5 flow control
* Improve ucx_info help message
* Fix segfault in UCP, due to int truncation in count_one_bits()
* Multiple other bugfixes (full list on github)
* Tested configurations:
* InfiniBand: MLNX_OFED 4.4-4.5, distribution inbox drivers, rdma-core
* CUDA: gdrcopy 1.2, cuda 9.1.85
* XPMEM: 2.6.2
* KNEM: 1.1.2
OBS-URL: https://build.opensuse.org/request/show/678966
OBS-URL: https://build.opensuse.org/package/show/science:HPC/openucx?expand=0&rev=27
