- Update to version 2021.9.1 * Revert AMM ReduceReplicas and parallel AMMs updates * Run multiple AMMs in parallel * AMM ReduceReplicas to iterate only on replicated tasks * Add type annotations to various functions within distributed.worker * Mark test_ucx_config_w_env_var flaky on UCX < 1.11 * Warn if CUDA context is created on incorrect device in UCX * Remove redundant timeouts from test_client * Allow Client to subscribe to events // Remote printing and warning * Test pickle protocols 4 & 5 * Fix-up test_pickle_empty * Increase timeout for test_worker_reconnects_mid_compute_multiple_states_on_scheduler * Add synced dict between cluster and scheduler to store cluster info * Update test_sub_submit_priority * Revert "Add test setup fixture * Fix flaky test_worker_reconnects_mid_compute * Use gen_test in test_adaptive * Increase worker.suspicious_counter threshold * Active Memory Manager framework + discard excess replicas * Add test setup fixture - Release version 2021.9.0 * Fix add_plugin warnings * Add list around iterator in handle_missing_dep * Jupyter-client 7 compatiblity * Replace atop with blockwise * Add pytest color to CI * Fix test_map and others * Use name argument with Scheduler.remove_plugin calls * Downgrade to jupyter_client 6 * Migrate Security HTML repr to Jinja2 * Migrate ProcessInterface HTML repr to Jinja2 * Add support for diskless machines to system monitor * Avoid during-iteration scheduler plugin changes * Remove GroupProgress scheduler plugin * Properly check for ipv6 availability * Improved IPv6 dask-worker support * Overwrite worker plugins * Refactor scheduler plugins; store in a dictionary * Fix "then" -> "than" typo in docs * Fix typo (remove extra verb "creates") in docs * Fix "fractiom" -> "fraction" typo in docstring * Fix "schedulers" -> "scheduler" typo in docs * Use non-histogram plots up to 100 workers - Release 2021.08.1 * Rename plots to fit in the labextension * Log messages for CommClosedError now includes information about remote address * Add target='_blank' for redirects of dashboard link * Update computation code retrieval logic * Minor polish on cfexecutor * Use development version of dask in gpuCI build * Use upstream dask.widgets * Fix flaky test_worker_reconnects_mid_compute * Update WorkerPlugin docstring about usage of TaskState objects * Worker Network Timeseries * Add HTML Repr for ProcessInterface class and all its subclasses * Fix an issue where a reconnecting worker could cause an invalid transition * Minor fixes for cfexecutor * Add HTML Repr for Security class * Fix performance report sizing issue * Drop RMM compatibility code from RAPIDS < 0.11 - Release 2021.08.0 * Include addresses in closed comm repr * Test nanny.environ precedence * Migrating HTML reprs to jinja2 * Fix test_process_executor_kills_process flakyness * Remove urllib3 as a dependency downloading preloads * Download preload urls in the Preload constructor * Avoid recursion error in profile.merge * Add text exceptions to the Scheduler * Use kwarg for Theme filename * Add a .git-ignore-revs file * Replace not not with bool() * Resolve deadlock cause by transition error after fetching dependency * Set z-index of data-table lower * Add no-worker - memory transition to scheduler * Deprecate worker plugin overwrite policy * Fix flaky tests in CI * Instructions for jemalloc with brew on macOS * Bump RAPIDS_VER to 21.10 * Tweak verbiage around async functions * Use Python 3 super() calls * Support asynchronous tasks * Rename total comm bytes and provide doc string * Add GPU executor if GPU is present * Fix RMM and UCX tests * Remove excessive timeout of test_steal_during_task_deserialization * Add gpuCI build script * Demote Worker.ensure_computing to function OBS-URL: https://build.opensuse.org/request/show/921004 OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=100
165 lines
5.9 KiB
RPMSpec
165 lines
5.9 KiB
RPMSpec
#
|
|
# spec file
|
|
#
|
|
# Copyright (c) 2021 SUSE LLC
|
|
#
|
|
# All modifications and additions to the file contributed by third parties
|
|
# remain the property of their copyright owners, unless otherwise agreed
|
|
# upon. The license for this file, and modifications and additions to the
|
|
# file, is the same license as for the pristine package itself (unless the
|
|
# license for the pristine package is not an Open Source License, in which
|
|
# case the license is the MIT License). An "Open Source License" is a
|
|
# license that conforms to the Open Source Definition (Version 1.9)
|
|
# published by the Open Source Initiative.
|
|
|
|
# Please submit bugfixes or comments via https://bugs.opensuse.org/
|
|
#
|
|
|
|
|
|
%global flavor @BUILD_FLAVOR@%{nil}
|
|
%if "%{flavor}" == "test"
|
|
%define psuffix -test
|
|
%bcond_without test
|
|
%else
|
|
%define psuffix %{nil}
|
|
%bcond_with test
|
|
%endif
|
|
%ifarch %{ix86} %{arm}
|
|
# cython optimizations not supported on 32-bit: https://github.com/dask/dask/issues/7489
|
|
%bcond_with cythonize
|
|
%else
|
|
%bcond_without cythonize
|
|
%endif
|
|
%if %{with cythonize}
|
|
%define cythonize --with-cython
|
|
%endif
|
|
%{?!python_module:%define python_module() python3-%{**}}
|
|
%define skip_python2 1
|
|
%define skip_python36 1
|
|
%define ghversiontag 2021.09.1
|
|
Name: python-distributed%{psuffix}
|
|
# Note: please always update together with python-dask
|
|
Version: 2021.9.1
|
|
Release: 0
|
|
Summary: Library for distributed computing with Python
|
|
License: BSD-3-Clause
|
|
URL: https://distributed.readthedocs.io/en/latest/
|
|
Source: https://github.com/dask/distributed/archive/refs/tags//%{ghversiontag}.tar.gz#/distributed-%{ghversiontag}-gh.tar.gz
|
|
Source99: python-distributed-rpmlintrc
|
|
BuildRequires: %{python_module base >= 3.7}
|
|
BuildRequires: %{python_module setuptools}
|
|
BuildRequires: fdupes
|
|
BuildRequires: python-rpm-macros
|
|
Requires: python-PyYAML
|
|
Requires: python-certifi
|
|
Requires: python-click >= 6.6
|
|
Requires: python-cloudpickle >= 1.5.0
|
|
Requires: python-dask = %{version}
|
|
Requires: python-msgpack
|
|
Requires: python-psutil >= 5.0
|
|
Requires: python-sortedcontainers
|
|
Requires: python-tblib
|
|
Requires: python-toolz >= 0.8.2
|
|
Requires: python-tornado >= 6.0.3
|
|
Requires: python-zict >= 0.1.3
|
|
Requires(post): update-alternatives
|
|
Requires(postun):update-alternatives
|
|
%if %{with cythonize}
|
|
BuildRequires: %{python_module Cython}
|
|
# the cythonized scheduler needs Cython also as runtime dep for some checks
|
|
Requires: python-Cython
|
|
%endif
|
|
%if %{with test}
|
|
BuildRequires: %{python_module PyYAML}
|
|
BuildRequires: %{python_module bokeh}
|
|
BuildRequires: %{python_module certifi}
|
|
BuildRequires: %{python_module click >= 6.6}
|
|
BuildRequires: %{python_module cloudpickle >= 1.5.0}
|
|
BuildRequires: %{python_module dask-all = %{version}}
|
|
BuildRequires: %{python_module distributed = %{version}}
|
|
BuildRequires: %{python_module ipykernel}
|
|
BuildRequires: %{python_module ipython}
|
|
BuildRequires: %{python_module jupyter_client}
|
|
BuildRequires: %{python_module msgpack}
|
|
BuildRequires: %{python_module psutil}
|
|
BuildRequires: %{python_module pytest-asyncio}
|
|
BuildRequires: %{python_module pytest-rerunfailures}
|
|
BuildRequires: %{python_module pytest-timeout}
|
|
BuildRequires: %{python_module pytest}
|
|
BuildRequires: %{python_module requests}
|
|
BuildRequires: %{python_module sortedcontainers}
|
|
BuildRequires: %{python_module sparse}
|
|
BuildRequires: %{python_module tblib}
|
|
BuildRequires: %{python_module toolz >= 0.8.2}
|
|
BuildRequires: %{python_module tornado >= 6.0.3}
|
|
BuildRequires: %{python_module zict >= 0.1.3}
|
|
%endif
|
|
%python_subpackages
|
|
|
|
%description
|
|
Dask.distributed is a library for distributed computing in Python. It
|
|
extends both the concurrent.futures and dask APIs to moderate sized
|
|
clusters.
|
|
|
|
%prep
|
|
%autosetup -p1 -n distributed-%{ghversiontag}
|
|
sed -i '/addopts/ {s/--durations=20//; s/--color=yes//}' setup.cfg
|
|
|
|
%build
|
|
%if ! %{with test}
|
|
%python_build %{?cythonize}
|
|
%endif
|
|
|
|
%install
|
|
%if ! %{with test}
|
|
%python_install %{?cythonize}
|
|
%python_clone -a %{buildroot}%{_bindir}/dask-ssh
|
|
%python_clone -a %{buildroot}%{_bindir}/dask-scheduler
|
|
%python_clone -a %{buildroot}%{_bindir}/dask-worker
|
|
%python_expand %fdupes %{buildroot}%{$python_sitearch}
|
|
%endif
|
|
|
|
%if %{with test}
|
|
%check
|
|
# many tests from multiple files are broken by new pytest-asyncio
|
|
# (see https://github.com/dask/distributed/pull/4212 and https://github.com/pytest-dev/pytest-asyncio/issues/168)
|
|
%if %{pkg_vcmp python3-pytest-asyncio >= 0.14}
|
|
donttest+=" or (test_client and test_get_client_functions_spawn_clusters)"
|
|
donttest+=" or (test_preload and test_web_preload)"
|
|
donttest+=" or (test_semaphore and test_access_semaphore_by_name)"
|
|
donttest+=" or (test_semaphore and test_close_async)"
|
|
donttest+=" or (test_semaphore and test_oversubscribing_leases)"
|
|
donttest+=" or (test_semaphore and test_release_once_too_many_resilience)"
|
|
donttest+=" or (test_semaphore and test_release_simple)"
|
|
donttest+=" or (test_semaphore and test_threadpoolworkers_pick_correct_ioloop)"
|
|
donttest+=" or (test_worker and test_worker_client_closes_if_created_on_worker_last_worker_alive)"
|
|
donttest+=" or (test_worker and test_worker_client_closes_if_created_on_worker_one_worker)"
|
|
%endif
|
|
# randomly fail even with old asyncio -- too slow for obs (?)
|
|
donttest+=" or (test_asyncprocess and test_exit_callback)"
|
|
donttest+=" or (test_worker and test_fail_write_to_disk)"
|
|
# rebalance fails on the server, but not when building locally
|
|
donttest+=" or (test_scheduler and test_rebalance)"
|
|
donttest+=" or (test_tls_functional and test_rebalance)"
|
|
%pytest_arch distributed/tests -r sfER -m "not avoid_ci" -k "not (${donttest:4})" --reruns 3 --reruns-delay 3
|
|
%endif
|
|
|
|
%if ! %{with test}
|
|
%post
|
|
%python_install_alternative dask-ssh dask-scheduler dask-worker
|
|
|
|
%postun
|
|
%python_uninstall_alternative dask-ssh
|
|
|
|
%files %{python_files}
|
|
%doc README.rst
|
|
%license LICENSE.txt
|
|
%python_alternative %{_bindir}/dask-ssh
|
|
%python_alternative %{_bindir}/dask-scheduler
|
|
%python_alternative %{_bindir}/dask-worker
|
|
%{python_sitearch}/distributed
|
|
%{python_sitearch}/distributed-%{version}*-info
|
|
%endif
|
|
|
|
%changelog
|