rpm/python-distributed
SHA256
1
0
Fork 0
forked from pool/python-distributed
Code Pull Requests Activity

Accepting request 1056224 from home:bnavigator:branches:devel:languages:python:numeric

Browse Source 
- Update to 2022.12.1
  ## Enhancements
  * SpillBuffer metrics (GH#7368) crusaderky
  * Prometheus: measure how much spilling blocks the event loop (GH#7370) crusaderky
  * Add transfer_outgoing_bytes_total metric (GH#7388) Gabe Joseph
  * Fail P2PShuffle gracefully upon worker failure (GH#7326) Hendrik Makait
  ## Bug Fixes
  * Select queued tasks in stimuli, not transitions (GH#7402) Gabe Joseph
  * Check ContextVar in default_client (GH#7369) Matthew Rocklin
  * Fix sending event messages to non-subscribers (GH#7014) Laurence Watts
  * Set sizing mode on Tabs to avoid layout collapse (GH#7365) Mateusz Paprocki
  ## Maintenance
  * Restructure P2PShuffle extensions (GH#7390) Hendrik Makait
  * Add Python 3.11 classifier (GH#7408) James Bourbeau
  * Add support for Python 3.11 (GH#7249) Thomas Grainger
  * Add test for using annotations with client.submit and client.map (GH#7399) James Bourbeau
  * Bump actions/checkout from 3.1.0 to 3.2.0 (GH#7393)
  * Remove superfluous ShuffleSchedulerExtension.barriers (GH#7389) Hendrik Makait
  * Remove ignore annotation-unchecked (GH#7379) crusaderky
  * Remove tornado max version from nightly recipe (GH#7376) Charles Blackmon-Luca
  * Remove the experimental feature warning for Semaphore (GH#7373) Florian Jetter
- Release 2022.12.0
  ## Enhancements
  * Expose event loop health metrics in Prometheus (GH#7360) Hendrik Makait
  * Allow log propagation by default (GH#5669) Florian Jetter
  * Clean up of unpack_remotedata() (GH#7322) Mads R. B. Kristensen
  * Upgrade to tornado 6.2 (GH#7286) Thomas Grainger
  * Introduce Server level comm counters (GH#7332) Florian Jetter
  * Prometheus debug log (GH#7302) Florian Jetter
  ## Bug Fixes
  * Catch BaseException s from user tasks (GH#5997) Gabe Joseph
  * Impossible use case of erred deps in transition to waiting (GH#7354) crusaderky
  * Fix a deadlock when queued tasks are resubmitted quickly in succession (GH#7348) Florian Jetter
  ## Maintenance
  * Fetch all artifacts (GH#7355) Enrico Minack
  * Delay fsspec and urllib3 import time (GH#6659) Florian Jetter
  * Bump mypy (GH#7349) crusaderky
  * Bump mypy and remove win specific run (GH#7344) Florian Jetter
  * Finish overhaul of SchedulerState annotations (GH#7333) crusaderky
  * Fix flaky test_pause_while_spilling (GH#7334) Gabe Joseph
  * Update gpuCI RAPIDS_VER to 23.02 (GH#7337)
- Drop distributed-pr7286-tornado-6-2.patch
- Move to PEP517 build

OBS-URL: https://build.opensuse.org/request/show/1056224
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=128
This commit is contained in:
Matej Cepl
2023-01-05 17:47:22 +00:00
committed by Git OBS Bridge
parent 40cce72467
commit 229223a891
5 changed files with 61 additions and 215 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
distributed-2022.11.1-gh.tar.gz
View File

@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d2e0e17d5f7addc5049829b3f84802bde9a89a8bed196b8a689d7c4f980e3a22
size 1785156

3
distributed-2022.12.1-gh.tar.gz Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9051cf8ea2a29e7737eb53a65f6689e3b20d7031f9eb326bb0aa66b48d7c18d2
size 1796755

204
distributed-pr7286-tornado-6-2.patch
View File

@@ -1,204 +0,0 @@
From 76f2c6d58187ee263b7df5db8eb8dc6829458c3e Mon Sep 17 00:00:00 2001
From: Thomas Grainger <tagrain@gmail.com>
Date: Thu, 10 Nov 2022 12:09:38 +0000
Subject: [PATCH 1/3] upgrade to tornado 6.2
Co-Authored-By: Lawrence Mitchell <lmitchell@nvidia.com>
---
continuous_integration/environment-3.10.yaml | 2 +-
continuous_integration/environment-3.8.yaml | 2 +-
continuous_integration/environment-3.9.yaml | 2 +-
requirements.txt | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/continuous_integration/environment-3.10.yaml b/continuous_integration/environment-3.10.yaml
index e5549dd19d..8cc9615cda 100644
--- a/continuous_integration/environment-3.10.yaml
+++ b/continuous_integration/environment-3.10.yaml
@@ -38,7 +38,7 @@ dependencies:
- sortedcollections
- tblib
- toolz
- - tornado<6.2
+ - tornado >=6.2
- zict # overridden by git tip below
- zstandard >=0.9.0
- pip:
diff --git a/continuous_integration/environment-3.8.yaml b/continuous_integration/environment-3.8.yaml
index 353be57689..fa5cbefe96 100644
--- a/continuous_integration/environment-3.8.yaml
+++ b/continuous_integration/environment-3.8.yaml
@@ -41,7 +41,7 @@ dependencies:
- sortedcollections
- tblib
- toolz
- - tornado<6.2
+ - tornado >=6.2
- zict
- zstandard >=0.9.0
- pip:
diff --git a/continuous_integration/environment-3.9.yaml b/continuous_integration/environment-3.9.yaml
index 46362af94b..dfd82c00f0 100644
--- a/continuous_integration/environment-3.9.yaml
+++ b/continuous_integration/environment-3.9.yaml
@@ -44,7 +44,7 @@ dependencies:
- tblib
- toolz
- torchvision # Only tested here
- - tornado<6.2
+ - tornado >=6.2
- zict
- zstandard >=0.9.0
- pip:
diff --git a/requirements.txt b/requirements.txt
index 6d0f0cdbca..8e7f7180b0 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -9,7 +9,7 @@ psutil >= 5.0
sortedcontainers !=2.0.0, !=2.0.1
tblib >= 1.6.0
toolz >= 0.10.0
-tornado >= 6.0.3, <6.2
+tornado >= 6.0.3
urllib3
zict >= 0.1.3
pyyaml
From 9f89068f1dde4afa0bfd2677f7e377cf4878edde Mon Sep 17 00:00:00 2001
From: Lawrence Mitchell <lmitchell@nvidia.com>
Date: Thu, 13 Oct 2022 10:49:18 +0100
Subject: [PATCH 2/3] Filter implicit deprecation warnings from tornado as well
tornado 6.2 deprecates functionality that relies on deprecated asyncio
calls (get_event_loop and friends); a few tests still use a pattern of
implicitly creating a new loop (and test that appropriate deprecation
warnings are uttered), so just catch these new tornado warnings for
now, until the functionality is removed in distributed.
---
distributed/deploy/tests/test_adaptive.py | 1 +
distributed/deploy/tests/test_spec_cluster.py | 1 +
distributed/tests/test_client.py | 4 ++++
distributed/tests/test_client_loop.py | 2 ++
distributed/tests/test_utils.py | 2 ++
5 files changed, 10 insertions(+)
diff --git a/distributed/deploy/tests/test_adaptive.py b/distributed/deploy/tests/test_adaptive.py
index d88615a756..59ec50eaab 100644
--- a/distributed/deploy/tests/test_adaptive.py
+++ b/distributed/deploy/tests/test_adaptive.py
@@ -281,6 +281,7 @@ async def test_no_more_workers_than_tasks():
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_basic_no_loop(cleanup):
loop = None
try:
diff --git a/distributed/deploy/tests/test_spec_cluster.py b/distributed/deploy/tests/test_spec_cluster.py
index 31b95e2802..e0e7301fcf 100644
--- a/distributed/deploy/tests/test_spec_cluster.py
+++ b/distributed/deploy/tests/test_spec_cluster.py
@@ -83,6 +83,7 @@ def test_spec_sync(loop):
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_loop_started_in_constructor(cleanup):
# test that SpecCluster.__init__ starts a loop in another thread
cluster = SpecCluster(worker_spec, scheduler=scheduler, loop=None)
diff --git a/distributed/tests/test_client.py b/distributed/tests/test_client.py
index 590fc13fd7..85e28262e7 100644
--- a/distributed/tests/test_client.py
+++ b/distributed/tests/test_client.py
@@ -2889,6 +2889,7 @@ async def test_startup_close_startup(s, a, b):
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_startup_close_startup_sync(loop):
with cluster() as (s, [a, b]):
with Client(s["address"], loop=loop) as c:
@@ -5613,6 +5614,8 @@ async def test_future_auto_inform(c, s, a, b):
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:clear_current is deprecated:DeprecationWarning")
def test_client_async_before_loop_starts(cleanup):
async def close():
async with client:
@@ -6924,6 +6927,7 @@ async def test_workers_collection_restriction(c, s, a, b):
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
@gen_cluster(client=True, nthreads=[("127.0.0.1", 1)])
async def test_get_client_functions_spawn_clusters(c, s, a):
# see gh4565
diff --git a/distributed/tests/test_client_loop.py b/distributed/tests/test_client_loop.py
index 35bb3a7e86..9d8316cdd0 100644
--- a/distributed/tests/test_client_loop.py
+++ b/distributed/tests/test_client_loop.py
@@ -30,6 +30,7 @@ def _check_cluster_and_client_loop(loop):
# Test if Client stops LoopRunner on close.
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_close_loop_sync_start_new_loop(cleanup):
with _check_loop_runner():
_check_cluster_and_client_loop(loop=None)
@@ -37,6 +38,7 @@ def test_close_loop_sync_start_new_loop(cleanup):
# Test if Client stops LoopRunner on close.
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_close_loop_sync_use_running_loop(cleanup):
with _check_loop_runner():
# Start own loop or use current thread's one.
diff --git a/distributed/tests/test_utils.py b/distributed/tests/test_utils.py
index 853ba578f1..b2dae76fdb 100644
--- a/distributed/tests/test_utils.py
+++ b/distributed/tests/test_utils.py
@@ -405,6 +405,7 @@ def assert_not_running(loop):
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_loop_runner(loop_in_thread):
# Implicit loop
loop = IOLoop()
@@ -488,6 +489,7 @@ def test_loop_runner(loop_in_thread):
@pytest.mark.filterwarnings("ignore:There is no current event loop:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:make_current is deprecated:DeprecationWarning")
def test_two_loop_runners(loop_in_thread):
# Loop runners tied to the same loop should cooperate
From fd74414f63d95f54bfcc25112b9d5b7d7f3168a8 Mon Sep 17 00:00:00 2001
From: Thomas Grainger <tagrain@gmail.com>
Date: Thu, 10 Nov 2022 15:01:59 +0000
Subject: [PATCH 3/3] get loop from fixture
---
distributed/tests/test_client.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/distributed/tests/test_client.py b/distributed/tests/test_client.py
index 85e28262e7..a063dd4eea 100644
--- a/distributed/tests/test_client.py
+++ b/distributed/tests/test_client.py
@@ -6593,9 +6593,9 @@ async def test_as_completed_condition_loop(c, s, a, b):
sys.version_info >= (3, 10),
reason="On Py3.10+ semaphore._loop is not bound until .acquire() blocks",
)
-def test_client_connectionpool_semaphore_loop(s, a, b):
- with Client(s["address"]) as c:
- assert c.rpc.semaphore._loop is c.loop.asyncio_loop
+def test_client_connectionpool_semaphore_loop(s, a, b, loop):
+ with Client(s["address"], loop=loop) as c:
+ assert c.rpc.semaphore._loop is loop.asyncio_loop
@pytest.mark.slow

47
python-distributed.changes
View File

@@ -1,3 +1,50 @@
-------------------------------------------------------------------
Mon Jan 2 21:03:06 UTC 2023 - Ben Greiner <code@bnavigator.de>
- Update to 2022.12.1
## Enhancements
* SpillBuffer metrics (GH#7368) crusaderky
* Prometheus: measure how much spilling blocks the event loop (GH#7370) crusaderky
* Add transfer_outgoing_bytes_total metric (GH#7388) Gabe Joseph
* Fail P2PShuffle gracefully upon worker failure (GH#7326) Hendrik Makait
## Bug Fixes
* Select queued tasks in stimuli, not transitions (GH#7402) Gabe Joseph
* Check ContextVar in default_client (GH#7369) Matthew Rocklin
* Fix sending event messages to non-subscribers (GH#7014) Laurence Watts
* Set sizing mode on Tabs to avoid layout collapse (GH#7365) Mateusz Paprocki
## Maintenance
* Restructure P2PShuffle extensions (GH#7390) Hendrik Makait
* Add Python 3.11 classifier (GH#7408) James Bourbeau
* Add support for Python 3.11 (GH#7249) Thomas Grainger
* Add test for using annotations with client.submit and client.map (GH#7399) James Bourbeau
* Bump actions/checkout from 3.1.0 to 3.2.0 (GH#7393)
* Remove superfluous ShuffleSchedulerExtension.barriers (GH#7389) Hendrik Makait
* Remove ignore annotation-unchecked (GH#7379) crusaderky
* Remove tornado max version from nightly recipe (GH#7376) Charles Blackmon-Luca
* Remove the experimental feature warning for Semaphore (GH#7373) Florian Jetter
- Release 2022.12.0
## Enhancements
* Expose event loop health metrics in Prometheus (GH#7360) Hendrik Makait
* Allow log propagation by default (GH#5669) Florian Jetter
* Clean up of unpack_remotedata() (GH#7322) Mads R. B. Kristensen
* Upgrade to tornado 6.2 (GH#7286) Thomas Grainger
* Introduce Server level comm counters (GH#7332) Florian Jetter
* Prometheus debug log (GH#7302) Florian Jetter
## Bug Fixes
* Catch BaseException s from user tasks (GH#5997) Gabe Joseph
* Impossible use case of erred deps in transition to waiting (GH#7354) crusaderky
* Fix a deadlock when queued tasks are resubmitted quickly in succession (GH#7348) Florian Jetter
## Maintenance
* Fetch all artifacts (GH#7355) Enrico Minack
* Delay fsspec and urllib3 import time (GH#6659) Florian Jetter
* Bump mypy (GH#7349) crusaderky
* Bump mypy and remove win specific run (GH#7344) Florian Jetter
* Finish overhaul of SchedulerState annotations (GH#7333) crusaderky
* Fix flaky test_pause_while_spilling (GH#7334) Gabe Joseph
* Update gpuCI RAPIDS_VER to 23.02 (GH#7337)
- Drop distributed-pr7286-tornado-6-2.patch
- Move to PEP517 build
------------------------------------------------------------------- -------------------------------------------------------------------
Mon Nov 21 19:00:10 UTC 2022 - Ben Greiner <code@bnavigator.de> Mon Nov 21 19:00:10 UTC 2022 - Ben Greiner <code@bnavigator.de>

19
python-distributed.spec
View File

@@ -1,7 +1,7 @@
# #
# spec file # spec file
# #
# Copyright (c) 2022 SUSE LLC # Copyright (c) 2023 SUSE LLC
# #
# All modifications and additions to the file contributed by third parties # All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed # remain the property of their copyright owners, unless otherwise agreed
@@ -44,21 +44,22 @@
Name: python-distributed%{psuffix} Name: python-distributed%{psuffix}
# ===> Note: python-dask MUST be updated in sync with python-distributed! <=== # ===> Note: python-dask MUST be updated in sync with python-distributed! <===
Version: 2022.11.1 Version: 2022.12.1
Release: 0 Release: 0
Summary: Library for distributed computing with Python Summary: Library for distributed computing with Python
License: BSD-3-Clause License: BSD-3-Clause
URL: https://distributed.dask.org URL: https://distributed.dask.org
# SourceRepository: https://github.com/dask/distributed
Source: https://github.com/dask/distributed/archive/refs/tags/%{version}.tar.gz#/distributed-%{version}-gh.tar.gz Source: https://github.com/dask/distributed/archive/refs/tags/%{version}.tar.gz#/distributed-%{version}-gh.tar.gz
Source99: python-distributed-rpmlintrc Source99: python-distributed-rpmlintrc
# PATCH-FIX-UPSTREAM distributed-pr7286-tornado-6-2.patch gh#dask/distributed#7286
Patch2: distributed-pr7286-tornado-6-2.patch
# PATCH-FIX-OPENSUSE distributed-ignore-off.patch -- ignore that we can't probe addresses on obs, code@bnavigator.de # PATCH-FIX-OPENSUSE distributed-ignore-off.patch -- ignore that we can't probe addresses on obs, code@bnavigator.de
Patch3: distributed-ignore-offline.patch Patch3: distributed-ignore-offline.patch
# PATCH-FIX-OPENSUSE distributed-ignore-thread-leaks.patch -- ignore leaking threads on obs, code@bnavigator.de # PATCH-FIX-OPENSUSE distributed-ignore-thread-leaks.patch -- ignore leaking threads on obs, code@bnavigator.de
Patch4: distributed-ignore-thread-leaks.patch Patch4: distributed-ignore-thread-leaks.patch
BuildRequires: %{python_module base >= 3.8} BuildRequires: %{python_module base >= 3.8}
BuildRequires: %{python_module pip}
BuildRequires: %{python_module setuptools} BuildRequires: %{python_module setuptools}
BuildRequires: %{python_module wheel}
BuildRequires: fdupes BuildRequires: fdupes
BuildRequires: python-rpm-macros BuildRequires: python-rpm-macros
Requires: python-PyYAML Requires: python-PyYAML
@@ -73,7 +74,7 @@ Requires: python-psutil >= 5.0
Requires: python-sortedcontainers Requires: python-sortedcontainers
Requires: python-tblib Requires: python-tblib
Requires: python-toolz >= 0.10.0 Requires: python-toolz >= 0.10.0
Requires: python-tornado >= 6.2 Requires: python-tornado >= 6.0.3
Requires: python-urllib3 Requires: python-urllib3
Requires: python-zict >= 0.1.3 Requires: python-zict >= 0.1.3
Requires(post): update-alternatives Requires(post): update-alternatives
@@ -109,16 +110,17 @@ clusters.
sed -e '/--durations=20/d' \ sed -e '/--durations=20/d' \
-e '/--color=yes/d' \ -e '/--color=yes/d' \
-e 's/timeout_method = thread/timeout_method = signal/' \ -e 's/timeout_method = thread/timeout_method = signal/' \
-e ' /^ error$/ a \ ignore:`np.bool8` is a deprecated alias for `np.bool_`' \
-i setup.cfg -i setup.cfg
%build %build
%if ! %{with test} %if ! %{with test}
%python_build %pyproject_wheel
%endif %endif
%install %install
%if ! %{with test} %if ! %{with test}
%python_install %pyproject_install
%python_clone -a %{buildroot}%{_bindir}/dask-ssh %python_clone -a %{buildroot}%{_bindir}/dask-ssh
%python_clone -a %{buildroot}%{_bindir}/dask-scheduler %python_clone -a %{buildroot}%{_bindir}/dask-scheduler
%python_clone -a %{buildroot}%{_bindir}/dask-worker %python_clone -a %{buildroot}%{_bindir}/dask-worker
@@ -152,6 +154,7 @@ donttest+=" or (test_resources and test_prefer_constrained)"
donttest+=" or (test_steal and test_steal_twice)" donttest+=" or (test_steal and test_steal_twice)"
donttest+=" or (test_variable and test_variable_in_task)" donttest+=" or (test_variable and test_variable_in_task)"
donttest+=" or (test_worker and test_worker_reconnects_mid_compute)" donttest+=" or (test_worker and test_worker_reconnects_mid_compute)"
donttest+=" or (test_worker_memory and test_digests)"
# server-side fail due to the non-network warning in a subprocess where the patched filter does not apply # server-side fail due to the non-network warning in a subprocess where the patched filter does not apply
donttest+=" or (test_client and test_quiet_close_process)" donttest+=" or (test_client and test_quiet_close_process)"
if [[ $(getconf LONG_BIT) -eq 32 ]]; then if [[ $(getconf LONG_BIT) -eq 32 ]]; then
@@ -201,7 +204,7 @@ notparallel+=" or test_client_async_before_loop_starts"
%python_alternative %{_bindir}/dask-scheduler %python_alternative %{_bindir}/dask-scheduler
%python_alternative %{_bindir}/dask-worker %python_alternative %{_bindir}/dask-worker
%{python_sitelib}/distributed %{python_sitelib}/distributed
%{python_sitelib}/distributed-%{version}*-info %{python_sitelib}/distributed-%{version}.dist-info
%endif %endif