- Update to version 2022.1.1
* P2P shuffle skeleton
* Fix <Task pending name='...' coro=<Client._handle_report()>
* Add distributed.client.security-loader config
* Avoid Client._handle_report cancelling itself on Client._close
* Paused workers shouldn't steal tasks
* Add option for timestamps from output of Node.get_logs
* Don't use time.time() or IOLoop.time()
* Raise plugin exceptions on Worker.start()
* Fixing docstrings
* Fix typo in Client.run docstring
* Update client.py docstrings
* Skip shuffle tests if pandas / dask.dataframe not installed
* Improve test coverage
* Test report improvements
* P2P shuffle: ignore row order in tests
* Fix flaky test_no_reconnect[--no-nanny]
* Pin coverage to 6.2
* Check for new name of timeouts artifact and be more fault
tolerant
* Revisit rebalance unit tests
* Update comment in rearrange_by_column_p2p
* Update gpuCI RAPIDS_VER to 22.04
* Fix groupby test after meta requirements got stricter in Dask
PR#8563
* Fix flaky test_close_gracefully and test_lifetime
* Fix flaky test_workspace_concurrency
* Fix flaky test_shuffle_extension.py::test_get_partition
* Fix flaky test_dump_cluster_unresponsive_remote_worker
* Dump cluster state on all test failures
* Update license format
* Fix flaky test_drop_with_paused_workers_with_running_tasks_3_4
* Do not raise an exception if the GitHub token cannot be found
(Resubmit for the noclean flag)
OBS-URL: https://build.opensuse.org/request/show/950288
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=104
- Update to version 2022.1.0
* full changelog at
https://distributed.readthedocs.io/en/latest/changelog.html
* New Features:
- Task group stacked area chart (GH#5320) Ian Rose
- Support configuring TLS min/max version (GH#5594) Jim
Crist-Harif
- Use asyncio for TCP/TLS comms (GH#5450) Jim Crist-Harif
- Enhancement release notes for 2021.12.0
* Support pytest fixures and parametrize with gen_test (GH#5532)
Fábio Rosado
* Allow idempotent scheduler plugins to be registered via the RPC
(GH#5545) Jacob Tomlinson
* AMM logging (GH#5530) crusaderky
* Raise error if asyncssh isn’t installed when using SSHCluster
(GH#5535) Fábio Rosado
* Allow None in UCX configuration schema (GH#5534) Fábio Rosado
* Add distributed.comm.ucx.create-cuda-context config (GH#5526)
Peter Andreas Entschev
- Security release note for 2021.10.0:
* This release fixed a potential security vulnerability relating
to single-machine Dask clusters. Clusters started with
dask.distributed.LocalCluster or dask.distributed.Client()
(which defaults to using LocalCluster) would mistakenly
configure their respective Dask workers to listen on external
interfaces (typically with a randomly selected high port)
rather than only on localhost. A Dask cluster created using
this method AND running on a machine that has these ports
exposed could be used by a sophisticated attacker to enable
remote code execution. Users running on machines with standard
firewalls in place should not be affected. This vulnerability
is documented in CVE-2021-42343, and is fixed in this release
(GH#5427). Thanks to Jean-Pierre van Riel for discovering and
reporting the issue.
- Skip python310: Not supported yet -- gh#dask/distributed#5350
OBS-URL: https://build.opensuse.org/request/show/948001
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=101
- Update to version 2021.9.1
* Revert AMM ReduceReplicas and parallel AMMs updates
* Run multiple AMMs in parallel
* AMM ReduceReplicas to iterate only on replicated tasks
* Add type annotations to various functions within
distributed.worker
* Mark test_ucx_config_w_env_var flaky on UCX < 1.11
* Warn if CUDA context is created on incorrect device in UCX
* Remove redundant timeouts from test_client
* Allow Client to subscribe to events // Remote printing and
warning
* Test pickle protocols 4 & 5
* Fix-up test_pickle_empty
* Increase timeout for
test_worker_reconnects_mid_compute_multiple_states_on_scheduler
* Add synced dict between cluster and scheduler to store cluster
info
* Update test_sub_submit_priority
* Revert "Add test setup fixture
* Fix flaky test_worker_reconnects_mid_compute
* Use gen_test in test_adaptive
* Increase worker.suspicious_counter threshold
* Active Memory Manager framework + discard excess replicas
* Add test setup fixture
- Release version 2021.9.0
* Fix add_plugin warnings
* Add list around iterator in handle_missing_dep
* Jupyter-client 7 compatiblity
* Replace atop with blockwise
* Add pytest color to CI
* Fix test_map and others
* Use name argument with Scheduler.remove_plugin calls
* Downgrade to jupyter_client 6
* Migrate Security HTML repr to Jinja2
* Migrate ProcessInterface HTML repr to Jinja2
* Add support for diskless machines to system monitor
* Avoid during-iteration scheduler plugin changes
* Remove GroupProgress scheduler plugin
* Properly check for ipv6 availability
* Improved IPv6 dask-worker support
* Overwrite worker plugins
* Refactor scheduler plugins; store in a dictionary
* Fix "then" -> "than" typo in docs
* Fix typo (remove extra verb "creates") in docs
* Fix "fractiom" -> "fraction" typo in docstring
* Fix "schedulers" -> "scheduler" typo in docs
* Use non-histogram plots up to 100 workers
- Release 2021.08.1
* Rename plots to fit in the labextension
* Log messages for CommClosedError now includes information
about remote address
* Add target='_blank' for redirects of dashboard link
* Update computation code retrieval logic
* Minor polish on cfexecutor
* Use development version of dask in gpuCI build
* Use upstream dask.widgets
* Fix flaky test_worker_reconnects_mid_compute
* Update WorkerPlugin docstring about usage of TaskState objects
* Worker Network Timeseries
* Add HTML Repr for ProcessInterface class and all its subclasses
* Fix an issue where a reconnecting worker could cause an
invalid transition
* Minor fixes for cfexecutor
* Add HTML Repr for Security class
* Fix performance report sizing issue
* Drop RMM compatibility code from RAPIDS < 0.11
- Release 2021.08.0
* Include addresses in closed comm repr
* Test nanny.environ precedence
* Migrating HTML reprs to jinja2
* Fix test_process_executor_kills_process flakyness
* Remove urllib3 as a dependency downloading preloads
* Download preload urls in the Preload constructor
* Avoid recursion error in profile.merge
* Add text exceptions to the Scheduler
* Use kwarg for Theme filename
* Add a .git-ignore-revs file
* Replace not not with bool()
* Resolve deadlock cause by transition error after fetching
dependency
* Set z-index of data-table lower
* Add no-worker - memory transition to scheduler
* Deprecate worker plugin overwrite policy
* Fix flaky tests in CI
* Instructions for jemalloc with brew on macOS
* Bump RAPIDS_VER to 21.10
* Tweak verbiage around async functions
* Use Python 3 super() calls
* Support asynchronous tasks
* Rename total comm bytes and provide doc string
* Add GPU executor if GPU is present
* Fix RMM and UCX tests
* Remove excessive timeout of
test_steal_during_task_deserialization
* Add gpuCI build script
* Demote Worker.ensure_computing to function
OBS-URL: https://build.opensuse.org/request/show/921004
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=100
- Update to version 2021.7.2
* Fix a deadlock connected to task stealing and task
deserialization
* Include maximum shard size in second to_frames method
* Minor dashboard style updates
* Cap maximum shard size at the size of an integer
* Document automatic MALLOC_TRIM_THRESHOLD_ environment variable
* Mark ucx-py tests for GPU
* Update individual profile plot sizing
* Handle NVMLError_Unknown in NVML diagnostics
* Unit tests to use a random port for the dashboard
* Ensure worker reconnect registers existing tasks properly
* Halve CI runtime!
* Add NannyPlugins
* Add WorkerNetworkBandwidth chart to dashboard
* Set nanny environment variables in config
* Read smaller frames to workaround OpenSSL bug
* Move UCX/RMM config variables to Distributed namespace
* Allow ws(s) messages greater than 10Mb
* Short-circuit root-ish check for many deps
-Release 2021.07.1
* Remove experimental feature warning from actors docs
* Keep dependents in worker dependency if TS is still known
* Add Scheduler.set_restrictions
* Make Actor futures awaitable and work with as_completed
* Simplify test_secede_balances
* Computation class
* Some light dashboard cleanup
* Don't package tests
* Add pytest marker for GPU tests
* Actor: don't hold key references on workers
* Collapse nav to hamburger sooner
* Verify that actors survive pickling
* Reenable UCX-Py tests that used to segfault
* Better support ProcessPoolExecutors
* Simplify test_worker_heartbeat_after_cancel
* Avoid property validation in Bokeh
* Reduce default websocket frame size and make configurable
* Disable pytest-timeout SIGALARM on MacOS
* rebalance() resilience to computations
* Improve CI stability
* Ensure heartbeats after cancelation do not raise KeyError s
* Add more useful exception message on TLS cert mismatch
* Add bokeh mode parameter to performance reports
- Use the GitHub tarball because the PyPI sdist does to provide the
tests anymore
* Remove extra conftest.py source
- Drop distributed-pr5022-improve_ci.patch merged upstream
OBS-URL: https://build.opensuse.org/request/show/910741
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=99
- Update to version 2021.7.0
* Fix Nbytes jitter - less expensive
* Use native GH actions cancel feature
* Don't require workers to report to scheduler if scheduler
shuts down
* Add pandas to the list of checked packages for client.
get_versions()
* Move worker preload before scheduler address is set
* Fix flaky test_oversubscribing_leases
* Update scheduling policy docs for #4967
* Add echo handler to Server class
* Also include pngs when bundling package
* Remove duplicated dashboard panes
* Fix worker memory dashboard flickering
* Tabs on bottom left corner on dashboard
* Rename nbytes widgets
* Co-assign root-ish tasks
* OSError tweaks
* Update imports to cudf.testing._utils
* Ensure shuffle split default durations uses proper prefix
* Follow up pyupgrade formatting
* Rename plot dropdown
* Pyupgrade
* Misc Sphinx tweaks
* No longer hold dependencies of erred tasks in memory
* Add maximum shard size to config
* Ensure shuffle split operations are blacklisted from work
stealing
* Add dropdown menu to access individual plots
* Edited the path to scheduler.py
* Task Group Graph Visualization
* Remove more internal references to deprecated utilities
* Restructure nbytes hover
* Except more errors in pynvml.nvmlInit()
* Add occupancy as individual plot
* Deprecate utilities which have moved to dask
* Ensure connectionpool does not leave comms if closed mid
connect
* Add support for registering scheduler plugins from Client
* Stealing dashboard fixes
* Allow requirements verification to be ignored when loading
backends from entrypoints
* Add Log and Logs to API docs
* Support fixtures and pytest.mark.parametrize with gen_cluster
- Release 2021.06.2
* Revert refactor to utils.Log[s] and Cluster.get_logs
* Use deprecation utility from Dask
* Add transition counter to Scheduler
* Remove nbytes_in_memory
- Release 2021.06.1
* Fix deadlock in handle_missing_dep if additional replicas are
available
* Add configuration to enable/disable NVML diagnostics
* Add scheduler log tab to performance reports
* Add HTML repr to scheduler_info and incorporate into client
and cluster reprs
* Fix error state typo
* Allow actor exceptions to propagate
* Remove importing apply from dask.compatibility
* Use more informative default name for WorkerPlugin s
* Removed unused utility functions
* Locally rerun successfully completed futures
* Forget erred tasks and fix deadlocks on worker
* Handle HTTPClientError in websocket connector
* Update dask_cuda usage in SSHCluster docstring
* Remove tests for process_time and thread_time
* Flake8 config cleanup
* Don't strip scheduler protocol when determining host
* Add more documentation on memory management
* Add range_query tests to NVML test suite
* No longer cancel result future in async process when using
timeouts
- Release 2021.06.0
* Multiple worker executors
* Ensure PyNVML works correctly when installed with no GPUs
* Show more in test summary
* Move SystemMonitor s GPU initialization back to constructor
* Mark test_server_comms_mark_active_handlers with pytest.mark.asyncio
* Who has has what html reprs v2
* O(1) rebalance
* Ensure repr and eq for cluster always works
- Release 2021.05.1
* Drop usage of WhoHas & WhatHas from Client
* Ensure adaptive scaling is properly awaited and closed
* Fix WhoHas/ HasWhat async usage
* Add HTML reprs for Client.who_has and Client.has_what
* Prevent accidentally starting multiple Worker s in the same
process
* Add system tab to performance reports
* Let servers close faster if there are no active handlers
* Fix UCX scrub config logging
* Ensure worker clients are closed
* Fix warning for attribute error when deleting a client
* Ensure exceptions are raised if workers are incorrectly started
* Update handling of UCX exceptions on endpoint closing
* Ensure busy workloads properly look up who_has
* Check distributed.scheduler.pickle in Scheduler.run_function
* Add performance_report to API docs
* Use dict _workers_dv in unordered use cases
* Bump pre-commit hook versions
* Do not mindlessly spawn workers when no memory limit is set
* test_memory to use gen_cluster
* Increase timeout of gen_test to 30s
- Work on the very flaky testsuite:
* Add missing conftest.py not packaged on PyPI
* Add distributed-pr5022-improve_ci.patch in the hope for better
stability -- gh#dask/distributed#5022
* Do not use pytest-xdist
- Add Cython as runtime dep because the scheduler checks the
presence
OBS-URL: https://build.opensuse.org/request/show/907354
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=98
- update to version 2021.5.0
* Merge global annotations on the client (GH#4691) Mads R. B.
Kristensen
* Add support for click 8 (GH#4810) James Bourbeau
* Add HTML reprs to some scheduler classes (GH#4795) James
Bourbeau
* Use JupyterLab theme variables (GH#4796) Ian Rose
* Allow the dashboard to run on multiple ports (GH#4786) Jacob
Tomlinson
* Remove release_dep from WorkerPlugin API (GH#4791) James
Bourbeau
* Support for UCX 1.10+ (GH#4787) Peter Andreas Entschev
* Reduce complexity of test_gather_allow_worker_reconnect
(GH#4739) Florian Jetter
* Fix doctests in utils.py (GH#4785) Jacob Tomlinson
* Ensure deps are actually logged in worker (GH#4753) Florian
Jetter
* Add stacklevel keyword into performance_report() to allow for
selecting calling code to be displayed (GH#4777) Nathan
Danielsen
* Unregister worker plugin (GH#4748) Naty Clementi
* Fixes some pickling issues in the Cythonized Scheduler
(GH#4768) jakirkham
* Improve graceful shutdown if nanny is involved (GH#4725)
Florian Jetter
* Update cythonization in CI (GH#4764) James Bourbeau
* Use contextlib.nullcontext (GH#4763) James Bourbeau
* Cython fixes for MemoryState (GH#4761) jakirkham
* Fix errors in check_thread_leak (GH#4747) James Bourbeau
* Handle missing key case in report_on_key (GH#4755) jakirkham
* Drop temporary set variables s (GH#4758) jakirkham
- skip even more flaky tests
- stress that upstream wants to use dask with a matching version
number
OBS-URL: https://build.opensuse.org/request/show/894365
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=96
- Update to 2021.3.1
* Add standalone dashboard page for GPU usage (GH#4556) Jacob
Tomlinson
* Handle stream is None case in TCP comm finalizer (GH#4631)
James Bourbeau
* Include LIST_PICKLE in NumPy array serialization (GH#4632)
James Bourbeau
* Rename annotation plugin in test_highlevelgraph.py (GH#4618)
James Bourbeau
* UCX use nbytes instead of len (GH#4621) Mads R. B. Kristensen
* Skip NumPy and pandas tests if not importable (GH#4563) Ben
Greiner
* Remove utils.shutting_down in favor of sys.is_finalizing
(GH#4624) James Bourbeau
* Handle async clients when closing (GH#4623) Matthew Rocklin
* Drop log from remove_key_from_stealable (GH#4609) jakirkham
* Introduce events log length config option (GH#4615) Fabian
Gebhart
* Upstream config serialization and inheritance (GH#4372) Jacob
Tomlinson
* Add check to scheduler creation in SpecCluster (GH#4605) Jacob
Tomlinson
* Make length of events deque configurable (GH#4604) Fabian
Gebhart
* Add explicit fetch state to worker TaskState (GH#4470) Gil
Forsyth
* Update develop.rst (GH#4603) Florian Jetter
* pickle_loads(): Handle empty memoryview (GH#4595) Mads R. B.
Kristensen
* Switch documentation builds for PRs to readthedocs (GH#4599)
James Bourbeau
* Track frame sizes along with frames (GH#4593) jakirkham
* Add support for a list of keys when using batch_size in client.
map (GH#4592) Sultan Orazbayev
* If SpecCluster fails to start attempt to gracefully close out
again (GH#4590) Jacob Tomlinson
* Multi-lock extension (GH#4503) Mads R. B. Kristensen
* Update PipInstall plugin command (GH#4584) James Bourbeau
* IPython magics: remove deprecated ioloop workarounds (GH#4530)
Min RK
* Add GitHub actions workflow to cancel duplicate builds
(GH#4581) James Bourbeau
* Remove outdated macOS build badge from README (GH#4576) James
Bourbeau
* Dask master -> main (GH#4569) Julia Signell
* Drop support for Python 3.6 (GH#4390) James Bourbeau
* Add docstring for dashboard_link property (GH#4572) Doug Davis
* Change default branch from master to main (GH#4495) Julia
Signell
* Msgpack handles extract serialize (GH#4531) Mads R. B.
Kristensen
- Drop importorskip-numpy-tests.patch merged upstream
- Skip python36 build. Upstream dropped support for Python < 3.7
OBS-URL: https://build.opensuse.org/request/show/882449
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=92
- Update to 2021.3.0
* This is the first release with support for Python 3.9 and the
last release with support for Python 3.6
* tcp.write(): cast memoryview to byte itemsize (#4555)
Mads R. B. Kristensen
* Refcount the thread_state.asynchronous flag (#4557) Mads
R. B. Kristensen
* Python 3.9 (#4460) crusaderky
* Better bokeh defaults for dashboard (#4554) Benjamin
Zaitlen
* Expose system monitor dashboard as individual plot for lab
extension (#4540) Jacob Tomlinson
* Pass on original temp dir from nanny to worker (#4549)
Martin Durant
* Serialize and split (#4541) Mads R. B. Kristensen
* Use the new HLG pack/unpack API in Dask (#4489) Mads R.
B. Kristensen
* Handle annotations for culled tasks (#4544) Tom
Augspurger
* Make sphinx autosummary and autoclass consistent (#4367)
Casey Clements
* Move _transition* to SchedulerState (#4545) jakirkham
* Migrate from travis to GitHub actions (#4504) crusaderky
* Move new_task to SchedulerState (#4527) jakirkham
* Batch more Scheduler sends (#4526) jakirkham
* transition_memory_released and get_nbytes() optimizations
(#4516) jakirkham
* Pin black pre-commit (#4533) James Bourbeau
* Read & write all frames in one pass (#4506) jakirkham
* Skip stream.write call for empty frames (#4507) jakirkham
* Prepend frame metadata header (#4505) jakirkham
* transition_processing_memory optimizations, etc. (#4487)
jakirkham
* Attempt to get client from worker in Queue and Variable
(#4490) James Bourbeau
* Use main branch for zict (#4499) jakirkham
* Use a callback to close TCP Comms, rather than check every
time (#4453) Matthew Rocklin
OBS-URL: https://build.opensuse.org/request/show/877788
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=90
- Update to version 2021.2.0
* Bump minimum Dask to 2021.02.0 (GH#4486) James Bourbeau
* Update TaskState documentation about dependents attribute
(GH#4440) Florian Jetter
* DOC: Autoreformat all functions docstrings (GH#4475) Matthias
Bussonnier
* Use cached version of is_coroutine_function in stream handling
to (GH#4481) Ian Rose
* Optimize transitions (GH#4451) jakirkham
* Create PULL_REQUEST_TEMPLATE.md (GH#4476) Ray Bell
* DOC: typo, directives ends with 2 colons :: (GH#4472) Matthias
Bussonnier
* DOC: Proper numpydoc syntax for distributed/protocol/*.py
(GH#4473) Matthias Bussonnier
* Update pytest.skip usage in test_server_listen (GH#4467) James
Bourbeau
* Unify annotations (GH#4406) Ian Rose
* Added worker resources from config (GH#4456) Tom Augspurger
* Fix var name in worker validation func (GH#4457) Gil Forsyth
* Refactor task_groups & task_prefixes (GH#4452) jakirkham
* Use parent._tasks in heartbeat (GH#4450) jakirkham
* Refactor SchedulerState from Scheduler (GH#4365) jakirkham
- Do not pin to tornado < 6.1. The referenced pull request reverted
that change and we have asyncio test problems anyway.
- Use new python_module boolean deps capability
OBS-URL: https://build.opensuse.org/request/show/870337
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=83
- Update to version 2021.1.1
* Make system monitor interval configurable (GH#4447) Matthew
Rocklin
* Add uvloop config value (GH#4448) Matthew Rocklin
* Additional optimizations to stealing (GH#4445) jakirkham
* Give clusters names (GH#4426) Jacob Tomlinson
* Use worker comm pool in Semaphore (GH#4195) Florian Jetter
* Set runspec on all new tasks to avoid deadlocks (GH#4432)
Florian Jetter
* Support TaskState objects in story methods (GH#4434) Matthew
Rocklin
* Support missing event loop in Client.asynchronous (GH#4436)
Matthew Rocklin
* Don’t require network to inspect tests (GH#4433) Matthew
Rocklin
- Changelog for 2021.1.0
* Add time started to scheduler info (GH#4425) Jacob Tomlinson
* Log adaptive error (GH#4422) Jacob Tomlinson
* Xfail normalization tests (GH#4411) Jacob Tomlinson
* Use dumps_msgpack and loads_msgpack when packing high level
graphs (GH#4409) Mads R. B. Kristensen
* Add nprocs auto option to dask-worker CLI (GH#4377) Jacob
Tomlinson
* Type annotation of _reevaluate_occupancy_worker (GH#4398)
jakirkham
* Type TaskGroup in active_states (GH#4408) jakirkham
* Fix test_as_current_is_thread_local (GH#4402) jakirkham
* Use list comprehensions to bind TaskGroup type (GH#4401)
jakirkham
* Make tests pass after 2028 (GH#4403) Bernhard M. Wiedemann
* Fix compilation warnings, decide_worker now a C func, stealing
improvements (GH#4375) jakirkham
* Drop custom __eq__ from Status (GH#4270) jakirkham
* test_performance_report: skip without bokeh (GH#4388) Bruno
Pagani
* Nanny now respects dask settings from ctx mgr (GH#4378)
Florian Jetter
* Better task duration estimates for outliers (GH#4213) selshowk
* Dask internal inherit config (GH#4364) Jacob Tomlinson
* Provide setup.py option to profile Cython code (GH#4362)
jakirkham
* Optimizations of *State and Task* objects and stealing
(GH#4358) jakirkham
* Cast SortedDict s to dict s in a few key places & other minor
changes (GH#4355) jakirkham
* Use task annotation priorities for user-level priorities
(GH#4354) James Bourbeau
* Added docs to highlevelgraph pack/unpack (GH#4352) Mads R. B.
Kristensen
* Optimizations in notable functions used by transitions
(GH#4351) jakirkham
* Silence exception when releasing futures on process shutdown
(GH#4309) Benjamin Zaitlen
- Deboard blind passenger requirement: scikit-build is only
available for Python > 3.6 (because of SciPy) and not needed here
OBS-URL: https://build.opensuse.org/request/show/867973
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=81
- update to version 2020.12.0
* Switched to CalVer for versioning scheme.
* The scheduler can now receives Dask HighLevelGraph s instead of
raw dictionary task graphs. This allows for a much more
efficient communication of task graphs from the client to the
scheduler.
* Added support for using custom Layer-level annotations like
priority, retries, etc. with the dask.annotations context
manager.
* Updated minimum supported version of Dask to 2020.12.0.
* Added many type annotations and updates to allow for gradually
Cythonizing the scheduler.
- Full changelog on
https://distributed.readthedocs.io/en/latest/changelog.html
- Cythonize scheduler: is platform dependent package now
- Iterative deselection of failing tests: kept adding tests to
pytest -k parameter until the test suite did not fail anymore.
* pytest-dev/pytest-asyncio#168, pytest-dev/pytest-asyncio#188
still not resolved
* upstream just pinned pytest-asyncio gh#dask/distributed#4212
- Support multiple python3 flavors gh#openSUSE/python-rpm-macros#66
- Move test into multibuild in order to avoid dependency cycle
with dask-distributed
OBS-URL: https://build.opensuse.org/request/show/862931
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=79
in 823921, there was a comment about enabling tests. I have to say that I don't understand bcond_with and how to set it to run tests (looks to me that they should be enabled?). Perhaps we can just accept this update and someone else can add the test independent of the update?
- update to version 2.26.0:
* Add logging for adaptive start and stop (GH#4101) Matthew Rocklin
* Don’t close a nannied worker if it hasn’t yet started (GH#4093)
Matthew Rocklin
* Respect timeouts when closing clients synchronously (GH#4096)
Matthew Rocklin
* Log when downloading a preload script (GH#4094) Matthew Rocklin
* dask-worker --nprocs accepts negative values (GH#4089) Dror
Speiser
* Support zero-worker clients (GH#4090) Matthew Rocklin
* Exclude fire-and-forget client from metrics (GH#4078) Tom
Augspurger
* Drop Serialized.deserialize() method (GH#4073) jakirkham
* Add timeout= keyword to Client.wait_for_workers method (GH#4087)
Matthew Rocklin
OBS-URL: https://build.opensuse.org/request/show/834027
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=67
- update to 2.20.0
* Link issue on using ``async`` with ``executor_submit`` (:pr:`3939`) `jakirkham`_
* Make dashboard server listens on all IPs by default even when interface is set explicitly (:pr:`3941`) `Loïc Estève`_
* Update logic for worker removal in check ttl (:pr:`3927`) `Benjamin Zaitlen`_
* Close a created cluster quietly (:pr:`3935`) `Matthew Rocklin`_
* Ensure ``Worker.run*`` handles ``kwargs`` correctly (:pr:`3937`) `jakirkham`_
* Restore ``Scheduler.time_started`` for Dask Gateway (:pr:`3934`) `Tom Augspurger`_
* Fix exception handling in ``_wait_until_connected`` (:pr:`3912`) `Alexander Clausen`_
* Make local directory if it does not exist (:pr:`3928`) `Matthew Rocklin`_
* Install vanilla status route if bokeh dependency is not satisfied (:pr:`3844`) `joshreback`_
* Make ``Worker.delete_data`` sync (:pr:`3922`) `Peter Andreas Entschev`_
* Fix ``ensure_bytes`` import location (:pr:`3919`) `jakirkham`_
* Fix race condition in repeated calls to ``cluster.adapt()`` (:pr:`3915`) `Jacob Tomlinson`_
2.19.0
* Notify worker plugins when a task is released (:pr:`3817`) `Nick Evans`_
* Update heartbeat checks in scheduler (:pr:`3896`) `Benjamin Zaitlen`_
* Make encryption default if ``Security`` is given arguments (:pr:`3887`) `Matthew Rocklin`_
* Show ``cpu_fraction`` on hover for dashboard workers circle plot. (:pr:`3906`) `Loïc Estève`_
* Prune virtual client on variable deletion (:pr:`3910`) `Marco Neumann`_
* Fix total aggregated metrics in dashboard (:pr:`3897`) `Loïc Estève`_
* Support Bokeh 2.1 (:pr:`3904`) `Matthew Rocklin`_
* Update ``related-work.rst`` (:pr:`3889`) `DomHudson`_
* Skip ``test_pid_file`` in older versions of Python (:pr:`3888`) `Matthew Rocklin`_
* Replace ``stream=`` with ``comm=`` in handlers (:pr:`3860`) `Julien Jerphanion`_
* Check hosts for ``None`` value in SSH cluster. (:pr:`3883`) `Matthias Bussonnier`_
* Allow dictionaries in ``security=`` keywords (:pr:`3874`) `Matthew Rocklin`_
* Use pickle protocol 5 with NumPy object arrays (:pr:`3871`) `jakirkham`_
* Cast any ``frame`` to ``uint8`` (same type as ``bytes``) (:pr:`3870`) `jakirkham`_
* Use ``Enum`` for worker, scheduler and nanny status. (:pr:`3853`) `Matthias Bussonnier`_
* Drop legacy ``buffer_interface`` assignment (:pr:`3869`) `jakirkham`_
OBS-URL: https://build.opensuse.org/request/show/819716
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=59
- Update to 2.1.0
* Fix typo that prevented error message
* Remove ``dask-mpi``
* Updates to use ``update_graph`` in task journey docs
* Fix Client repr with ``memory_info=None``
* Fix case where key, rather than ``TaskState``, could end up in ``ts.waiting_on``
* Use Keyword-only arguments
* Relax check for worker references in cluster context manager
* Add HTTPS support for the dashboard
* Use ``dask.utils.format_bytes``
- Update to 2.0.1
* Add python_requires entry to setup.py
* Correctly manage tasks beyond deque limit in TaskStream plot
* Fix diagnostics page for memory_limit=None
- Update to 2.0.0
* **Drop support for Python 2**
* Relax warnings before release
* Deprecate --bokeh/--no-bokeh CLI
* Typo in bokeh service_kwargs for dask-worker
* Update command line cli options docs
* Remove "experimental" from TLS docs
* Add warnings around ncores= keywords
* Add --version option to scheduler and worker CLI
* Raise when workers initialization times out
* Replace ncores with nthreads throughout codebase
* Add unknown pytest markers
* Delay lookup of allowed failures.
* Change address -> worker in ColumnDataSource for nbytes plot
* Remove module state in Prometheus Handlers
* Add stress test for UCX
* Add nanny logs
* Move some of the adaptive logic into the scheduler
* Add SpecCluster.new_worker_spec method
* Worker dashboard fixes
* Add async context managers to scheduler/worker classes
* Fix the resource key representation before sending graphs
* Allow user to configure whether workers are daemon.
* Pin pytest >=4 with pip in appveyor and python 3.5
* Add Experimental UCX Comm
* Close nannies gracefully
* add kwargs to progressbars
* Add back LocalCluster.__repr__.
* Move bokeh module to dashboard
* Close clusters at exit
* Add SchedulerPlugin TaskState example
* Add SpecificationCluster
* Replace register_worker_callbacks with worker plugins
* Proxy worker dashboards from scheduler dashboard
* Add docstring to Scheduler.check_idle_saturated
* Refer to LocalCluster in Client docstring
* Remove special casing of Scikit-Learn BaseEstimator serialization
* Fix two typos in Pub class docstring
* Support uploading files with multiple modules
* Change the main workers bokeh page to /status
* Cleanly stop periodic callbacks in Client
* Disable pan tool for the Progress, Byte Stored and Tasks Processing plot
* Except errors in Nanny's memory monitor if process no longer exists
* Handle heartbeat when worker has just left
* Modify styling of histograms for many-worker dashboard plots
* Add method to wait for n workers before continuing
* Support computation on delayed(None)
* Cleanup localcluster
* Use 'temporary-directory' from dask.config for Worker's directory
* Remove support for Iterators and Queues
- Update to 1.28.1
* Use config accessor method for "scheduler-address"
- Update to 1.28.0
* Add Type Attribute to TaskState
* Add waiting task count to progress title bar
* DOC: Clean up reference to cluster object
* Allow scheduler to politely close workers as part of shutdown
* Check direct_to_workers before using get_worker in Client
* Fixed comment regarding keeping existing level if less verbose
* Add idle timeout to scheduler
* Avoid deprecation warnings
* Use an LRU cache for deserialized functions
* Rename Worker._close to Worker.close
* Add Comm closed bookkeeping
* Explain LocalCluster behavior in Client docstring
* Add last worker into KilledWorker exception to help debug
* Set working worker class for dask-ssh
* Add as_completed methods to docs
* Add timeout to Client._reconnect
* Limit test_spill_by_default memory, reenable it
* Use proper address in worker -> nanny comms
* Fix deserialization of bytes chunks larger than 64MB
- Update to 1.27.1
* Adaptive: recommend close workers when any are idle
* Increase GC thresholds
* Add interface= keyword to LocalCluster
* Add worker_class argument to LocalCluster
* Remove Python 2.7 from testing matrix
* Add number of trials to diskutils test
* Fix parameter name in LocalCluster docstring
* Integrate stacktrace for low-level profiling
* Apply Black to standardize code styling
* added missing whitespace to start_worker cmd
* Updated logging module doc links from docs.python.org/2 to docs.python.org/3.
- Update to 1.27.0
* Add basic health endpoints to scheduler and worker bokeh.
* Improved description accuracy of --memory-limit option.
* Check self.dependencies when looking at dependent tasks in memory
* Add RabbitMQ SchedulerPlugin example
* add resources to scheduler update_graph plugin
* Use ensure_bytes in serialize_error
* Specify data storage explicitly from Worker constructor
* Change bokeh port keywords to dashboard_address
* .detach_() pytorch tensor to serialize data as numpy array.
* Add warning if creating scratch directories takes a long time
* Fix typo in pub-sub doc.
* Allow return_when='FIRST_COMPLETED' in wait
* Forward kwargs through Nanny to Worker
* Use ensure_dict instead of dict
* Specify protocol in LocalCluster
- Update to 1.26.1
* Fix LocalCluster to not overallocate memory when overcommitting threads per worker
* Make closing resilient to lacking an address
* fix typo in comment
* Fix double init of prometheus metrics
* Skip test_duplicate_clients without bokeh.
* Add blocked_handlers to servers
* Always yield Server.handle_comm coroutine
* Use yaml.safe_load
* Fetch executables from build root.
* Fix Torando 6 test failures
* Fix test_sync_closed_loop
- Update to 1.26.0
* Update style to fix recent flake8 update
* Fix typo in gen_cluster log message
* Allow KeyError when closing event loop
* Avoid thread testing for TCP ThreadPoolExecutor
* Find Futures inside SubgraphCallable
* Avoid AttributeError when closing and sending a message
* Add deprecation warning to dask_mpi.py
* Relax statistical profiling test
* Support alternative --remote-dask-worker SSHCluster() and dask-ssh CLI
* Iterate over full list of plugins in transition
* Create Prometheus Endpoint
* Use pytest.importorskip for prometheus test
* MAINT skip prometheus test when no installed
* Fix intermittent testing failures
* Avoid using nprocs keyword in dask-ssh if set to one
* Bump minimum Tornado version to 5.0
OBS-URL: https://build.opensuse.org/request/show/717982
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=25
