- update to version 2020.12.0
* Switched to CalVer for versioning scheme.
* The scheduler can now receives Dask HighLevelGraph s instead of
raw dictionary task graphs. This allows for a much more
efficient communication of task graphs from the client to the
scheduler.
* Added support for using custom Layer-level annotations like
priority, retries, etc. with the dask.annotations context
manager.
* Updated minimum supported version of Dask to 2020.12.0.
* Added many type annotations and updates to allow for gradually
Cythonizing the scheduler.
- Full changelog on
https://distributed.readthedocs.io/en/latest/changelog.html
- Cythonize scheduler: is platform dependent package now
- Iterative deselection of failing tests: kept adding tests to
pytest -k parameter until the test suite did not fail anymore.
* pytest-dev/pytest-asyncio#168, pytest-dev/pytest-asyncio#188
still not resolved
* upstream just pinned pytest-asyncio gh#dask/distributed#4212
- Support multiple python3 flavors gh#openSUSE/python-rpm-macros#66
- Move test into multibuild in order to avoid dependency cycle
with dask-distributed
OBS-URL: https://build.opensuse.org/request/show/862931
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=79
in 823921, there was a comment about enabling tests. I have to say that I don't understand bcond_with and how to set it to run tests (looks to me that they should be enabled?). Perhaps we can just accept this update and someone else can add the test independent of the update?
- update to version 2.26.0:
* Add logging for adaptive start and stop (GH#4101) Matthew Rocklin
* Don’t close a nannied worker if it hasn’t yet started (GH#4093)
Matthew Rocklin
* Respect timeouts when closing clients synchronously (GH#4096)
Matthew Rocklin
* Log when downloading a preload script (GH#4094) Matthew Rocklin
* dask-worker --nprocs accepts negative values (GH#4089) Dror
Speiser
* Support zero-worker clients (GH#4090) Matthew Rocklin
* Exclude fire-and-forget client from metrics (GH#4078) Tom
Augspurger
* Drop Serialized.deserialize() method (GH#4073) jakirkham
* Add timeout= keyword to Client.wait_for_workers method (GH#4087)
Matthew Rocklin
OBS-URL: https://build.opensuse.org/request/show/834027
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=67
- update to 2.20.0
* Link issue on using ``async`` with ``executor_submit`` (:pr:`3939`) `jakirkham`_
* Make dashboard server listens on all IPs by default even when interface is set explicitly (:pr:`3941`) `Loïc Estève`_
* Update logic for worker removal in check ttl (:pr:`3927`) `Benjamin Zaitlen`_
* Close a created cluster quietly (:pr:`3935`) `Matthew Rocklin`_
* Ensure ``Worker.run*`` handles ``kwargs`` correctly (:pr:`3937`) `jakirkham`_
* Restore ``Scheduler.time_started`` for Dask Gateway (:pr:`3934`) `Tom Augspurger`_
* Fix exception handling in ``_wait_until_connected`` (:pr:`3912`) `Alexander Clausen`_
* Make local directory if it does not exist (:pr:`3928`) `Matthew Rocklin`_
* Install vanilla status route if bokeh dependency is not satisfied (:pr:`3844`) `joshreback`_
* Make ``Worker.delete_data`` sync (:pr:`3922`) `Peter Andreas Entschev`_
* Fix ``ensure_bytes`` import location (:pr:`3919`) `jakirkham`_
* Fix race condition in repeated calls to ``cluster.adapt()`` (:pr:`3915`) `Jacob Tomlinson`_
2.19.0
* Notify worker plugins when a task is released (:pr:`3817`) `Nick Evans`_
* Update heartbeat checks in scheduler (:pr:`3896`) `Benjamin Zaitlen`_
* Make encryption default if ``Security`` is given arguments (:pr:`3887`) `Matthew Rocklin`_
* Show ``cpu_fraction`` on hover for dashboard workers circle plot. (:pr:`3906`) `Loïc Estève`_
* Prune virtual client on variable deletion (:pr:`3910`) `Marco Neumann`_
* Fix total aggregated metrics in dashboard (:pr:`3897`) `Loïc Estève`_
* Support Bokeh 2.1 (:pr:`3904`) `Matthew Rocklin`_
* Update ``related-work.rst`` (:pr:`3889`) `DomHudson`_
* Skip ``test_pid_file`` in older versions of Python (:pr:`3888`) `Matthew Rocklin`_
* Replace ``stream=`` with ``comm=`` in handlers (:pr:`3860`) `Julien Jerphanion`_
* Check hosts for ``None`` value in SSH cluster. (:pr:`3883`) `Matthias Bussonnier`_
* Allow dictionaries in ``security=`` keywords (:pr:`3874`) `Matthew Rocklin`_
* Use pickle protocol 5 with NumPy object arrays (:pr:`3871`) `jakirkham`_
* Cast any ``frame`` to ``uint8`` (same type as ``bytes``) (:pr:`3870`) `jakirkham`_
* Use ``Enum`` for worker, scheduler and nanny status. (:pr:`3853`) `Matthias Bussonnier`_
* Drop legacy ``buffer_interface`` assignment (:pr:`3869`) `jakirkham`_
OBS-URL: https://build.opensuse.org/request/show/819716
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=59
- Update to 2.1.0
* Fix typo that prevented error message
* Remove ``dask-mpi``
* Updates to use ``update_graph`` in task journey docs
* Fix Client repr with ``memory_info=None``
* Fix case where key, rather than ``TaskState``, could end up in ``ts.waiting_on``
* Use Keyword-only arguments
* Relax check for worker references in cluster context manager
* Add HTTPS support for the dashboard
* Use ``dask.utils.format_bytes``
- Update to 2.0.1
* Add python_requires entry to setup.py
* Correctly manage tasks beyond deque limit in TaskStream plot
* Fix diagnostics page for memory_limit=None
- Update to 2.0.0
* **Drop support for Python 2**
* Relax warnings before release
* Deprecate --bokeh/--no-bokeh CLI
* Typo in bokeh service_kwargs for dask-worker
* Update command line cli options docs
* Remove "experimental" from TLS docs
* Add warnings around ncores= keywords
* Add --version option to scheduler and worker CLI
* Raise when workers initialization times out
* Replace ncores with nthreads throughout codebase
* Add unknown pytest markers
* Delay lookup of allowed failures.
* Change address -> worker in ColumnDataSource for nbytes plot
* Remove module state in Prometheus Handlers
* Add stress test for UCX
* Add nanny logs
* Move some of the adaptive logic into the scheduler
* Add SpecCluster.new_worker_spec method
* Worker dashboard fixes
* Add async context managers to scheduler/worker classes
* Fix the resource key representation before sending graphs
* Allow user to configure whether workers are daemon.
* Pin pytest >=4 with pip in appveyor and python 3.5
* Add Experimental UCX Comm
* Close nannies gracefully
* add kwargs to progressbars
* Add back LocalCluster.__repr__.
* Move bokeh module to dashboard
* Close clusters at exit
* Add SchedulerPlugin TaskState example
* Add SpecificationCluster
* Replace register_worker_callbacks with worker plugins
* Proxy worker dashboards from scheduler dashboard
* Add docstring to Scheduler.check_idle_saturated
* Refer to LocalCluster in Client docstring
* Remove special casing of Scikit-Learn BaseEstimator serialization
* Fix two typos in Pub class docstring
* Support uploading files with multiple modules
* Change the main workers bokeh page to /status
* Cleanly stop periodic callbacks in Client
* Disable pan tool for the Progress, Byte Stored and Tasks Processing plot
* Except errors in Nanny's memory monitor if process no longer exists
* Handle heartbeat when worker has just left
* Modify styling of histograms for many-worker dashboard plots
* Add method to wait for n workers before continuing
* Support computation on delayed(None)
* Cleanup localcluster
* Use 'temporary-directory' from dask.config for Worker's directory
* Remove support for Iterators and Queues
- Update to 1.28.1
* Use config accessor method for "scheduler-address"
- Update to 1.28.0
* Add Type Attribute to TaskState
* Add waiting task count to progress title bar
* DOC: Clean up reference to cluster object
* Allow scheduler to politely close workers as part of shutdown
* Check direct_to_workers before using get_worker in Client
* Fixed comment regarding keeping existing level if less verbose
* Add idle timeout to scheduler
* Avoid deprecation warnings
* Use an LRU cache for deserialized functions
* Rename Worker._close to Worker.close
* Add Comm closed bookkeeping
* Explain LocalCluster behavior in Client docstring
* Add last worker into KilledWorker exception to help debug
* Set working worker class for dask-ssh
* Add as_completed methods to docs
* Add timeout to Client._reconnect
* Limit test_spill_by_default memory, reenable it
* Use proper address in worker -> nanny comms
* Fix deserialization of bytes chunks larger than 64MB
- Update to 1.27.1
* Adaptive: recommend close workers when any are idle
* Increase GC thresholds
* Add interface= keyword to LocalCluster
* Add worker_class argument to LocalCluster
* Remove Python 2.7 from testing matrix
* Add number of trials to diskutils test
* Fix parameter name in LocalCluster docstring
* Integrate stacktrace for low-level profiling
* Apply Black to standardize code styling
* added missing whitespace to start_worker cmd
* Updated logging module doc links from docs.python.org/2 to docs.python.org/3.
- Update to 1.27.0
* Add basic health endpoints to scheduler and worker bokeh.
* Improved description accuracy of --memory-limit option.
* Check self.dependencies when looking at dependent tasks in memory
* Add RabbitMQ SchedulerPlugin example
* add resources to scheduler update_graph plugin
* Use ensure_bytes in serialize_error
* Specify data storage explicitly from Worker constructor
* Change bokeh port keywords to dashboard_address
* .detach_() pytorch tensor to serialize data as numpy array.
* Add warning if creating scratch directories takes a long time
* Fix typo in pub-sub doc.
* Allow return_when='FIRST_COMPLETED' in wait
* Forward kwargs through Nanny to Worker
* Use ensure_dict instead of dict
* Specify protocol in LocalCluster
- Update to 1.26.1
* Fix LocalCluster to not overallocate memory when overcommitting threads per worker
* Make closing resilient to lacking an address
* fix typo in comment
* Fix double init of prometheus metrics
* Skip test_duplicate_clients without bokeh.
* Add blocked_handlers to servers
* Always yield Server.handle_comm coroutine
* Use yaml.safe_load
* Fetch executables from build root.
* Fix Torando 6 test failures
* Fix test_sync_closed_loop
- Update to 1.26.0
* Update style to fix recent flake8 update
* Fix typo in gen_cluster log message
* Allow KeyError when closing event loop
* Avoid thread testing for TCP ThreadPoolExecutor
* Find Futures inside SubgraphCallable
* Avoid AttributeError when closing and sending a message
* Add deprecation warning to dask_mpi.py
* Relax statistical profiling test
* Support alternative --remote-dask-worker SSHCluster() and dask-ssh CLI
* Iterate over full list of plugins in transition
* Create Prometheus Endpoint
* Use pytest.importorskip for prometheus test
* MAINT skip prometheus test when no installed
* Fix intermittent testing failures
* Avoid using nprocs keyword in dask-ssh if set to one
* Bump minimum Tornado version to 5.0
OBS-URL: https://build.opensuse.org/request/show/717982
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=25
- Fix the PyYAML dependency
- specfile:
* updated dask requirement, added pyyaml
- update to version 1.22.0:
* Overhaul configuration (GH#1948) Matthew Rocklin
* Replace get= keyword with scheduler= (GH#1959) Matthew Rocklin
* Use tuples in msgpack (GH#2000) Matthew Rocklin and Marius van
Niekerk
* Unify handling of high-volume connections (GH#1970) Matthew
Rocklin
* Automatically scatter large arguments in joblib connector
(GH#2020) (GH#2030) Olivier Grisel
* Turn click Python 3 locales failure into a warning (GH#2001)
Matthew Rocklin
* Rely on dask implementation of sizeof (GH#2042) Matthew Rocklin
* Replace deprecated workers.iloc with workers.values() (GH#2013)
Grant Jenks
* Introduce serialization families (GH#1912) Matthew Rocklin
* Add PubSub (GH#1999) Matthew Rocklin
* Add Dask stylesheet to documentation Matthew Rocklin
* Avoid recomputation on partially-complete results (GH#1840)
Matthew Rocklin
* Use sys.prefix in popen for testing (GH#1954) Matthew Rocklin
* Include yaml files in manifest Matthew Rocklin
* Use self.sync so Client.processing works in asynchronous context
(GH#1962) Henry Doupe
* Fix bug with bad repr on closed client (GH#1965) Matthew Rocklin
* Parse –death-timeout keyword in dask-worker (GH#1967) Matthew
OBS-URL: https://build.opensuse.org/request/show/620192
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/python-distributed?expand=0&rev=4
- update to version 1.21.6:
* Fix numeric environment variable configuration (GH#1885) Joseph
Atkins-Kurkish
* support bytearrays in older lz4 library (GH#1886) Matthew Rocklin
* Remove started timeout in nanny (GH#1852) Matthew Rocklin
* Don’t log errors in sync (GH#1894) Matthew Rocklin
* downgrade stale lock warning to info logging level (GH#1890)
Matthew Rocklin
* Fix UnboundLocalError for key (GH#1900) John Kirkham
* Resolve deployment issues in Python 2 (GH#1905) Matthew Rocklin
* Support retries and priority in Client.get method (GH#1902)
Matthew Rocklin
* Add additional attributes to task page if applicable (GH#1901)
Matthew Rocklin
* Add count method to as_completed (GH#1897) Matthew Rocklin
* Extend default timeout to 10s (GH#1904) Matthew Rocklin
- changes from version 1.21.5:
* Increase default allowable tick time to 3s (GH#1854) Matthew
Rocklin
* Handle errant workers when another worker has data (GH#1853)
Matthew Rocklin
* Close multiprocessing queue in Nanny to reduce open file
descriptors (GH#1862) Matthew Rocklin
* Extend nanny started timeout to 30s, make configurable (GH#1865)
Matthew Rocklin
* Comment out the default config file (GH#1871) Matthew Rocklin
* Update to fix bokeh 0.12.15 update errors (GH#1872) Matthew
Rocklin
* Downgrade Event Loop unresponsive warning to INFO level (GH#1870)
Matthew Rocklin
* Add fifo timeout to control priority generation (GH#1828) Matthew
Rocklin
* Add retire_workers API to Client (GH#1876) Matthew Rocklin
* Catch NoSuchProcess error in Nanny.memory_monitor (GH#1877)
Matthew Rocklin
* Add uid to nanny queue communitcations (GH#1880) Matthew Rocklin
- changes from version 1.21.4:
* Avoid passing bytearrays to snappy decompression (GH#1831) Matthew
Rocklin
* Specify IOLoop in Adaptive (GH#1841) Matthew Rocklin
* Use connect-timeout config value throughout client (GH#1839)
Matthew Rocklin
* Support direct= keyword argument in Client.get (GH#1845) Matthew
Rocklin
- changes from version 1.21.3:
* Add cluster superclass and improve adaptivity (GH#1813) Matthew
Rocklin
* Fixup tests and support Python 2 for Tornado 5.0 (GH#1818) Matthew
Rocklin
* Fix bug in recreate_error when dependencies are dropped (GH#1815)
Matthew Rocklin
* Add worker time to live in Scheduler (GH#1811) Matthew Rocklin
* Scale adaptive based on total_occupancy (GH#1807) Matthew Rocklin
* Support calling compute within worker_client (GH#1814) Matthew
Rocklin
* Add percentage to profile plot (GH#1817) Brett Naul
* Overwrite option for remote python in dask-ssh (GH#1812) Sven
Kreiss
- changes from version 1.21.2:
* Fix bug where we didn’t check idle/saturated when stealing
(GH#1801) Matthew Rocklin
* Fix bug where client was noisy when scheduler closed unexpectedly
(GH#1806) Matthew Rocklin
* Use string-based timedeltas (like '500 ms') everywhere (GH#1804)
Matthew Rocklin
* Keep logs in scheduler and worker even if silenced (GH#1803)
Matthew Rocklin
* Support minimum, maximum, wait_count keywords in Adaptive
(GH#1797) Jacob Tomlinson and Matthew Rocklin
* Support async protocols for LocalCluster, replace start= with
asynchronous= (GH#1798) Matthew Rocklin
* Avoid restarting workers when nanny waits on scheduler (GH#1793)
Matthew Rocklin
* Use IOStream.read_into() when available (GH#1477) Antoine Pitrou
* Reduce LocalCluster logging threshold from CRITICAL to WARN
(GH#1785) Andy Jones
* Add futures_of to API docs (GH#1783) John Kirkham
* Make diagnostics link in client configurable (GH#1810) Matthew
Rocklin
- changes from version 1.21.1:
* Fixed an uncaught exception in distributed.joblib with a
LocalCluster using only threads (GH#1775) Tom Augspurger
* Format bytes in info worker page (GH#1752) Matthew Rocklin
* Add pass-through arguments for scheduler/worker –preload
modules. (GH#1634) Alexander Ford
* Use new LZ4 API (GH#1757) Thrasibule
* Replace dask.optimize with dask.optimization (GH#1754) Matthew
Rocklin
* Add graph layout engine and bokeh plot (GH#1756) Matthew Rocklin
* Only expand name with –nprocs if name exists (GH#1776) Matthew
Rocklin
* specify IOLoop for stealing PeriodicCallback (GH#1777) Matthew
Rocklin
* Fixed distributed.joblib with no processes Tom Augspurger
* Use set.discard to avoid KeyErrors in stealing (GH#1766) Matthew
Rocklin
* Avoid KeyError when task has been released during steal (GH#1765)
Matthew Rocklin
* Add versions routes to avoid the use of run in Client.get_versions
(GH#1773) Matthew Rocklin
* Add write_scheduler_file to Client (GH#1778) Joe Hamman
* Default host to tls:// if tls information provided (GH#1780)
Matthew Rocklin
- Update descriptions.
OBS-URL: https://build.opensuse.org/request/show/603175
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/python-distributed?expand=0&rev=2
