* Fix to_json() with orient="table" to correctly use the
“string” type in the JSON Table Schema for StringDtype columns
* Boolean operations (|, &, ^) with bool-dtype objects on the
left and StringDtype objects on the right now cast the string
to bool, with a deprecation warning
* Fixed match(), fullmatch() and contains() string methods with
compiled regex for the Arrow-backed string dtype
* Bug in Series.replace() and DataFrame.replace() inconsistently
replacing matching values when missing values are present
for string dtypes
- Change skipped tests to also support Python 3.13.
- Skip two tests that fail with Numpy 2.1.
- Prepare for Python 3.13, by skipping it if we aren't building for it.
- enable py312 testing, remove py39 testing
RangeIndex
the "c" and "python" engines
* Bug in Series.str.split() and Series.str.rsplit() with expand=True
(GH42915)
* Fixed regression in DataFrame.groupby.rolling.cov() and
* The deprecated attributes _AXIS_NAMES and _AXIS_NUMBERS of
* Bumped minimum fastparquet version to 0.4.0 to avoid
* Bumped minimum pymysql version to 0.8.1 to avoid test failures
* Added reference to backwards incompatible check_freq arg of
testing.assert_frame_equal() and testing.assert_series_equal()
* WARNING:
no longer maintained. The xlrd package is now only for reading
Previously, the default argument engine=None to read_excel()
would result in using the xlrd engine in many cases, including
new Excel 2007+ (.xlsx) files. If openpyxl is installed, many
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=189
- Update to 2024.12.0
* Revert "Add LLM chatbot to Dask docs (#11556)" @dchudz (#11577)
* Automatically rechunk if array in to_zarr has irregular chunks
@phofl (#11553)
* Blockwise uses Task class @fjetter (#11568)
* Migrate rechunk and reshape to task spec @phofl (#11555)
* Cache svg-representation for arrays @dcherian (#11560)
* Fix empty input for containers @fjetter (#11571)
* Convert Bag graphs to TaskSpec graphs during optimization
@fjetter (#11569)
* add LLM chatbot to Dask docs @dchudz (#11556)
* Add support for Python 3.13 @phofl (#11456)
* Fuse data nodes in linear fusion too @phofl (#11549)
* Migrate slicing code to task spec @phofl (#11548)
* Speed up ArraySliceDep tokenization @phofl (#11551)
* Fix fusing of p2p barrier tasks @phofl (#11543)
* Remove infra/mentions of GPU CI @charlesbluca (#11546)
* Temporarily disable gpuCI update CI job @jrbourbeau (#11545)
* Use BlockwiseDep to implement map_blocks keywords @phofl
(#11542)
* Remove optimize_slices @phofl (#11538)
* Make reshape_blockwise a noop if shape is the same @phofl
(#11541)
* Remove read-only flag from open_arry in open_zarr @phofl
(#11539)
* Implement linear_fusion for task spec class @phofl (#11525)
* Remove recursion from TaskSpec @fjetter (#11477)
* Fixup test after dask-expr change @phofl (#11536)
* Bump codecov/codecov-action from 3 to 5 @dependabot (#11532)
* Create dask-expr frame directly without roundtripping @phofl
OBS-URL: https://build.opensuse.org/request/show/1228726
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=187
* profile._f_lineno: handle next_line being None in Python 3.13
* Cache global query-planning config
* Python 3.13 fixes
* Fix test_map_freq_to_period_start for pandas=3
* Tokenizing memmap arrays will now avoid materializing the array into
memory.
* Fix test_dt_accessor with query planning disabled
* Remove deprecated dask.compatibility module
* Ensure compatibility for xarray.NamedArray
* Avoid rounding error in
test_prometheus_collect_count_total_by_cost_multipliers
* Log key collision count in update_graph log event
* Rename safe to expected in Scheduler.remove_worker
* Eagerly update aggregate statistics for TaskPrefix instead of
calculating them on-demand
* Improve graph submission time for P2P rechunking by avoiding unpack
recursion into indices
* Add safe keyword to remove-worker event
* Improved errors and reduced logging for P2P RPC calls
* Adjust P2P tests for dask-expr
* Iterate over copy of Server.digests_total_since_heartbeat to avoid
RuntimeError
* Add Prometheus gauge for task groups
* Fix too strict assertion in shuffle code for pandas subclasses
* Reduce noise from erring tasks that are not supposed to be running
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=169
* see corresponding dask update:
* https://docs.dask.org/en/stable/changelog.html#v2023-12-0
* Bokeh 3.3.0 compatibility
- Update to version 2022.11.1
* Automatically mark all WorkerState tests (:pr:`6706`) crusaderky
- Update to 2022.6.1
- add distributed-fix-python310.patch to fix build with python 3.10
- Update to version 2021.9.1
* Fix add_plugin warnings
* Add list around iterator in handle_missing_dep
* Jupyter-client 7 compatiblity
* Add standalone dashboard page for GPU usage (GH#4556) Jacob
* Handle stream is None case in TCP comm finalizer (GH#4631)
* Include LIST_PICKLE in NumPy array serialization (GH#4632)
* Rename annotation plugin in test_highlevelgraph.py (GH#4618)
* Skip NumPy and pandas tests if not importable (GH#4563) Ben
* Remove utils.shutting_down in favor of sys.is_finalizing
* Introduce events log length config option (GH#4615) Fabian
* Upstream config serialization and inheritance (GH#4372) Jacob
* Add check to scheduler creation in SpecCluster (GH#4605) Jacob
* Make length of events deque configurable (GH#4604) Fabian
* Add explicit fetch state to worker TaskState (GH#4470) Gil
* pickle_loads(): Handle empty memoryview (GH#4595) Mads R. B.
* Switch documentation builds for PRs to readthedocs (GH#4599)
* If SpecCluster fails to start attempt to gracefully close out
* IPython magics: remove deprecated ioloop workarounds (GH#4530)
* Add GitHub actions workflow to cancel duplicate builds
* Remove outdated macOS build badge from README (GH#4576) James
* Change default branch from master to main (GH#4495) Julia
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=149
- Update to 2023.5.1
* This release drops support for Python 3.8
## Enhancements
* Exclude IPython code from computations (GH#7788) Miles
* Drop Python 3.8 support (GH#7840) Thomas Grainger
* Add storage_options to performance_report (GH#7636) ypogorelova
* Don’t warn about mismatched msgpack (GH#7839) Irina Truong
* Clean up sys.path on Server shutdown (GH#7838) James Bourbeau
* Dashboard: Fine Performance Metrics (GH#7725) Miles
## Bug Fixes
* Properly handle unknown chunk sizes in P2P rechunking (GH#7856)
Hendrik Makait
* Minimal change to work around (GH#7726) / support for UCX
(GH#7851) Benjamin Zaitlen
* Don’t end computations until cluster is truly idle (GH#7790)
crusaderky
- Release 2023.5.0
## Enhancements
* Client.upload_file send to both Workers and Scheduler and
rename scratch directory (GH#7802) Miles
* Allow dashboard to be used with bokeh prereleases (GH#7814)
James Bourbeau
## Bug Fixes
* Ensure log_event of non-msgpack serializable object do not kill
servers (GH#7472) Florian Jetter
* Fix test_nanny.py duplicated pytestmark definitions (GH#7819)
Thomas Grainger
* Fix flaky
test_dask_worker.py::test_single_executable_deprecated
(GH#7817) Thomas Grainger
OBS-URL: https://build.opensuse.org/request/show/1090160
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=142
- Update to 2023.3.2
## Enhancements
* Enhanced thread-safety in zict.File (GH#7691) crusaderky
* Future deserialization without available client (GH#7580)
Florian Jetter
* Support adjusting GIL monitoring interval (GH#7650) Miles
* Gracefully stop GIL monitoring if running (GH#7652) Miles
* Fine performance metrics for execute, gather_dep, etc.
(GH#7586) crusaderky
* Add GIL metric to dashboard (GH#7646) Miles
* Expose scheduler idle via RPC and HTTP API (GH#7642) Jacob
Tomlinson
* Add full dashboard link in scheduler logs (GH#7631) Miles
## Bug Fixes
* Tell workers when their peers have left (so they don’t hang
fetching data from them) (GH#7574) Thomas Grainger
* Fix regression in dashboard after (GH#7586) (GH#7683)
crusaderky
* Fix OverflowError in Cluster._sync_cluster_info() (GH#7648)
Hendrik Makait
* Ensure that serialized data is measured correctly (GH#7593)
Florian Jetter
## Maintenance
* Avoid bool object has no attribute close in @gen_cluster
(GH#7657) Thomas Grainger
* Fix failing test_server_close_stops_gil_monitoring (GH#7659)
James Bourbeau
* Remove weakref finalizer for Offload Executor (GH#7644) Florian
Jetter
- Enable python311 build and test: numba not a strict requirement
OBS-URL: https://build.opensuse.org/request/show/1074755
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=138
- Skip more flaky tests
- Update to 2023.3.1
## Enhancements
* Add Jupyter link to dashboard menu if --jupyter flag is set
(GH#7638) Jacob Tomlinson
* Bump minimum click version from 7.0 to 8.0 (GH#7637) Miles
* Extend dask metapackage dependencies (GH#7630) James Bourbeau
* Further improvements to Client.restart_workers (GH#7620) Miles
* P2P offload get_output_partition (GH#7587) Florian Jetter
* Initial integration of GIL contention metric (GH#7624) Miles
* Add dashboard documentation links (GH#7610) Miles
* Rename shuffle/rechunk config option/kwarg to method (GH#7623)
Hendrik Makait
* Return results in restart_workers (GH#7606) Miles
* Ensure client key cancellation uses ordered messages (GH#7583)
Florian Jetter
## Bug Fixes
* Fix undefined async_wait_for -> async_poll_for (GH#7627) Miles
* Don’t send client heartbeat without a scheduler_comm (GH#7612)
James Bourbeau
* Do not unspill on free-keys (GH#7607) crusaderky
## Documentation
* Add notes to Client.submit, Client.map, and Client.scatter with
the description of the current task graph resolution algorithm
limitations (GH#7588) Eugene Druzhynin
## Maintenance
* Use range with pickle protocol versions (GH#7635) jakirkham
* Share thread pool among P2P shuffle runs (GH#7621) Hendrik
Makait
OBS-URL: https://build.opensuse.org/request/show/1071308
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=136
- Update to 2023.1.1
## Enhancements
* P2P shuffle deduplicates data and can be run several times
(GH#7486) Hendrik Makait
* Reverse order of get_logs() and get_worker_logs() (GH#7475)
Nicholas R. Knezek
* Add prometheus metric for time and memory used per task prefix
(GH#7406) Thomas Grainger
* Additive worker counts in Prometheus (GH#7468) crusaderky
* Add help tool for taskstream (GH#7478) Florian Jetter
## Bug Fixes
* Do not allow for a worker to reject a drop replica request
(GH#7490) Hendrik Makait
* Fix un/packing for namedtuples with custom constructors
(GH#7465) antonymayi
* Documentation
* Remove timeout= from docstring example for worker_client
(GH#7497) Florian Jetter
## Maintenance
* Ignore get_default_shuffle_algorithm linting issue (GH#7506)
Florian Jetter
* Remove set_config when using default client (GH#7482) Florian
Jetter
* Update gpuCI RAPIDS_VER to 23.04 (GH#7501)
* Fix test_balance_expensive_tasks and improve helper functions
in test_steal.py (GH#7253) Hendrik Makait
* Sign every compute task with run ID to correlate response
(GH#7463) Hendrik Makait
- Release 2023.1.0
## New Features
* Add local SubprocessCluster that runs workers in separate
processes (GH#7431) Hendrik Makait
## Enhancements
* Ensure client session is quiet after cluster.close() or
client.shutdown() (GH#7429) James Bourbeau
* Set lifetime-stagger default value to None (GH#7445)
bstadlbauer
* Memory thresholds should never be exactly 0.0 (GH#7458) Stuart
Berg
* Remove the Incorrect-Sizeof-Warning (GH#7450) Mads R. B.
Kristensen
* Log exceptions in P2P shuffle tasks (GH#7442) Hendrik Makait
* Add support for packing namedtuple and add test for future
resolution in submit (GH#7292) Andrew
* Avoid deep copy on lz4 decompression (GH#7437) crusaderky
* Avoid deep copy of numpy buffers on unspill (GH#7435)
crusaderky
* Don’t error when clicking on empty task stream plot (GH#7432)
James Bourbeau
* Do not count spilled memory when comparing vs. process memory
(GH#7430) crusaderky
* Stop Client periodic callbacks during shutdown() (GH#7428)
James Bourbeau
* Add dask spec CLI (GH#7427) Matthew Rocklin
* Create new zstd (de)compressor for each compression call
(GH#7404) Dylan Wragge
* Rename managed_in_memory etc. to match GUI (GH#7418) crusaderky
* Warn users when sizeof() returns inflated output (GH#7419)
crusaderky
## Bug Fixes
* Ensure dicts are properly recognized as msgpack serializable
(GH#7473) Florian Jetter
* Reset state of ShuffleSchedulerExtension on restart (GH#7446)
Hendrik Makait
* Reject non-string column names in P2P shuffle (GH#7447) Hendrik
Makait
* Avoid int32 in dashboard (GH#7443) Matthew Rocklin
* Fix P2PShuffle serialization for categorical data (GH#7410)
Hendrik Makait
* WorkerPorcess blocks on kill if still starting (GH#7424)
Matthew Rocklin
## Documentation
* Move Prometheus docs from dask/dask (GH#7405) crusaderky
## Maintenance
* Various cleanups in semaphore (GH#5885) Florian Jetter
* test_rlimit fails on MacOSX (GH#7457) crusaderky
* Bump actions/checkout from 3.2.0 to 3.3.0 (GH#7464)
* Remove conditional imports of psutil (GH#7462) crusaderky
* Drop support for zict < 2.1.0 (GH#7456) crusaderky
* Fix flaky test_digests (GH#7454) crusaderky
* Add minimum dependency testing to CI (GH#7285) Charles
Blackmon-Luca
* Avoid overflow in statitics.mean (GH#7426) Matthew Rocklin
* Ignore numpy bool8 deprecation (GH#7423) Matthew Rocklin
* Add missing skips for pyarrow (GH#7416) Elliott Sales de
Andrade
* Be more permissive about expected ciphers in tests (GH#7417)
Elliott Sales de Andrade
* Revert “TST: Fetch executables from build root (GH#2551)”
(GH#7415) Elliott Sales de Andrade
OBS-URL: https://build.opensuse.org/request/show/1063268
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=130
- Update to 2022.12.1
## Enhancements
* SpillBuffer metrics (GH#7368) crusaderky
* Prometheus: measure how much spilling blocks the event loop (GH#7370) crusaderky
* Add transfer_outgoing_bytes_total metric (GH#7388) Gabe Joseph
* Fail P2PShuffle gracefully upon worker failure (GH#7326) Hendrik Makait
## Bug Fixes
* Select queued tasks in stimuli, not transitions (GH#7402) Gabe Joseph
* Check ContextVar in default_client (GH#7369) Matthew Rocklin
* Fix sending event messages to non-subscribers (GH#7014) Laurence Watts
* Set sizing mode on Tabs to avoid layout collapse (GH#7365) Mateusz Paprocki
## Maintenance
* Restructure P2PShuffle extensions (GH#7390) Hendrik Makait
* Add Python 3.11 classifier (GH#7408) James Bourbeau
* Add support for Python 3.11 (GH#7249) Thomas Grainger
* Add test for using annotations with client.submit and client.map (GH#7399) James Bourbeau
* Bump actions/checkout from 3.1.0 to 3.2.0 (GH#7393)
* Remove superfluous ShuffleSchedulerExtension.barriers (GH#7389) Hendrik Makait
* Remove ignore annotation-unchecked (GH#7379) crusaderky
* Remove tornado max version from nightly recipe (GH#7376) Charles Blackmon-Luca
* Remove the experimental feature warning for Semaphore (GH#7373) Florian Jetter
- Release 2022.12.0
## Enhancements
* Expose event loop health metrics in Prometheus (GH#7360) Hendrik Makait
* Allow log propagation by default (GH#5669) Florian Jetter
* Clean up of unpack_remotedata() (GH#7322) Mads R. B. Kristensen
* Upgrade to tornado 6.2 (GH#7286) Thomas Grainger
* Introduce Server level comm counters (GH#7332) Florian Jetter
* Prometheus debug log (GH#7302) Florian Jetter
## Bug Fixes
* Catch BaseException s from user tasks (GH#5997) Gabe Joseph
* Impossible use case of erred deps in transition to waiting (GH#7354) crusaderky
* Fix a deadlock when queued tasks are resubmitted quickly in succession (GH#7348) Florian Jetter
## Maintenance
* Fetch all artifacts (GH#7355) Enrico Minack
* Delay fsspec and urllib3 import time (GH#6659) Florian Jetter
* Bump mypy (GH#7349) crusaderky
* Bump mypy and remove win specific run (GH#7344) Florian Jetter
* Finish overhaul of SchedulerState annotations (GH#7333) crusaderky
* Fix flaky test_pause_while_spilling (GH#7334) Gabe Joseph
* Update gpuCI RAPIDS_VER to 23.02 (GH#7337)
- Drop distributed-pr7286-tornado-6-2.patch
- Move to PEP517 build
OBS-URL: https://build.opensuse.org/request/show/1056224
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=128
- Update to version 2022.11.1
## Enhancements
* Restrict bokeh=3 support (GH#7329) Gabe Joseph (ignored in rpm
fixed by bokek 3.0.2, see gh#dask/dask#9659)
* Respect death timeout when waiting for scheduler file (GH#7296)
Florian Jetter
* Always raise exception if P2PShuffle s send fails (GH#7317)
Hendrik Makait
## Maintenance
* Remove exception handling from transitions (GH#7316) crusaderky
* Turn private functions into private SchedulerState methods
(GH#7260) Hendrik Makait
* Bump toolz minimum version to 0.10.0 (GH#7309) Sam Grayson
- Release 2022.11.0
## Note
* This release changes the default scheduling mode to use
queuing. This will significantly reduce cluster memory use in
most cases, and generally improve stability and performance.
Learn more here and please provide any feedback on this
discussion.
* In rare cases, this could make some workloads slower. See the
documentation for more information, and how to switch back to
the old mode.
## New Features
* Add ForwardOutput worker plugin to forward stdout and stderr to
client. (GH#7297) Hendrik Makait
* Duration counters on prefix level (GH#7288) Florian Jetter
* Include button for launching JupyterLab layout in repr
(GH#7218) Ian Rose
## Enhancements
* Support MIG parsing during CUDA context creation in UCX
initialization (GH#6720) Peter Andreas Entschev
* Handle /metrics endpoint without prometheus-client installed
(GH#7234) Hendrik Makait
* Add support for unpacking namedtuples in remote data (GH#7282)
Andrew
* Enable queuing by default (GH#7279) Florian Jetter
* Fix exists->``exist`` typo in scheduler error messages
(GH#7281) Matthew Plough
* If there’s an exception in the Client async context manager
body then close fast (GH#6920) Thomas Grainger
## Bug Fixes
* Ensure category is optional when logging "warn" events
(GH#7169) James Bourbeau
* Edge and impossible transitions to memory (GH#7205) crusaderky
## Maintenance
* Allow bokeh=3 (GH#5648) James Bourbeau
* Fix typos in P2P shuffle code (GH#7304) Hendrik Makait
* Reenable test_bad_disk (GH#7300) Florian Jetter
* Reduce max-runs in test reports (GH#7299) Florian Jetter
* Revert idle classification when worker-saturation is set
(GH#7278) Florian Jetter
* Fix flaky deadline_expiration (GH#7287) Florian Jetter
* Rewrite of P2P control flow (GH#7268) Florian Jetter
* Bump minimum bokeh version to 2.4.2 (GH#7271) James Bourbeau
* Remove deprecated code calls to IOLoop.make_current() (GH#7240)
Thomas Grainger
* Improved test for balancing expensive tasks (GH#7272) Hendrik
Makait
* Refactor semaphore._Watch into general-purpose Deadline utility
(GH#7238) Hendrik Makait
* Require Click 7.0+ (GH#7226) jakirkham
* Drop tests (GH#7269) Hendrik Makait
* Replace test_(do_not_)steal_communication_heavy_tasks tests
with more robust versions (GH#7243) Hendrik Makait
* xfail test_bad_disk (GH#7265) crusaderky
* Move transition_log from Scheduler to SchedulerState (GH#7254)
crusaderky
* Remove Scheduler.log (GH#7258) crusaderky
* Use latest pickle (GH#5826) jakirkham
* Polish parsing of worker-saturation from config (GH#7255)
crusaderky
* Avoid expensive occupancy calculation when unused (GH#7257)
Gabe Joseph
* Un-skip test_nested_compute (GH#7247) Gabe Joseph
* Review test_do_not_steal_communication_heavy_tasks (GH#7250)
crusaderky
* Fix test_stress_creation_and_deletion (GH#7215) crusaderky
* Raise exceptions in Server.handle_stream instead of
swallowing/logging (GH#7162) Hendrik Makait
* Fix _update_scheduler_info hanging failed tests (GH#7225) Gabe
Joseph
* Bump xarray-contrib/ci-trigger from 1.1 to 1.2 (GH#7232)
- Release 2022.10.2
* Reverted a bug where Bokeh was accidentally made non-optional
(GH#7230) Oliver Holworthy
* Schedule a queued task when a task secedes (GH#7224) Gabe
Joseph
* This was a hotfix release
- Release 2022.10.1
## New Features
* Add Client.restart_workers method (GH#7154) James Bourbeau
* Implement PackageInstall plugin for pip and conda (GH#7126)
Hendrik Makait
## Enhancements
* Add prometheus collector for work-stealing (GH#7206) Hendrik
Makait
* Track reason of workers closing and restarting (GH#7166)
Hendrik Makait
* Show no-worker on task progress bar (GH#7171) Florian Jetter
* Set OPENBLAS_NUM_THREADS by default (GH#7177) James Bourbeau
* Optionally provide local directory to data constructor
(GH#7153) Lawrence Mitchell
* Introduce distributed.comm.ucx.environment config slot
(GH#7164) Lawrence Mitchell
* Log information about memory limit (GH#7160) Florian Jetter
* Improve log messages on scheduler for restart (GH#7150) Florian
Jetter
* More comprehensive WorkerState task counters (GH#7167)
crusaderky
## Bug Fixes
* Update minimum bokeh version message (GH#7172) James Bourbeau
* Revamped implementations of remote print() and warn(), fixing
#7095 (GH#7129) Max Bane
## Maintenance
* Temporarily restrict bokeh<3 (GH#7219) James Bourbeau
* Make Scheduler.reschedule private (GH#7216) crusaderky
* Fix decide_worker_rootish_queuing_disabled assert (GH#7065)
Gabe Joseph
* Fix flaky test_include_communication_in_occupancy (GH#7212)
Gabe Joseph
* Do not raise on leaked websockets (GH#7199) Florian Jetter
* Update nightly recipes with CLI tests, dependency changes
(GH#7201) Charles Blackmon-Luca
* Make p2p shuffle submodules private (GH#7186) Florian Jetter
* Backport tornado PeriodicCallback (GH#7165) Florian Jetter
* Fix mypy failure on CI (GH#7198) Florian Jetter
* User a layer for p2p shuffle (GH#7180) Florian Jetter
* Type annotations for shuffle (GH#7185) Florian Jetter
* Do not close worker on comm error in heartbeat (GH#7163)
Hendrik Makait
* Errors when setting TCP timeouts log as error (GH#7161) Florian
Jetter
* Remove incorrect advice from pre-commit config (GH#7159)
crusaderky
* Use functionalities network for codecov uploader (GH#7148)
Florian Jetter
* Use counter metric type where appropriate, incoming_count was
reporting bytes (GH#7125) Nat Tabris
- Add allow-bokeh3.patch -- raise upper bokeh version, reverts part
of gh#dask/distributed#7329, see also gh#dask/dask#9659
- Replace support-tornado-6-2.patch with
distributed-pr7286-tornado-6-2.patch -- gh#dask/distributed#7286
OBS-URL: https://build.opensuse.org/request/show/1036962
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=124
* Display unexpected state in Worker.execute validation (:pr:`6856`)
James Bourbeau
* cancelled / resumed -> rescheduled transition (:pr:`6913`) crusaderky
* Fix resource deallocation for resumed tasks (:pr:`6914`) crusaderky
* Only close scheduler in SpecCluster if it exists (:pr:`6888`)
Matthew Rocklin
* Always close BatchedSend write coroutines (:pr:`6865`) Gabe Joseph
* Harden preamble of Worker.execute against race conditions (:pr:`6878`)
crusaderky
* Duplicated code: in_flight_tasks validation (:pr:`6917`) crusaderky
* Overhaul transitions for the resumed state (:pr:`6699`) crusaderky
* Clean up cluster process reaping (:pr:`6840`) Gabe Joseph
* Don't use bokeh Figure in tests (:pr:`6721`) Bryan Van de Ven
* Work around incompatibility of crick with setuptools 65 (:pr:`6887`)
crusaderky
* Replace port = random.randint( with port = d.utils.open_port (:pr:`6883`)
Thomas Grainger
* Fix flaky test_wall_clock (:pr:`6879`) crusaderky
* Add descriptive error message to assert (:pr:`6871`) Hendrik Makait
* Increase timeout in test_quiet_process (:pr:`6857`) Florian Jetter
* Remove redundant use of with clean(): (:pr:`6852`) Thomas Grainger
* Update pre-commit dependencies (:pr:`6851`) Hendrik Makait
* Call exit callback even if AsyncProcess is reaped elsewhere (:pr:`6684`)
Thomas Grainger
* Avoid function calls in argument defaults (:pr:`6812`) Hendrik Makait
* Add Cluster.get_client() method (:pr:`6745`) Julia Signell
* Start bokeh app to activate bokeh's clean session callbacks (:pr:`6728`)
Martí Zamora
* Don't connect to cluster subprocesses at shutdown (:pr:`6829`) Gabe Joseph
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=115
- Update to 2022.6.1
* Highlights
- This release includes the Worker State Machine refactor. The
expectation should be that the worker state is its own
synchronous subclass. Pulling all the state out into its own
class allows us to write targeted unit tests without invoking
any concurrent or asynchronous code.
* Enhancements
- Make worker state machine methods private (GH#6564)
crusaderky
- Yank state machine out of Worker class (GH#6566) crusaderky
- Track worker_state_machine.TaskState instances (GH#6525)
Hendrik Makait
- Trivial tweaks to the Worker State Machine (GH#6586)
crusaderky
- Replace loop.call_later and loop.add_callback with background
tasks added to Server. (GH#6603) Thomas Grainger
- Support for neater WorkerState tests (GH#6609) crusaderky
- Limit TCP writes with Tornado to 2GB (GH#6557) hhuuggoo
- Enable no_implicit_optional for scheduler (GH#6622) Thomas
Grainger
* Bug Fixes
- Partial revert of compute-task message format (GH#6626)
Florian Jetter
- Restore log message about received signals in CLI (GH#6618)
Florian Jetter
- Handle empty memoryviews of bytearrays when (de)serializing
(GH#6576) Benjamin Zaitlen
- Ensure steal requests from same-IP but distinct workers are
rejected (GH#6585) Florian Jetter
- Fix tls_(min|max)_ version having no effect on openssl 1.1.0g
or lower (GH#6562) Thomas Grainger
- Fix idle_timeout and unxfail test (GH#6563) Matthew Rocklin
- Fix crashing debug statement in _purge_state (GH#6589)
crusaderky
- Abort connections on CancelledError (GH#6574) Thomas Grainger
- Fix Active Memory Manager ignores nbytes thresholds (GH#6583)
crusaderky
* Deprecations
- Deprecate WorkerState accessors (GH#6579)
crusaderky
- Release 2022.6.0
* Enhancements
- Make disk access in system monitor configurable (GH#6537)
Matthew Rocklin
- Log and except errors on preload start (GH#6553) Matthew
Rocklin
- Fix Scheduler.restart logic (GH#6504) Gabe Joseph
- Don’t heartbeat while Worker is closing (GH#6543) Gabe Joseph
- No longer retry LocalCluster on errno.EADDRINUSE (GH#6369)
Thomas Grainger
- Don’t invoke log_event from state machine (GH#6512)
crusaderky
- Add config option to disable profiling and disable it in many
tests per default (GH#6490) Hendrik Makait
- Encapsulate Worker.batched_stream.send() (GH#6475) crusaderky
* Bug Fixes
- refresh-who-has can break the worker state machine (GH#6529)
crusaderky
- Restart worker if it’s unrecognized by scheduler (GH#6505)
Gabe Joseph
- Fix import error when distributed.rmm.pool-size is set
(GH#6482) KoyamaSohei
* Deprecations
- Restore signature compatibility for dask-gateway (GH#6561)
Tom Augspurger
- Deprecate the io_loop and loop kwarg to Server, Worker, and
Nanny (GH#6473) Thomas Grainger
- Deprecate the loop kwarg to Scheduler (GH#6443) Thomas
Grainger
- Release 2022.05.2
* Enhancements
- Add a lock to distributed.profile for better concurrency
control (GH#6421) Hendrik Makait
- Send SIGKILL after SIGTERM when passing 95% memory (GH#6419)
crusaderky
* Bug Fixes
- Log rather than raise exceptions in preload.teardown()
(GH#6458) Matthew Rocklin
- Handle failing plugin.close() calls during scheduler shutdown
(GH#6450) Matthew Rocklin
- Fix slicing bug in ensure_memoryview (GH#6449) jakirkham
- Generalize UCX errors on connect() and correct pytest
fixtures (GH#6434) Peter Andreas Entschev
- Run cluster widget periodic callbacks on the correct event
loop (GH#6444) Thomas Grainger
- Release 2022.05.1
* New Features
- Add HTTP API to scheduler (GH#6270) Matthew Murray
- Shuffle Service with Scheduler Logic (GH#6007) Matthew
Rocklin
* Enhancements
- Follow-up on removing report and safe from Worker.close
(GH#6423) Gabe Joseph
- Server close faster (GH#6415) Florian Jetter
- Disable HTTP API by default (GH#6420) Jacob Tomlinson
- Remove report and safe from Worker.close (GH#6363) Florian
Jetter
- Allow deserialized plugins in register_scheduler_plugin
(GH#6401) Matthew Rocklin
- WorkerState are different for different addresses (GH#6398)
Florian Jetter
- Do not filter tasks before gathering data (GH#6371)
crusaderky
- Remove worker reconnect (GH#6361) Gabe Joseph
- Add SchedulerPlugin.log_event handler (GH#6381) Matthew
Rocklin
- Ensure occupancy tracking works as expected for long running
tasks (GH#6351) Florian Jetter
- stimulus_id for all Instructions (GH#6347) crusaderky
- Refactor missing-data command (GH#6332) crusaderky
- Add idempotent to register_scheduler_plugin client (GH#6328)
Alex Ford
- Add option to specify a scheduler address for workers to use
(GH#5944) Enric Tejedor
* Bug Fixes
- Remove stray breakpoint (GH#6417) Thomas Grainger
- Fix API JSON MIME type (GH#6397) Jacob Tomlinson
- Remove wrong assert in handle compute (GH#6370) Florian
Jetter
- Ensure multiple clients can cancel their key without
interference (GH#6016) Florian Jetter
- Fix Nanny shutdown assertion (GH#6357) Gabe Joseph
- Fix fail_hard for sync functions (GH#6269) Gabe Joseph
- Prevent infinite transition loops; more aggressive
validate_state() (GH#6318) crusaderky
- Ensure cleanup of many GBs of spilled data on terminate
(GH#6280) crusaderky
- Fix WORKER_ANY_RUNNING regression (GH#6297) Florian Jetter
- Race conditions from fetch to compute while AMM requests
replica (GH#6248) Florian Jetter
- Ensure resumed tasks are not accidentally forgotten (GH#6217)
Florian Jetter
- Do not allow closing workers to be awaited again (GH#5910)
Florian Jetter
* Deprecations
- Move wait_for_signals to private module and deprecate
distributed.cli.utils (GH#6367) Hendrik Makait
- Release 2022.05.0
* Highlights
- This is a bugfix release for this issue.
* Enhancements
- Handle writeable in buffer_callback (GH#6238) jakirkham
- Use .data with NumPy array allocation (GH#6242) jakirkham
* Bug Fixes
- Close executor in event loop if interpreter is closing
(GH#6256) Matthew Rocklin
- Release 2022.04.2
* Enhancements
- Unblock event loop while waiting for ThreadpoolExecutor to
shut down (GH#6091) Florian Jetter
- RetireWorker policy is done if removed (GH#6234) Gabe Joseph
- Pause to disable dependency gathering (GH#6195) crusaderky
- Add EOFError to nanny multiprocessing.queue except list
(GH#6213) Matthew Rocklin
- Re-interpret error in lost worker scenario (GH#6193) Matthew
Rocklin
- Add Stimulus IDs to Scheduler (GH#6161) Florian Jetter
- Set a five minute TTL for Dask workers (GH#6200) Matthew
Rocklin
- Add distributed.metrics.monotonic (GH#6181) crusaderky
- Send worker validation errors to scheduler and err on test
completion (GH#6192) Matthew Rocklin
- Redesign worker exponential backoff on busy-gather (GH#6173)
crusaderky
- Log all invalid worker transitions to scheduler (GH#6134)
Matthew Rocklin
- Make Graph dashboard plot have invisible axes (GH#6149)
Matthew Rocklin
- Remove Nanny auto_restart state (GH#6138) Matthew Rocklin
* Bug Fixes
- Ensure scheduler events do not hold on to TaskState objects
(GH#6226) Florian Jetter
- Allow pausing and choke event loop while spilling (GH#6189)
crusaderky
- Do not use UUID in stealing (GH#6179) Florian Jetter
- Handle int worker names in info page (GH#6158) Brett Naul
- Fix psutil dio counters none (GH#6093) ungarj
- Join Nanny watch thread (GH#6146) Matthew Rocklin
- Improve logging when closing workers (GH#6129) Matthew
Rocklin
- Avoid stack overflow in profiling (GH#6141) Matthew Rocklin
- Clean up SSHCluster if failure to start (GH#6130) Matthew
Rocklin
* Deprecations
- Deprecate rpc synchronous context manager (GH#6171) Thomas
Grainger
- Release 2022.04.1
* New Features
- Add KillWorker Plugin (GH#6126) Matthew Rocklin
* Enhancements
- Sort worker list in info pages (GH#6135) Matthew Rocklin
- Add back Worker.transition_fetch_missing (GH#6112) Matthew
Rocklin
- Log state machine events (GH#6092) crusaderky
- Migrate ensure_executing transitions to new WorkerState event
mechanism - part 1 (GH#6003) crusaderky
- Migrate ensure_executing transitions to new WorkerState event
mechanism - part 2 (GH#6062) crusaderky
- Annotate worker transitions to error (GH#6012) crusaderky
* Bug Fixes
- Avoid transitioning from memory/released to missing in worker
(GH#6123) Matthew Rocklin
- Don’t try to reconnect client on interpreter shutdown
(GH#6120) Matthew Rocklin
- Wrap UCX init warnings in importable functions (GH#6121)
Charles Blackmon-Luca
- Cancel asyncio tasks on worker close (GH#6098) crusaderky
- Avoid port collisions when defining port ranges (GH#6054)
crusaderky
- Release 2022.04.0
* This is the first release with support for Python 3.10
* New Features
- Add Python 3.10 support (GH#5952) Thomas Grainger
- New cluster dump utilities (GH#5920) Simon Perkins
- New ClusterDump SchedulerPlugin for dumping cluster state on
close (GH#5983) Simon Perkins
- Track Event Loop intervals in dashboard plot (GH#5964)
Matthew Rocklin
- ToPickle - Unpickle on the Scheduler (GH#5728) Mads R. B.
Kristensen
* Enhancements
- Retry on transient error codes in preload (GH#5982) Matthew
Rocklin
- More idiomatic mypy configuration (GH#6022) crusaderky
- Name extensions and enable extension heartbeats (GH#5957)
Matthew Rocklin
- Better error message on misspelled executor annotation
(GH#6009) crusaderky
- Clarify that SchedulerPlugin must be subclassed (GH#6008)
crusaderky
- Remove duplication from stealing (GH#5787) Duncan McGregor
- Remove cache in iscoroutinefunction to avoid holding on to
refs (GH#5985) Florian Jetter
- Add title to individual plots (GH#5967) Matthew Rocklin
- Specify average in timeseries titles (GH#5974) Matthew
Rocklin
* Bug Fixes
- Do not catch CancelledError in CommPool (GH#6005) Florian
Jetter
* Deprecations
- Remove distributed._ipython_utils and dependents (GH#6036)
Thomas Grainger
- Remove support for PyPy (GH#6029) James Bourbeau
- Drop runtime dependency to setuptools (GH#6017) crusaderky
- Remove heartbeats from events (GH#5989) Matthew Rocklin
- Cythonization has been abandoned upstream: Back to noarch.
- Add distributed-ignore-offline.patch
- Drop distributed-pr5952-py310.patch
OBS-URL: https://build.opensuse.org/request/show/985664
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=111
- Update to 2022.02.1:
Add the ability for Client to run preload code
Optionally use NumPy to allocate buffers
Add git hash to distributed-impl version
Immediately raise exception when trying to connect to a closed cluster
Lazily get dask version information
Remove the requirements to add comm to every handler
Raise on unclosed comms in check_instances
Constrained spill
Remove redundant str() conversions
Cluster dump now excludes run_spec by default
Dump more objects with dump_cluster_state
Do not connect to any sockets on import
Avoid deadlock when two tasks are concurrently waiting for an unresolved ActorFuture
Drop Python 3.7
Remove support for UCX < 1.11.1
Document and test spill->target hysteresis cycle
Fix flaky test_remove_replicas_while_computing
Fix time based test_assert_worker_story_malformed_story parameterize
Remove xfail from test_no_unnecessary_imports_on_worker
Start building pre-releases with cythonized scheduler
Do not mark tests xfail if they don't come up in time
Use gen_cluster where possible in test_dask_worker.py
Generate junit report when pytest-timeout kills pytest
Decrease timeout-minutes for GHA jobs
Bump pre-release version to be greater than stable releases
Do not run schedule jobs on forks
Remove pillow<9 pin in CI
Show scheduled test runs in report
Add obvious exclusions with pragma statement
OBS-URL: https://build.opensuse.org/request/show/960175
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=106
- Update to version 2022.1.1
* P2P shuffle skeleton
* Fix <Task pending name='...' coro=<Client._handle_report()>
* Add distributed.client.security-loader config
* Avoid Client._handle_report cancelling itself on Client._close
* Paused workers shouldn't steal tasks
* Add option for timestamps from output of Node.get_logs
* Don't use time.time() or IOLoop.time()
* Raise plugin exceptions on Worker.start()
* Fixing docstrings
* Fix typo in Client.run docstring
* Update client.py docstrings
* Skip shuffle tests if pandas / dask.dataframe not installed
* Improve test coverage
* Test report improvements
* P2P shuffle: ignore row order in tests
* Fix flaky test_no_reconnect[--no-nanny]
* Pin coverage to 6.2
* Check for new name of timeouts artifact and be more fault
tolerant
* Revisit rebalance unit tests
* Update comment in rearrange_by_column_p2p
* Update gpuCI RAPIDS_VER to 22.04
* Fix groupby test after meta requirements got stricter in Dask
PR#8563
* Fix flaky test_close_gracefully and test_lifetime
* Fix flaky test_workspace_concurrency
* Fix flaky test_shuffle_extension.py::test_get_partition
* Fix flaky test_dump_cluster_unresponsive_remote_worker
* Dump cluster state on all test failures
* Update license format
* Fix flaky test_drop_with_paused_workers_with_running_tasks_3_4
* Do not raise an exception if the GitHub token cannot be found
(Resubmit for the noclean flag)
OBS-URL: https://build.opensuse.org/request/show/950288
OBS-URL: https://build.opensuse.org/package/show/devel:languages:python:numeric/python-distributed?expand=0&rev=104
